DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Currently pending Claim(s):
Amended claim(s):
Canceled claim(s):
New Claim(s):
1-2 and 4-21
1, 7, 11, 17, and 20
3
21
Response to Arguments
This office action is responsive to Applicant's Arguments/Remarks made in an Amendment received on 06/08/2026.
In view of the new claim amendments and applicant arguments, [Remarks] filed on 06/08/2026, with respect to the 35 U.S.C. 101 claim rejections have been carefully considered and the claims rejections to claims 1-20 under 35 U.S.C. 101 are withdrawn.
Applicant's Reply (06/08/2026) includes substantive amendments to the claims. This Office action has been updated with new grounds of rejection addressing those amendments. Further Applicant's Arguments/Remarks with respect the pending claims have been considered but are moot because the arguments do not apply to any of the references being used in the current rejection and the arguments are now rejected by newly cited art ‘Kosyanchuk (US 2024/0242616 A1)’ as explained in the body of rejection below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4, 6-8, 10-12, 14, 16-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kosyanchuk (US 2024/0242616 A1) in view of Chen et al. (US 2022/0234752 A1) (hereinafter, “Chen”); and further in view of Evans et al. (US 2022/0198703 A1) (“hereinafter, “Evans”).
Regarding claim 1, Kosyanchuk discloses a computing system comprising (Paragraph [0027] "The aircraft 100 may be in communication with one or more exterior systems (not depicted). As depicted, the aircraft 100 may include the flight displays 104, aircraft sensors 203, processors 204, memory 210, and the like."):
at least one input sensor comprising one or a plurality of cameras located onboard an aircraft (Paragraph [0073] "The EVS camera may be any type of imaging sensors, such as, but not limited to, a color camera, an infrared camera (e.g., a forward looking infrared (FLIR) camera), and/or a radar. The EVS camera may be mounted to a surface of the aircraft");
processing circuitry (Paragraph [0027] "The aircraft 100 may be in communication with one or more exterior systems (not depicted). As depicted, the aircraft 100 may include the flight displays 104, aircraft sensors 203, processors 204, memory 210, and the like."); and
a memory storing a runway database and executable instructions that, in response to execution by the processing circuitry, cause the processing circuitry to (Paragraph [0032] "The memory 210 may include a runway database 302, traffic data 304, flight plan 306, expected glideslope 308, program instructions 310, and the like."):
collect a plurality of images related to at least an environment from the one or the plurality of cameras (Paragraph [0073] "The EVS camera may capture images extending forward from the aircraft. The images may include one or more visual cues of the airport. For example, the EVS camera may capture a portion of the runway, taxiways, and the like. The EVS cameras may be located to capture a front-view from the aircraft 100. The image 800 may include a field-of-view of the airport.");
[execute a feature extractor to extract features for the plurality of images];
determine a plurality of candidate runways (the plurality of runways in Paragraph [0003] equates to a plurality of candidate runways) among known runways in the runway database (Paragraph [0003] “the program instructions cause the one or more processors to select an expected landing runway from the plurality of runways based on the plurality of track alignments, the plurality of vector alignments, the plurality of vertical alignments, the plurality of in-approach geometry parameters, and the plurality of in-flight plan parameters”; Paragraph [0032] "the runway database 302 may include one or more of a runway 312, a bearing 314, a threshold 316, and the like.") at least based on the [extracted] features [by matching the extracted features with known runway and runway-associated features] of the plurality of candidate runways (multiple of the runways 312 in Paragraph [0077] equate to plurality of candidate runways) in the runway database (Paragraph [0076] "The processors 204 may also detect which runway is in the image based on the bearing 314 printed on the runway 312. The parameters 336 of the weighting algorithm 334 may then include locations of the runways 312 detected by the bounding box 802."; Paragraph [0077] "the processors 204 may apply hysteresis before determining the overrun probability from the expected landing runway 354. The hysteresis may be applied to reduce processing requirements stemming from changes in the expected landing runway 354. For example, the expected landing runway 354 may change between multiple of the runways 312 within a time of a few seconds.");
generate, as a runway identification (expected landing runway 354 in Paragraph [0077] equates to a runway identification), one candidate runway among the plurality of candidate runways [based on the matching] (Paragraph [0076] "the location of the runways 312 may indicate the aircraft is or is not aligned with the runway. In this regard, the processors 204 may select the expected landing runway from the runways based on the one or more runways detected in the image 800."; Paragraph [0077] “the expected landing runway 354 may change between multiple of the runways 312 within a time of a few seconds."; ; Examiner interprets multiple of the runways 312, included in the runway database 302, are used to determine the expected landing runway i.e. runway identification); and
output the runway identification (Paragraph [0044] “the expected landing runway 354 may be provided to the runway overrun alert and awareness system 212 for determining the runway overrun for the expected landing runway 354. The runway overrun alert and awareness system 212 may determine the runway overrun in response to the processors 204 selecting the expected landing runway 354 from the plurality of runways 312”).
However, Kosyanchuk fails to teach execute a feature extractor to extract features for the plurality of images and by matching the extracted features with known runway and runway-associated features.
Chen teaches execute a feature extractor to extract features for the plurality of images (Paragraph [0026] “The data 52 may be three or four corners of the runway in some embodiments but other trackable features of the runway may be used provided the location of corresponding features can be extracted by the computer vision processing module 26.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk’s reference to include execute a feature extractor to extract features for the plurality of images taught by Chen’s reference. The motivation for doing so would have been to determine the position of the camera and the real world aircraft position as suggested by Chen (see Chen, Paragraph [0023]).
However, Kosyanchuk and Chen both fail to teach by matching the extracted features with known runway and runway-associated features.
Evans teaches by matching the extracted features with known runway and runway-associated features (Paragraph [0024] “applying the mask to a corner detector to detect interest points on the mask and thereby the runway or the runway marking in the image; matching the interest points on the runway or the runway marking in the image, to corresponding points on the runway or the runway marking that have known runway-framed local coordinates; and performing a perspective-n-point (PnP) estimation, using the interest points and the known runway-framed local coordinates”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk in view of Chen to include by matching the extracted features with known runway and runway-associated features taught by Evans reference. The motivation for doing so would have been to determine the position of an aircraft relative to the runway as suggested by Evans (see Evans, Paragraph [0024]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Evan with Kosyanchuk and Chen to obtain the invention specified in claim 1.
Regarding claim 2, which claim 1 is incorporated, fails to teach Kosyanchuk wherein the feature extractor is a machine perception system.
Chen teaches wherein the feature extractor is a machine perception system (Paragraph [0022] “Numerous runway detection algorithms could be constructed… a machine learning classifier is used that is trained with images including a runway. One example classifier is the Cascade Classifier…object finding function returns a positive or negative indication concerning whether a runway has been found and returns a region of interest (ROI) outline (e.g. a rectangle) encompassing the identified runway.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk’s reference to include wherein the feature extractor is a machine perception system taught by Chen’s reference. The motivation for doing so would have been to determine the position of the camera and the real world aircraft position as suggested by Chen (see Chen, Paragraph [0023]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Chen with Kosyanchuk and Evans to obtain the invention specified in claim 2.
Regarding claim 4, which claim 1 is incorporated, Kosyanchuk and Chen both fail to teach wherein the machine perception system is a convolutional neural network..
Evans teaches wherein the machine perception system is a convolutional neural network (Paragraph [0067] “The machine learning model may include a CNN, as well as a fully-connected dense network and an n-DOF regression, such as a 6DOF regression or a 2DOF regression.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk in view of Chen to include wherein the machine perception system is a convolutional neural network taught by Evans’ reference. The motivation for doing so would have been to process full-resolution images as suggested by Evans (see Evans, Paragraphs [0067]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Evans with Kosyanchuk and Chen to obtain the invention specified in claim 4.
Regarding claim 6, which claim 1 is incorporated, Kosyanchuk and Chen both fail to teach wherein the extracted features are estimated locations of interest points of runways based on probability estimations at geographic locations.
Evans teaches wherein the extracted features are estimated locations of interest points of runways based on probability estimations at geographic locations (Paragraph [0090] …runway 202 may be assumed to be a rectangular, level plane at unknown runway-framed local coordinates (x, 0, z), and an unknown orientation along the z/x axis. The runway may also be assumed to have an unknown width. The relation between real world position and rotation and projected representation, then, may be described with… (a, b)=pixel position (x, y); (c, d)=screen center (x, y); o=scaling factor; (p, q, r)=rotation angles around (x, y, z); x=reference point world runway-framed local coordinate x; and z=runway-framed local coordinate z. When reducing roll angle to zero (rotation of the input image to level the horizon), it can be shown that only two points (e.g. two threshold center points) may be needed for a single solution for the lateral and (if the runway length is known) the vertical angular deviation”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk in view of Chen to include wherein the extracted features are estimated locations of interest points of runways based on probability estimations at geographic locations taught by Evans’ reference. The motivation for doing so would have been to determine the position of an aircraft relative to the runway as suggested by Evans (see Evans, Paragraph [0024]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Evans with Kosyanchuk and Chen to obtain the invention specified in claim 6.
Regarding claim 7, which claim 6 is incorporated, Kosyanchuk and Chen both fail to teach wherein the runway identification is generated by matching interest points or features in the extracted features with interest points or features of the known runways in the runway database. Evans teaches wherein the runway identification is generated by matching interest points or features in the extracted features with interest points or features of the known runways in the runway database (Paragraph [0024] “applying the mask to a corner detector to detect interest points on the mask and thereby the runway or the runway marking in the image; matching the interest points on the runway or the runway marking in the image, to corresponding points on the runway or the runway marking that have known runway-framed local coordinates; and performing a perspective-n-point (PnP) estimation, using the interest points and the known runway-framed local coordinates”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk in view of Chen to include wherein the runway identification is generated by matching interest points or features in the extracted features with interest points or features of the known runways in the runway database taught by Evans’ reference. The motivation for doing so would have been to determine the position of an aircraft relative to the runway as suggested by Evans (see Evans, Paragraph [0024]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Evans with Kosyanchuk and Chen to obtain the invention specified in claim 7.
Regarding claim 8, which claim 7 is incorporated, Kosyanchuk discloses wherein the interest points or features in the runway database correspond to at least one of threshold markings, aiming point markings, designation markings, side stripes, or thresholds of registered runways or runway-associated features (Paragraph [0033] " the runway database 302 may include one or more of a runway 312, a bearing 314, a threshold 316, and the like.").
Regarding claim 10, which claim 1 is incorporated, Kosyanchuk discloses [wherein the at least one input sensor comprises at least an altimeter or a magnetometer;
sensor data from the at least the altimeter or the magnetometer is used to generate localization data; and]
the localization data is taken into account when generating the runway identification (Paragraph [0045] "the weighting algorithm 334 may select the expected landing runway 354 from the runways 312 based on the track alignments 338, the vector alignments 340, the vertical alignments 342, the in-approach geometry 344 parameters, and the in-flight plan 346 parameters for each of the runways 312. ").
However, Kosyanchuk fails to teach wherein the at least one input sensor comprises at least an altimeter or a magnetometer; sensor data from the at least the altimeter or the magnetometer is used to generate localization data.
Chen teaches wherein the at least one input sensor comprises at least an altimeter or a magnetometer (Paragraph [0043] “when the aircraft altitude is less than 1000 feet. The aircraft altitude may be determined based on altitude data from a radio altimeter included in the aircraft sensors 24.”);
sensor data from the at least the altimeter or the magnetometer is used to generate localization data (Paragraph [0043] “The aircraft position deviation 64 can then be determined to allow a go-around decision to be output when the deviation is too great.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk’s reference to include wherein the at least one input sensor comprises at least an altimeter or a magnetometer; sensor data from the at least the altimeter or the magnetometer is used to generate localization data taught by Chen’s reference. The motivation for doing so would have been to provide a go-around or continue approach decision as suggested by Chen (see Chen, Paragraph [0043]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Chen with Kosyanchuk and Evans to obtain the invention specified in claim 10.
Regarding claim 11, Kosyanchuk discloses a computing method comprising: collecting a plurality of images related to at least an environment from one or a plurality of cameras located onboard an aircraft (Paragraph [0073] "The EVS camera may be mounted to a surface of the aircraft. The EVS camera may capture images extending forward from the aircraft…the EVS camera may capture a portion of the runway, taxiways, and the like. The EVS cameras may be located to capture a front-view from the aircraft 100. The image 800 may include a field-of-view of the airport. The image may include one or more runways, taxiways, and the like of the airport.");
[executing a feature extractor to extract features for the plurality of images];
determining a plurality of candidate runways (the plurality of runways in Paragraph [0003] equates to a plurality of candidate runways) among known runways in a runway database (Paragraph [0032] "the runway database 302 may include one or more of a runway 312, a bearing 314, a threshold 316, and the like.") at least based on the [extracted] features [by matching the extracted features with known runway and runway-associated features] of the plurality of candidate runways (multiple of the runways 312 in Paragraph [0077] equate to plurality of candidate runways) in the runway database (Paragraph [0076] "The processors 204 may also detect which runway is in the image based on the bearing 314 printed on the runway 312. The parameters 336 of the weighting algorithm 334 may then include locations of the runways 312 detected by the bounding box 802."; Paragraph [0077] "the processors 204 may apply hysteresis before determining the overrun probability from the expected landing runway 354. The hysteresis may be applied to reduce processing requirements stemming from changes in the expected landing runway 354. For example, the expected landing runway 354 may change between multiple of the runways 312 within a time of a few seconds.");
generating, as a runway identification (expected landing runway 354 in Paragraph [0077] equates to a runway identification), one candidate runway among the plurality of candidate runways [based on the matching] (Paragraph [0076] "the location of the runways 312 may indicate the aircraft is or is not aligned with the runway. In this regard, the processors 204 may select the expected landing runway from the runways based on the one or more runways detected in the image 800."; Paragraph [0077] “the expected landing runway 354 may change between multiple of the runways 312 within a time of a few seconds."); and
outputting the runway identification (Paragraph [0044] “the expected landing runway 354 may be provided to the runway overrun alert and awareness system 212 for determining the runway overrun for the expected landing runway 354. The runway overrun alert and awareness system 212 may determine the runway overrun in response to the processors 204 selecting the expected landing runway 354 from the plurality of runways 312”).
However, Kosyanchuk fails to teach executing a feature extractor to extract features for the plurality of images and by matching the extracted features with known runway and runway-associated features.
Chen teaches executing a feature extractor to extract features for the plurality of images (Paragraph [0026] “The data 52 may be three or four corners of the runway in some embodiments but other trackable features of the runway may be used provided the location of corresponding features can be extracted by the computer vision processing module 26.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk’s reference to include executing a feature extractor to extract features for the plurality of images taught by Chen’s reference. The motivation for doing so would have been to determine the position of the camera and the real world aircraft position as suggested by Chen (see Chen, Paragraph [0023]).
However, Kosyanchuk and Chen both fail to teach by matching the extracted features with known runway and runway-associated features.
Evans teaches by matching the extracted features with known runway and runway-associated features (Paragraph [0024] “applying the mask to a corner detector to detect interest points on the mask and thereby the runway or the runway marking in the image; matching the interest points on the runway or the runway marking in the image, to corresponding points on the runway or the runway marking that have known runway-framed local coordinates; and performing a perspective-n-point (PnP) estimation, using the interest points and the known runway-framed local coordinates”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk in view of Chen to include by matching the extracted features with known runway and runway-associated features taught by Evans reference. The motivation for doing so would have been to determine the position of an aircraft relative to the runway as suggested by Evans (see Evans, Paragraph [0024]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Evan with Kosyanchuk and Chen to obtain the invention specified in claim 11.
Regarding claim 12 (drawn to a method), claim 12 is rejected the same as claim 2 and the arguments similar to that presented above for claim 2 are equally applicable to the claim 12, and all the other limitations similar to claim 2 are not repeated herein, but incorporated by reference.
Regarding claim 14 (drawn to a method), claim 14 is rejected the same as claim 4 and the arguments similar to that presented above for claim 4 are equally applicable to the claim 14, and all the other limitations similar to claim 4 are not repeated herein, but incorporated by reference.
Regarding claim 16 (drawn to a method), claim 16 is rejected the same as claim 6 and the arguments similar to that presented above for claim 6 are equally applicable to the claim 16, and all the other limitations similar to claim 6 are not repeated herein, but incorporated by reference.
Regarding claim 17 (drawn to a method), claim 17 is rejected the same as claim 7 and the arguments similar to that presented above for claim 7 are equally applicable to the claim 17, and all the other limitations similar to claim 7 are not repeated herein, but incorporated by reference.
Regarding claim 18 (drawn to a method), claim 18 is rejected the same as claim 8 and the arguments similar to that presented above for claim 8 are equally applicable to the claim 18, and all the other limitations similar to claim 8 are not repeated herein, but incorporated by reference.
Regarding claim 20, Kosyanchuk discloses a computing system comprising (Paragraph [0027] "The aircraft 100 may be in communication with one or more exterior systems (not depicted). As depicted, the aircraft 100 may include the flight displays 104, aircraft sensors 203, processors 204, memory 210, and the like."):
at least one input sensor comprising one or a plurality of cameras located onboard an aircraft (Paragraph [0073] "The EVS camera may be any type of imaging sensors, such as, but not limited to, a color camera, an infrared camera (e.g., a forward looking infrared (FLIR) camera), and/or a radar. The EVS camera may be mounted to a surface of the aircraft");
processing circuitry (Paragraph [0027] "The aircraft 100 may be in communication with one or more exterior systems (not depicted). As depicted, the aircraft 100 may include the flight displays 104, aircraft sensors 203, processors 204, memory 210, and the like."); and
a memory storing a runway database and executable instructions that, in response to execution by the processing circuitry cause the processing circuitry to (Paragraph [0032] "The memory 210 may include a runway database 302, traffic data 304, flight plan 306, expected glideslope 308, program instructions 310, and the like."):
collect a plurality of images related to at least a portion of an environment from the one or the plurality of cameras (Paragraph [0073] "The EVS camera may capture images extending forward from the aircraft. The images may include one or more visual cues of the airport. For example, the EVS camera may capture a portion of the runway, taxiways, and the like. The EVS cameras may be located to capture a front-view from the aircraft 100. The image 800 may include a field-of-view of the airport.");
[execute a convolutional neural network to extract features for the plurality of images];
determine a plurality of candidate runways (the plurality of runways in Paragraph [0003] equates to a plurality of candidate runways) among known runways in the runway database by performing an analysis (Paragraph [0003] “the program instructions cause the one or more processors to select an expected landing runway from the plurality of runways based on the plurality of track alignments, the plurality of vector alignments, the plurality of vertical alignments, the plurality of in-approach geometry parameters, and the plurality of in-flight plan parameters”; Paragraph [0032] "the runway database 302 may include one or more of a runway 312, a bearing 314, a threshold 316, and the like.") [to match interest points in the extracted features to interest points on a runway or runway marking in a mask in the runway database];
[analyze spatial distributions and expected patterns of runway features to infer likely locations of missing interest points in the extracted features];
generate, as a runway identification (expected landing runway 354 in Paragraph [0077] equates to a runway identification), one candidate runway among the plurality of candidate runways (multiple of the runways 312 in Paragraph [0077] equate to plurality of candidate runways) [based on the mask that matches the extracted features] (Paragraph [0076] "the location of the runways 312 may indicate the aircraft is or is not aligned with the runway. In this regard, the processors 204 may select the expected landing runway from the runways based on the one or more runways detected in the image 800."; Paragraph [0077] “the expected landing runway 354 may change between multiple of the runways 312 within a time of a few seconds.");
and output the runway identification (Paragraph [0044] “the expected landing runway 354 may be provided to the runway overrun alert and awareness system 212 for determining the runway overrun for the expected landing runway 354. The runway overrun alert and awareness system 212 may determine the runway overrun in response to the processors 204 selecting the expected landing runway 354 from the plurality of runways 312”).
However, Kosyanchuk fails to teach execute a convolutional neural network to extract features for the plurality of images; match interest points in the extracted features to interest points on a runway or runway marking in a mask in the runway database, analyze spatial distributions and expected patterns of runway features to infer likely locations of missing interest points in the extracted features.
Chen teaches execute [a convolutional neural network] to extract features for the plurality of images (Paragraph [0026] “The data 52 may be three or four corners of the runway in some embodiments but other trackable features of the runway may be used provided the location of corresponding features can be extracted by the computer vision processing module 26.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk’s reference to include execute [a convolutional neural network] to extract features for the plurality of images taught by Chen’s reference. The motivation for doing so would have been to determine the position of the camera and the real world aircraft position as suggested by Chen (see Chen, Paragraph [0023]).
However, Kosyanchuk and Chen both fail to teach a convolutional neural network; match interest points in the extracted features to interest points on a runway or runway marking in a mask in the runway database; analyze spatial distributions and expected patterns of runway features to infer likely locations of missing interest points in the extracted features.
Evans teaches a convolutional neural network (Paragraph [0067] “The machine learning model may include a CNN, as well as a fully-connected dense network and an n-DOF regression, such as a 6DOF regression or a 2DOF regression.”);
match interest points in the extracted features to interest points on a runway or runway marking in a mask in the runway database (Paragraph [0024] “applying the mask to a corner detector to detect interest points on the mask and thereby the runway or the runway marking in the image; matching the interest points on the runway or the runway marking in the image, to corresponding points on the runway or the runway marking that have known runway-framed local coordinates; and performing a perspective-n-point (PnP) estimation, using the interest points and the known runway-framed local coordinates”);
analyze spatial distributions and expected patterns of runway features to infer likely locations of missing interest points in the extracted features (Paragraph [0103] “ In some implementations in which a portion of the runway 202 is clipped (not visible) in the image, the mask 410 may include up to six edges, as shown in FIG. 7 for a mask 700… the pose-estimation engine may be configured to determine all intersection points of all of the edges of the mask, and only consider those having a vanishing point near the horizon as probable sides of the runway.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective filing date to modify Kosyanchuk in view of Chen to include a convolutional neural network; match interest points in the extracted features to interest points on a runway or runway marking in a mask in the runway database; analyze spatial distributions and expected patterns of runway features to infer likely locations of missing interest points in the extracted features taught by Evans’ reference. The motivation for doing so would have been to process full-resolution images, determine the position of an aircraft relative to the runway, and control an accuracy parameter so that an intended number of edges are detected as suggested by Evans (see Evans, Paragraphs [0024], [0067], and [0103]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Evans with Kosyanchuk and Chen to obtain the invention specified in claim 20.
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Kosyanchuk (US 2024/0242616 A1) in view of Chen et al. (US 2022/0234752 A1) (hereinafter, “Chen”); further in view of Evans et al. (US 2022/0198703 A1) (“hereinafter, “Evans”) and Badrinarayanan ("Segnet: A deep convolutional encoder-decoder architecture for image segmentation", IEEE transactions on pattern analysis and machine intelligence 39.12 (2017): 2481-2495).
Regarding claim 5, which claim 4 is incorporated, Kosyanchuk, Chen and Evans fail to teach wherein a plurality of down-convolutional layers with ReLU activation and max pooling are applied to the plurality of images, followed by applying a plurality of up-convolutional layers to the plurality of images.
Badrinarayanan teaches wherein a plurality of down-convolutional layers with ReLU activation and max pooling are applied to the plurality of images (Page 2485 [left column paragraph 1] “Each encoder in the encoder network performs convolution with a filter bank to produce a set of feature maps. These are then batch normalized [50], [51]). Then an element-wise rectified-linear non-linearity (ReLU) max(0,x) is applied. Following that, max-pooling with a 2×2 window and stride 2 (non-overlapping window) is performed and the resulting output is sub-sampled by a factor of 2.”), followed by applying a plurality of up-convolutional layers to the plurality of images (Page 2485 [left column paragraph 2] “The appropriate decoder in the decoder network upsamples its input feature map(s) using the memorized max-pooling indices from the corresponding encoder feature map(s).”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective
filing date to modify Kosyanchuk in view of Chen; and further in view of Evans to include wherein a plurality of down-convolutional layers with ReLU activation and max pooling are applied to the plurality of images, followed by applying a plurality of up-convolutional layers to the plurality of images taught by Badrinarayanan’s reference. The motivation for doing so would have been to produce features that are useful for accurate boundary localization as suggested by Badrinarayanan (see Badrinarayanan, Introduction [left column paragraph 1]).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Badrinarayanan with Kosyanchuk, Chen and Evans to obtain the invention specified in claim 5.
Regarding claim 15 (drawn to a method), claim 15 is rejected the same as claim 5 and the arguments similar to that presented above for claim 5 are equally applicable to the claim 15, and all the other limitations similar to claim 5 are not repeated herein, but incorporated by reference.
Claims 9 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kosyanchuk (US 2024/0242616 A1) in view of Chen et al. (US 2022/0234752 A1) (hereinafter, “Chen”); further in view of Evans et al. (US 2022/0198703 A1) (“hereinafter, “Evans”) and Gong et al. ("A survey of techniques for detection and tracking of airport runways", 44th AIA A Aerospace Sciences Meeting and Exhibit. 2006) (hereinafter, Gong).
Regarding claim 9, which claim 1 is incorporated, Kosyanchuk, Chen and Evans fail to teach wherein the feature extractor is executed to classify each pixel in at least a portion of the plurality of images as runway or not runway.
Gong teaches wherein the feature extractor is executed to classify each pixel in at least a portion of the plurality of images as runway or not runway (Page 8, paragraph 4 “In step 2, each pixel is compared to a threshold value, and is then assigned to either foreground or background of the image. In step 3, the upper portion of the image is removed from further consideration. This part of the image contains the sky, which was very bright in appearance. The lower portion of the image becomes the region of interest (ROI) for the remaining steps. Step 4 assigns a unique label to each set of connected foreground points within the ROI. This allows each connected set of pixels, known as a region, to be analyzed separately. In step 5, small regions are removed from further consideration, based on the expected size of a runway in the image.”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective
filing date to modify Kosyanchuk in view of Chen; and further in view of Evans to include wherein the feature extractor is executed to classify each pixel in at least a portion of the plurality of images as runway or not runway taught by Gong’s reference. The motivation for doing so would have been to assign a pixel to either the foreground or the background, to determine the region of interest as suggested by Gong (see Gong, Page 8, paragraph 4).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Gong with Kosyanchuk, Chen and Evans to obtain the invention specified in claim 9.
Regarding claim 19 (drawn to a method), claim 19 is rejected the same as claim 9 and the arguments similar to that presented above for claim 9 are equally applicable to the claim 19, and all the other limitations similar to claim 9 are not repeated herein, but incorporated by reference.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Kosyanchuk (US 2024/0242616 A1) in view of Chen et al. (US 2022/0234752 A1) (hereinafter, “Chen”); further in view of Evans et al. (US 2022/0198703 A1) (“hereinafter, “Evans”) and Datla et al. ("A multimodal semantic segmentation for airport runway delineation in panchromatic remote sensing images." Fourteenth International Conference on Machine Vision (ICMV 2021). Vol. 12084. SPIE, 2022.) (hereinafter, “Datla”).
Regarding claim 13, which claim 12 is incorporated, Kosyanchuk, Chen and Evans fail to teach wherein the machine perception system is a vision transformer.
Datla teaches wherein the machine perception system is a vision transformer (Page 4, last paragraph “We use ViT [16] as encoder containing 12 Transformers. The hybrid encoder is designed by combining ResNet-50 and ViT. Our network provides promising results by training for 75 epochs comprising of 210 iterations with batch size of 8 samples”).
Therefore, it would have been obvious to one of ordinary skill of the art before the effective
filing date to modify Kosyanchuk in view of Chen; and further in view of Evans to include wherein the machine perception system is a vision transformer taught by Datla’s reference. The motivation for doing so would have been to exercise the innate self-attention capability of Transformers as suggested by Datla (see Datla, Page 2, paragraph 2).
Further, one skilled in the art could have combined the elements described above by known methods with no change to the respective functions, and the combination would have yielded nothing more that predictable results. Therefore, it would have been obvious to combine Datal with Kosyanchuk, Chen and Evans to obtain the invention specified in claim 13.
Allowable Subject Matter
Claim 21 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 21 contains subject matter that is not disclosed or made obvious in the cited art.
In regards to claim 21, when considering claim 21 as a whole, prior art fails to disclose or render obvious, alone or in combination:
“[…] wherein the plurality of candidate runways are determined among the known runways in the runway database without reliance on navigational aids external to the computing system.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Toumazet et al. (US 2022/0348353 A1) discloses an airport approach assistance system the is configured to extract a region of interest from a captured image and compare the region to a second image obtained stored in a database and generate a signal according to the comparison result.
Zheng et al. (US 2021/0241641 A1) discloses storing a database comprising a map of plurality of runways and determining whether a vehicle is active on a runway that matches a runway determined from a received clearance.
Ishihara et al. (US 10,204,523 B1) discloses a runway awareness system that processes aircraft control data and runway position and location data to determine an aircraft runway status and provide an associated audio alert.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to UROOJ FATIMA whose telephone number is (571)272-2096. The examiner can normally be reached M-F 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at (571) 272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/UROOJ FATIMA/Examiner, Art Unit 2676
/Henok Shiferaw/Supervisory Patent Examiner, Art Unit 2676
Prosecution Insights