CTFR 18/508,001 CTFR 99317 DETAILED ACTION The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. The Amendment filed 27 March 2026 has been entered and considered. Claims 1, 9-14, 21-22, and 24-28 have been amended. Claims 2-5, 8, 15-19, and 29-30 have been cancelled. Claims 1, 6-7, 9-14, and 20-28 are all the claims pending in the application. Claims 1, 6-7, 9-14, and 20-28 are rejected. All new grounds of rejection set forth in the present action were necessitated by Applicant’s claim amendments; accordingly, this action is made final. Response to Amendment Prior Art Rejections In view of the amendments to independent Claims 1 and 28 , and their dependent claims by extension, the rejection under 35 USC 103 using previously cited art Rodriguez Hervas in view of Kocamaz is withdrawn. However, a rejection under 35 USC 103 is presented using previously cited art Rodriguez Hervas in view of Kocamaz and Xu . On pages 8-9 of the Amendment, the Applicant clarifies in Claim 1 the specificities of the object-to-lane association points and that Rodriguez Hervas and Kocamaz do not disclose this. Examiner agrees and thus presents previously cited pertinent art Xu. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claims 1, 6-7, 9-14, and 20-28 are re jected under 35 U.S.C. 103 as being unpatentable over Ro driguez Hervas et al. (U.S. Patent App. Pub No. 2022/0379913 A1, hereafter referred as Rodriguez Hervas) in view of Kocamaz et al. (U.S. Patent App. Pub No. 2023/0099494 A1, hereafter referred as Kocamaz) and Xu et al. (U.S. Patent App. Pub No. 2023/0267701 A1, hereafter referred as Xu). Re garding Claim 1 : Rodriguez Hervas teaches an apparatus for determining lane information (Rodriguez Hervas: Par. [0004]; the systems and methods of the present disclosure provide for using perception systems of machines (e.g., vehicles, robots, etc.) to detect and/or interpret signs and lanes—such as to associate signs with lanes) , the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory and configured to (Rodriguez Hervas: Par. [0037]; various functions may be carried out by a processor executing instructions stored in memory) : obtain an image representative of one or more lanes of a road and an object (Rodriguez Hervas: Par. [0057-0058]) and Fig. 4; depicts examples of lanes, signs, parameters, and attributes that may be derived from input data; the input data may correspond to an input frame 400, the input frame 400 (and/or other images and/or other sensor data representations (e.g., point clouds, projection images, etc.) generated using one or more sensors of the vehicle or machine 1000)) adjacent to the road (Rodriguez Hervas: Par. [0058] and Fig. 4; depicts signs 404A-C being adjacent to lanes 402A-C) . Rodriguez Hervas fails to teach determine coordinates of object-to-lane association points, wherein the object-to-lane association points are associated with the object and at least one lane of the one or more lanes of the road, and wherein the coordinates of the object-to-lane association points comprise image coordinates corresponding to lane edges of the at least one lane in the image; obtain, based on a map, three-dimensional coordinates of lane boundaries of the one or more lanes of the road; and associate the object with the at least one lane based on a relationship between the coordinates of the object-to-lane association points and three-dimensional coordinates of lane boundaries of the at least one lane. Kocamaz and Xu, like Rodriguez Hervas, are directed to determining lane information. Kocamaz and Xu in combination with Rodriguez Hervas does teach determine coordinates of object-to-lane association points, wherein the object-to-lane association points are associated with the object and at least one lane of the one or more lanes of the road (Kocamaz: Par. [0055-0056] and Fig. 3; determining at least a subset of pixels of the image corresponds to the at least one bounding shape, assigning an object class and a lane identifier corresponding to the respective combination of object class and lane identifier to at least the subset of pixels) , and wherein the coordinates of the object-to-lane association points comprise image coordinates corresponding to lane edges of the at least one lane in the image (Xu: Par. [0127]; the down-sampled polygon 472 may then be masked to represent the portion of the down-sampled image 412 that corresponds to the lane marking, boundary, and/or other feature, the mask may correspond to a binary mask (e.g., as illustrated in masked image 414); the masked image 414 may then be used as ground truth data to train the machine learning model(s) 108 to detect lane markings, road boundaries, and/or features that correspond to the location of the down-sampled polygon 472 in real-world coordinates) ; obtain, based on a map (Rodriguez Hervas: Par. [0023]; sign-to-path relevance information provided by a live perception system in accordance with various embodiments of the disclosure can be fused with information from a map (e.g., an HD map) to further enhance robustness and provide coverage in a wide range of real-world scenarios) , three-dimensional coordinates of lane boundaries of the one or more lanes of the road (Xu: Par. [0170] and Fig. C; FIG. 7C is a data flow diagram illustrating a three-dimensional (3D) KPI measured from lane detection and ground truth polyline vertices (or points) in accordance with some embodiments of the present disclosure; the 2D pixel locations in both the ground truth mask 744 (e.g. 414, 418, 426, etc.) and the detection masks 732 may be converted to 3D real-world coordinates (e.g., GPS coordinates, GNSS coordinates, etc.)) ; and associate the object with the at least one lane based on a relationship between the coordinates of the object-to-lane association points and three-dimensional coordinates of lane boundaries of the at least one lane (Rodriguez Hervas: Par. [0025-0026]; the lanes and/or the attributes may be derived from sensor data generated using at least one sensor of a machine (e.g., a vehicle); examples of the lane attributes are those that represent geometry of lanes (e.g., curvature, shape, coordinates, lane boundaries); a sign may be assigned to a lane based at least on correlating one or more lane attributes associated with the lane with one or more sign attributes associated with the sign; examples of sign attributes are those that represent a sign type, whether a sign is a primary sign or a supplemental sign, and/or geometry of signs (e.g., bounding boxes, contours, coordinates, shapes)) . It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Rodriguez Hervas to utilize associating the pixels with the object and the 2D to 3D coordinate conversion, as taught by Kocamaz and Xu, to arrive at the claimed invention discussed above. Such a modification is the result of combining prior art elements according to known methods to yield predictable results. As taught by Kocamaz, the proposed modification would better allow for the association of objects to lanes with reduced processing time (Kocamaz: Par. [0005]) . As taught by Xu, the proposed modification would help to determine early and accurate failures of the separate components of the lane detection system (Xu: Par. [0170]) . In regards to Claim 6 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the object comprises a sign that relates to the at least one lane (Rodriguez Hervas: Par. [0059] and Fig. 4; the geometry associator 126 may determine a segment, for example, based at least on determining a location associated with a sign along a lane and defining the segment of the lane using the location) . In regards to Claim 7 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the object comprises a road sign providing information that pertains to the at least one lane (Rodriguez Hervas: Par. [0059] and Fig. 4; the geometry associator 126 may determine a segment, for example, based at least on determining a location associated with a sign along a lane and defining the segment of the lane using the location) . In regards to Claim 9 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the coordinates of the object-to-lane association points that are laterally offset in the image from the object in the image (Kocamaz: Fig. 5A; showcases determining pixels that are laterally offset from the object) . In regards to Claim 10 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the coordinates of the object-to-lane association points are at a level of the road in the image (Kocamaz: Fig. 5A; showcases determining pixels that are level to the road) . In regards to Claim 11 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the coordinates of the object-to-lane association points are lower in the image than the object (Kocamaz: Fig. 5A; showcases determining pixels that are lower than the object) . In regards to Claim 12 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein a line between the image coordinates is substantially perpendicular to a direction of travel of the at least one lane (Kocamaz: Fig. 5A; 530 showcases determining pixels within the lane and being able to produce a line perpendicular to travel direction) . In regards to Claim 13 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein, to determine the coordinates of the object-to-lane association points, the at least one processor is configured to: provide the image to a neural network trained to determine coordinates representative of object-to-lane association points associated with objects; and obtain the coordinates of the object-to-lane association points from the neural network (Kocamaz: Par. [0070-0074] and Fig. 6; computing, using a neural network and based at least in part on sensor data generated using one or more sensors, one or more output masks, each output mask of the one or more output masks corresponding to an object class and a lane identifier; includes assigning an object class and a lane identifier to the object based at least in part on the at least one object class label and the at least one lane identifier label associated with the one or more points) . In regards to Claim 14 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 13, wherein the neural network is trained to determine image coordinates of object-to-lane association points (Kocamaz: Par. [0070-0074] and Fig. 6; computing, using a neural network and based at least in part on sensor data generated using one or more sensors, one or more output masks, each output mask of the one or more output masks corresponding to an object class and a lane identifier; includes assigning an object class and a lane identifier to the object based at least in part on the at least one object class label and the at least one lane identifier label associated with the one or more points) . In regards to Claim 20 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the at least one processor is further configured to: provide the image to a neural network trained to determine bounding boxes; and obtain a bounding box related to the object from the neural network (Kocamaz: Par. [0070-0074] and Fig. 6; computing, using a neural network and based at least in part on sensor data generated using one or more sensors, one or more output masks, each output mask of the one or more output masks corresponding to an object class and a lane identifier; determining a bounding shape corresponding to at least one object based at least in part on the sensor data; includes assigning an object class and a lane identifier to the object based at least in part on the at least one object class label and the at least one lane identifier label associated with the one or more points) . In regards to Claim 21 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 20, wherein the coordinates of the object-to-lane association points are determined based on the bounding box (Kocamaz: Par. [0070-0074] and Fig. 6; computing, using a neural network and based at least in part on sensor data generated using one or more sensors, one or more output masks, each output mask of the one or more output masks corresponding to an object class and a lane identifier; determining a bounding shape corresponding to at least one object based at least in part on the sensor data; includes assigning an object class and a lane identifier to the object based at least in part on the at least one object class label and the at least one lane identifier label associated with the one or more points) . In regards to Claim 22 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the at least one processor is further configured to determine bird’s-eye-view coordinates corresponding to the object-to-lane association points of the at least one lane of the one or more lanes of the road based on the coordinates of the object-to-lane association points (Kocamaz: Par. [0021]; in other embodiments, the output masks may be in three-dimensional (3D) world space, such that the input data is from a top-down or bird's eye view representation of an environment around the ego-vehicle; as such, the pixel coordinates may correspond to (x, y) locations in 3D world space relative to an origin of the ego-vehicle) . In regards to Claim 23 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 22, wherein the at least one processor is further configured to track the bird’s-eye-view coordinates based on successive images (Kocamaz: Par. [0021]; in other embodiments, the output masks may be in three-dimensional (3D) world space, such that the input data is from a top-down or bird's eye view representation of an environment around the ego-vehicle; as such, the pixel coordinates may correspond to (x, y) locations in 3D world space relative to an origin of the ego-vehicle; obvious that the input data could be successive images) . In regards to Claim 24 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the at least one processor is further configured to determine three-dimensional coordinates corresponding to the object-to-lane association points of the at least one lane based on the coordinates of the object-to-lane association points (Kocamaz: Par. [0021]; in other embodiments, the output masks may be in three-dimensional (3D) world space, such that the input data is from a top-down or bird's eye view representation of an environment around the ego-vehicle; as such, the pixel coordinates may correspond to (x, y) locations in 3D world space relative to an origin of the ego-vehicle) . In regards to Claim 25 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 24, wherein the at least one processor is further configured to track the three-dimensional coordinates of the object-to-lane association points based on successive images (Kocamaz: Par. [0021]; in other embodiments, the output masks may be in three-dimensional (3D) world space, such that the input data is from a top-down or bird's eye view representation of an environment around the ego-vehicle; as such, the pixel coordinates may correspond to (x, y) locations in 3D world space relative to an origin of the ego-vehicle; obvious that the input data could be successive images) . In regards to Claim 26 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the at least one processor is further configured to control a vehicle based on the coordinates of the object-to-lane association points (Rodriguez Hervas: Par. [0036]; disclosed embodiments may be comprised in a variety of different systems such as automotive systems (e.g., a control system for an autonomous or semi-autonomous machine, a perception system for an autonomous or semi-autonomous machine)) . In regards to Claim 27 , Rodriguez Hervas as modified by Kocamaz and Xu further teaches the apparatus of claim 1, wherein the at least one processor is further configured to provide information to a driver of a vehicle based on the coordinates of the object-to-lane association points (Rodriguez Hervas: Par. [0089]; to enable autonomous driving and/or to assist a human driver in driving the vehicle 1000) . Regarding Claim 28 : Rodriguez Hervas as modified by Kocamaz and Xu further teaches a method for determining lane information (Rodriguez Hervas: Par. [0004]; the systems and methods of the present disclosure provide for using perception systems of machines (e.g., vehicles, robots, etc.) to detect and/or interpret signs and lanes—such as to associate signs with lanes) , the method comprising: obtaining an image representative of one or more lanes of a road and an object (Rodriguez Hervas: Par. [0057-0058]) and Fig. 4; depicts examples of lanes, signs, parameters, and attributes that may be derived from input data; the input data may correspond to an input frame 400, the input frame 400 (and/or other images and/or other sensor data representations (e.g., point clouds, projection images, etc.) generated using one or more sensors of the vehicle or machine 1000)) adjacent to the road (Rodriguez Hervas: Par. [0058] and Fig. 4; depicts signs 404A-C being adjacent to lanes 402A-C) ; determining coordinates of object-to-lane association points, wherein the object-to-lane association points are associated with the object an at least one lane of the one or more lanes of the road (Kocamaz: Par. [0055-0056] and Fig. 3; determining at least a subset of pixels of the image corresponds to the at least one bounding shape, assigning an object class and a lane identifier corresponding to the respective combination of object class and lane identifier to at least the subset of pixels) , and wherein the coordinates of the object-to-lane association points comprise image coordinates corresponding to lane edges of the at least one lane in the image (Xu: Par. [0127]; the down-sampled polygon 472 may then be masked to represent the portion of the down-sampled image 412 that corresponds to the lane marking, boundary, and/or other feature, the mask may correspond to a binary mask (e.g., as illustrated in masked image 414); the masked image 414 may then be used as ground truth data to train the machine learning model(s) 108 to detect lane markings, road boundaries, and/or features that correspond to the location of the down-sampled polygon 472 in real-world coordinates) ; obtaining, based on a map (Rodriguez Hervas: Par. [0023]; sign-to-path relevance information provided by a live perception system in accordance with various embodiments of the disclosure can be fused with information from a map (e.g., an HD map) to further enhance robustness and provide coverage in a wide range of real-world scenarios) , three-dimensional coordinates of lane boundaries of the one or more lanes of the road (Xu: Par. [0170] and Fig. C; FIG. 7C is a data flow diagram illustrating a three-dimensional (3D) KPI measured from lane detection and ground truth polyline vertices (or points) in accordance with some embodiments of the present disclosure; the 2D pixel locations in both the ground truth mask 744 (e.g. 414, 418, 426, etc.) and the detection masks 732 may be converted to 3D real-world coordinates (e.g., GPS coordinates, GNSS coordinates, etc.)) ; and associating the object with the at least one lane based on a relationship between the coordinates of the object-to-lane association points and three-dimensional coordinates of lane boundaries of the at least one lane (Rodriguez Hervas: Par. [0025-0026]; the lanes and/or the attributes may be derived from sensor data generated using at least one sensor of a machine (e.g., a vehicle); examples of the lane attributes are those that represent geometry of lanes (e.g., curvature, shape, coordinates, lane boundaries); a sign may be assigned to a lane based at least on correlating one or more lane attributes associated with the lane with one or more sign attributes associated with the sign; examples of sign attributes are those that represent a sign type, whether a sign is a primary sign or a supplemental sign, and/or geometry of signs (e.g., bounding boxes, contours, coordinates, shapes)) . Conclusion 07-40 AIA Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action. Any inquiry concerning this communication or earlier communications from the examiner should be directed to RENAE BITOR whose telephone number is (703)756-5563. The examiner can normally be reached Monday to Friday: 8:00 - 5:30 but off the 1st Friday of the biweek. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, GREG MORSE can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /RENAE A BITOR/Examiner, Art Unit 2663 /GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698 Application/Control Number: 18/508,001 Page 2 Art Unit: 2663 Application/Control Number: 18/508,001 Page 4 Art Unit: 2663 Application/Control Number: 18/508,001 Page 5 Art Unit: 2663 Application/Control Number: 18/508,001 Page 6 Art Unit: 2663 Application/Control Number: 18/508,001 Page 7 Art Unit: 2663 Application/Control Number: 18/508,001 Page 8 Art Unit: 2663 Application/Control Number: 18/508,001 Page 9 Art Unit: 2663 Application/Control Number: 18/508,001 Page 10 Art Unit: 2663 Application/Control Number: 18/508,001 Page 11 Art Unit: 2663 Application/Control Number: 18/508,001 Page 12 Art Unit: 2663 Application/Control Number: 18/508,001 Page 13 Art Unit: 2663 Application/Control Number: 18/508,001 Page 14 Art Unit: 2663 Application/Control Number: 18/508,001 Page 15 Art Unit: 2663
Prosecution Insights