DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Office Action is in response to the Office Action Response dated June 24, 2026. Claims 1-20 are presently pending and are presented for examination.
Response to Arguments
Applicant’s amendments and arguments overcome the drawing and claim objections, claim rejections under 35 USC 101, 102 and 112(b). However, the claims are now rejected under 35 USC 103, based upon the discovery of new references, as indicated herein.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on August 5, 2026 is in compliance with the provisions of 37 CFT 1.97. Accordingly, the information disclosure statement has been considered by the examiner.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5-12 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No. 2025/0022143, to Li et al. (hereinafter Li), in view of U.S. Patent Publication No. 2025/0028326, to Leung et al. (hereinafter Leung).
As per claim 1, and similarly with respect to claims 11 and 12, Li discloses one or more processors comprising processing circuitry to: detect, based at least on one or more neural networks (NNs) extracting features from image data and LiDAR data corresponding to an environment of an ego-machine…one or more obstacles in the environment (e.g. see Figs. 1 and 3, and paras 0030-0050, wherein an autonomous vehicle (AV) (i.e. ego-machine) is provided including a sensing system 110 having LiDAR 112 and cameras 118 transmitting data (LiDAR and image data) to a perception system 130 including an object tracking pipeline (OTP) 132, wherein the OTP includes machine learning models 220-240 comprising deep neural networks having various neural networks and transformers; also see para 0050, wherein the neural network tracks objects (i.e. extracting features from the image and LiDAR data)),…; and control one or more operations of the ego-machine based at least on the one or more obstacles (e.g. see para 0039, wherein based upon the monitoring and prediction component the AV is controlled by the AV control system 140).
Li fails to disclose all of the features of the detection of the one or more obstacles being further based upon one or more transformers of the one or more NNs processing features sampled from the extracted features at one or more locations corresponding to one or more three-dimensional (3D) transformer queries. However, Leung teaches a neural network including a transformer, wherein the transformer uses query-based attention to determine three-dimensional occupancy features of an object surrounding a vehicle (e.g. see paras 0109-0110 and 0134). It would have been obvious to a person of ordinary skill in the art at the time of Applicants’ invention to modify the system of Li to include generating 3D transformer queries for the purpose of detecting objects surrounding a vehicle and avoiding the same during travel.
As per claim 5, and similarly with respect to claim 16, Li, as modified by Leung, teaches the features of claims 1 and 12, respectively, and Li further discloses wherein the one or more processors are further to generate the sampled features based at least on projecting one or more keypoints associated with the one or more locations into corresponding extracted image features and extracted LiDAR features of the extracted features (e.g. see Fig. 4, wherein the transformation, based upon camera and LiDAR data, is based upon keypoints, shown as dots on objects detected by the LiDAR 404)).
As per claim 6, and similarly with respect to claim 17, Li, as modified by Leung, teaches the features of claims 1 and 12, respectively, and Li further discloses wherein the processing circuitry is further to generate a plurality of transformer queries based at least on one or more candidate bounding shapes extracted from the image data or the LiDAR data (e.g. see Fig. 4 and paras 0047-0051, wherein the transformation, based upon camera and LiDAR data, is based upon the formation of bounding boxes 412, 422), one or more randomly initialized 3D positions (e.g. see Fig. 4, wherein position dots are identified by LiDAR 404) and one or more ego-motion compensated transformer predictions (e.g. see para 0038, wherein the AV includes prediction component 134).
As per claim 7, and similarly with respect to claim 18, Li, as modified by Leung, teaches the features of claims 1 and 12, respectively, and Li further discloses wherein the processing circuitry is further to detect the one or more obstacles based at least on the one or more transformers (e.g. see para 0033, wherein the perception system detects the presence of road objects, which would include perceptible hazards): generating classification data representing whether there is road debris predicted at a 3D location corresponding to at least one individual transformer query of the one or more 3D transformer queries (e.g. see para 0037, wherein the objects are identified and classified; also see rejection of claim 1), and regressing a representation of a bounding shape of the road debris at the 3D location (e.g. see Fig. 4, the Office further notes that while vehicle and sign objects are identified and included within bounding shapes, given the object identification and classification other objects would also be identified and bounded within a shape).
As per claim 8, and similarly with respect to claim 19, Li, as modified by Leung, teaches the features of claims 1 and 12, respectively, and Li further discloses wherein the processing circuitry is further to update the one or more transformers based at least on omitting transformer queries representing reference 3D locations outside a ground truth navigable space (e.g. see Fig. 3 and para 0043-0046, wherein the models 220, 230, 240, which include one or more transformers, are trained with historic data (i.e. omitted transformer queries) from various location).
As per claim 9, Li, as modified by Leung, teaches the features of claim 1, and Li further discloses wherein the processing circuitry is further to control the motion of the ego-machine based at least on avoiding or compensating for the one or more obstacles (e.g. see Fig. 3 and rejection of claim 1, wherein an AVCS 140 is provided for controlling the vehicle based upon detected objects).
As per claim 10, and similarly with respect to claim 20, Li, as modified by Leung, teaches the features of claims 1 and 12, respectively, and Li further discloses wherein the one or more processors are comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine (e.g. see Fig. 1 and para 0037, wherein prediction model 134 is part of a perception system of the AV); a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system implementing one or more vision language models (VLMs); a system implementing one or more multi-modal language models; a system for generating synthetic data; a system for generating synthetic data using AI; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (e.g. see Fig. 3 and para 0043-0046, wherein the models 220, 230, 240, which include one or more transformers, are trained with historic data (i.e. omitted transformer queries) from various location).
Claims 2, 3, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No. 2025/0022143, to Li et al. (hereinafter Li), in view of U.S. Patent Publication No. 2025/0028326, to Leung et al. (hereinafter Leung), and in further view of U.S. Patent Publication No. 20250021712, to Gopalan et al. (hereinafter Gopalan).
As per claim 2, and similarly with respect to claim 13, Li, as modified by Leung, discloses the features of claims 1 and 12, respectively, but fails to particularly teach wherein the processing comprises generating the one or more 3D transformer queries based at least on one or more two-dimensional candidate bounding shapes extracted from the image data using the one or more NNs, and applying one or more sampled two-dimensional image features extracted from the image data using the one or more NNs to the one or more transformers. However, Gopalan teaches neural network having transformers utilizing two-dimensional grid formed about an autonomous vehicle, which would be based upon captured camera/LiDAR data, for generating the three-dimensional queries (e.g. see at least para 0076). It would have been obvious to a person of ordinary skill in the art at the time of Applicants’ invention to modify the system of Li to include generating two-dimensional boundary about a vehicle for the purpose for establishing an area of interest to limit processing resources.
As per claim 3, and similarly with respect to claim 14, Li, as modified by Leung, teaches the features of claims 1 and 12, respectively, but fails to particularly teach wherein the processing comprises generating the one or more 3D transformer queries based at least on one or more two-dimensional candidate bounding shapes extracted from the LiDAR data using the one or more NNs, and applying one or more sampled two-dimensional LiDAR features extracted from the LiDAR data using the one or more NNs to the one or more transformers. Gopalan teaches neural network having transformers utilizing two-dimensional grid formed about an autonomous vehicle, which would be based upon captured camera/LiDAR data, for generating the three-dimensional queries (e.g. see at least paras 0055 and 0076). It would have been obvious to a person of ordinary skill in the art at the time of Applicants’ invention to modify the system of Li to include generating two-dimensional boundary about a vehicle for the purpose for establishing an area of interest to limit processing resources.
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Patent Publication No. 2025/0022143, to Li et al. (hereinafter Li), in view of U.S. Patent Publication No. 2025/0028326, to Leung et al. (hereinafter Leung), and in further view of U.S. Patent Publication No. 2025/0381983, to Zhang et al. (hereinafter Zhang).
As per claim 4, and similarly with respect to claim 15, Li, as modified by Leung, teaches the features of claims 1 and 12, respectively, but fail to teach wherein the processing of the sampled features comprises fusing one or more sampled two-dimensional image features and one or more sampled two-dimensional LiDAR features corresponding to the sampled features using one or more cross-attention layers of the one or more transformers. However, Zhang teaches fusing camera and LiDAR data together in a two-dimensional array using a transformer (e.g. see paras 0031, 0086, 0114). It would have been obvious to a person of ordinary skill in the art at the time of Applicants’ invention to modify the system of Li to include fusing two-dimensional image and LiDAR data for the purpose of obtaining improved situational awareness.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James M. McPherson whose telephone number is (313) 446-6543. The examiner can normally be reached on 7:30 AM - 5PM Mon-Fri Eastern Alt Fri. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Flynn can be reached on 571 272-9855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES M MCPHERSON/Primary Examiner, Art Unit 3663B
Prosecution Insights