Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
RESTRICTION RESPONSE
Applicant’s election without traverse of (Group 1) Claims 1-8 & 14-20 in the reply filed on 1/26/2026 is acknowledged.
Claims 9-13 are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected group, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on (1/26/2026). Applicant is reminded claims 9-13 are required to be cancelled.
INFORMATION DISCLOSURE STATEMENT
The information disclosure statement (IDS) submitted on 10/30/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
CLAIM REJECTIONS - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-8 & 14-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to as ineligible under subject eligibility test. In the Subject Matter Eligibility Test for Products and Processes (Federal Register, Vol. 79, No. 241, dated Tuesday, December 16, 2014, page 74621), The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional device elements, which are recited at a high level of generality, provide conventional computer functions that do not add meaningful limits to practicing the abstract idea.
Claims 1 & 14
Step 1
This step inquires “is the claim to a process, article of machine, manufacture or composition of matter?” Yes,
Claim 1 - “Processors” are machines.
Claim 14 – “Method” is a process.
Step 2A - Prong 1
This step inquires “does the claim recite an abstract idea, law or natural phenomenon”. This claim appears to directed to an abstract idea.
The limitation of “receiving, using one or more processors, a first sensor image detected at a first time point and a second sensor image detected at a second time point; determining, using the one or more processors and a neural network, and based at least on the first sensor image and the second sensor image, one or more features represented by the first sensor image and the second sensor image; determining, using the one or more processors and the neural network, a grid of the scene in which the one or more features are respectively assigned to a cell of the grid; and at least one of (i) assigning, using the one or more processors, the grid to a map data structure or (ii) presenting, using the one or more processors and a display device, the grid.”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind (e.g. mathematical concepts, mental processes or certain methods of organizing human activity) but for the recitation of generic computer components. That is, other than reciting “one or more processors; neural network” nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “one or more processors; neural network” language, “receiving, determining, assigning, presenting” in the context of this claim encompasses covers performance of the limitation in the mind (e.g. mathematical concepts, mental processes or certain methods of organizing human activity).
STEP 2A – PRONG 1 - CONCLUSION
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A - Prong 2
This step inquires “does the claim recite additional elements that integrate the judicial exception into a practical application”. This judicial exception is not integrated into a practical application. In particular, the claim recites two additional element – using a “one or more processors; neural network” to perform “receiving, determining, assigning, presenting” steps. The “one or more processors; neural network” are recited at a high-level of generality (i.e., as a generic processor) receiving, using one or more processors, a first sensor image detected at a first time point and a second sensor image detected at a second time point; determining, using the one or more processors and a neural network, and based at least on the first sensor image and the second sensor image, one or more features represented by the first sensor image and the second sensor image; determining, using the one or more processors and the neural network, a grid of the scene in which the one or more features are respectively assigned to a cell of the grid; and at least one of (i) assigning, using the one or more processors, the grid to a map data structure or (ii) presenting, using the one or more processors and a display device, the grid such that it amounts no more than mere instructions to apply the exception using a generic computer component.
STEP 2A – PRONG 2 - CONCLUSION
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B
The critical inquiry here is does the claim recite additional elements that amount to “significantly more” than the judicial exception? The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using a “one or more processors; neural network” to perform “receiving, determining, assigning, presenting” steps amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Dependent Claims
As to claims 2 & 15, this claim is directed to generic computer components (“circuits/processor”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claims 3 & 16, this claim is directed to generic computer components (“circuits/processor”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claims 4 & 17, this claim is directed to generic computer components (“circuits/processor”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claims 5 & 18, this claim is directed to generic computer components (“circuits/processor”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claims 6 & 19, this claim is directed to generic computer components (“circuits/processor”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claims 7 & 20, this claim is directed to generic computer components (“circuits/processor”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
As to claim 8, this claim is directed to generic computer components (“circuits/processor”), and insignificant extra-solution activity (“field of use/environmental limitations”). Thus, this claim does not integrate the abstract idea into a practical application or constitute significantly more than the abstract.
CLAIM REJECTIONS - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 5-8, 14, 16 & 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over
Kim et al. (U.S. Publication 2022/0155096) in view of Emmons et al. (U.S. Publication 2023/0057509)
As to claims 1 & 14, Kim discloses determining, using the one or more processors and the neural network, a grid of the scene in which the one or more features are respectively assigned to a cell of the grid ([0011] discloses generating a global feature map including respective features for each of a plurality of locations in the top-down representation. [0023] discloses identifying a grid representation discretizes into a plurality of pillars, with each points being assigned.).
Kim is silent to receiving, using one or more processors, a first sensor image detected at a first time point and a second sensor image detected at a second time point; determining, using the one or more processors and a neural network, and based at least on the first sensor image and the second sensor image, one or more features represented by the first sensor image and the second sensor image; and at least one of (i) assigning, using the one or more processors, the grid to a map data structure or (ii) presenting, using the one or more processors and a display device the grid.
However, Emmon’s discloses receiving, using one or more processors, a first sensor image detected at a first time point and a second sensor image detected at a second time point ([0082] discloses selecting from a temporally indexed queue and frames spread apart in time. [0090] discloses the system obtains images from image sensor.); determining, using the one or more processors and a neural network, and based at least on the first sensor image and the second sensor image, one or more features represented by the first sensor image and the second sensor image ([0090] discloses computing a forward pass through backbone networks. The output may represent features.); and at least one of (i) assigning, using the one or more processors, the grid to a map data structure or (ii) presenting, using the one or more processors and a display device ([0093] discloses display presentation. [0096] discloses a display 708 may present graphical depictions. ) the grid.
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kim’s disclosure to include the above limitations in order to leverage temporally separated camera frames to form/update the scene representation and present that representation to a driver/passenger.
As to claims 3 & 16, Kim in view of Emmons discloses everything as disclosed in claim 1. In addition, Emmons discloses wherein the one or more circuits are to provide, as input to the neural network, a position representation of at least one of a camera center of the first sensor image, a camera center of the second sensor image, a vector of a ray to a feature of the first sensor image, or a vector of a ray to a feature of the second sensor image. ([0064] discloses a lookup table used with extrinsic and intrinsic camera parameters. [0065] discloses each pixel represents a ray out of an image wherein the ray extends in the virtual camera space.)
As to claims 5 & 18, Kim in view of Emmons discloses everything as disclosed in claim 1. In addition, Kim discloses wherein the grid comprises a two- dimensional representation of the scene in a top-down frame of reference, and the one or more circuits are to determine, for each feature of the one or more features, a polyline representing the feature, the polyline comprising a plurality of points indicating a plurality of line segments, wherein the one or more circuits are to assign the feature to the cell by assigning the polyline to the cell. ([0053] discloses discretize into an orthogonal top down 2D grid map. [0055] discloses road elements annotated in the form of continuous curves. Also, see wherein we uniformly sample the road segment represented as an unordered set of points.)
As to claims 6 & 19, Kim in view of Emmons discloses everything as disclosed in claim 1. In addition, Kim discloses wherein the one or more circuits are to assign at least one of a height of the feature or a class of the feature to the cell. ([0055] discloses type vector q encodes the road element type (e.g. lane, road, boundary).)
As to claims 7 & 20, Kim in view of Emmons discloses everything as disclosed in claim 1. In addition, Emmons discloses wherein the first sensor image and the second sensor image comprise camera data. ([0030] discloses image data may include images captured by any type of image sensor (e.g. camera))
As to claim 8, Kim in view of Emmons discloses everything as disclosed in claim 1. In addition, Kim discloses wherein the processor is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources. ([0039] discloses the autonomous driving system may include an onboard computer, a planning system, generates a driving route and vehicle controls actuate steering braking throttle.)
Claims 2 & 15 are rejected under 35 U.S.C. 103 as being unpatentable over
Kim et al. (U.S. Publication 2022/0155096) in view of Emmons et al. (U.S. Publication 2023/0057509) as applied in claims 1 & 14 above further in view Musk et al. (U.S. Publication 2020/0257317)
As to claims 2 & 15, Kim in view of Emmons discloses everything as disclosed in claims 1 & 14 but is silent to determining, using the one or more processors, the one or more features using at least one of radio detection and ranging (RADAR) data, light detection and ranging (LIDAR) data, or ultrasound data corresponding to at least one of the first sensor image or the second sensor image.
However, Musk discloses determining, using the one or more processors, the one or more features using at least one of radio detection and ranging (RADAR) data, light detection and ranging (LIDAR) data, or ultrasound data corresponding to at least one of the first sensor image or the second sensor image. ([0036] discloses vision data supplemented with additional sensor data. [0079] discloses the model input includes camera sensor data and auxiliary sensor data, such as ultrasonic sensor data. [0053] discloses image data may be augmented based on auxiliary data (including radar, ultrasonic)
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kim in view of Emmons’s disclosure to include the above limitations in order to improve robustness/accuracy of feature determination when image cues are degraded (e.g. occlusion/low visibility) in order to improve the reliability of the resulting scene/grid representation.
Claims 4 & 17 are rejected under 35 U.S.C. 103 as being unpatentable over
Kim et al. (U.S. Publication 2022/0155096) in view of Emmons et al. (U.S. Publication 2023/0057509) as applied in claims 1 & 14 above further in view JAEGLE et al. (WO 2022/248727)
As to claims 4 & 17, Kim in view of Emmons discloses everything as disclosed in claims 1 & 14 but is silent to wherein the neural network comprises: a featurizer to convert image data of the first sensor image and the second sensor image into a plurality of tokens in a latent data space; an encoder cross-attention processor to process the plurality of tokens and a latent data representation maintained by one or more self-attention modules; and a decoder cross-attention processor to process an intermediate output of the neural network and the latent data representation to determine the grid of the scene.
However, JAEGLE discloses wherein the neural network comprises: a featurizer to convert image data of the first sensor image and the second sensor image into a plurality of tokens in a latent data space (Pages 9-10 disclose converting image inputs into embeddings, including per pixel embeddings, and forming latent embeddings. See For Example if the network input includes an image the data element embeddings can correspond to each pixel. The encoder block can generate a representation as a set of latent embeddings in a latent space.); an encoder cross-attention processor to process the plurality of tokens and a latent data representation maintained by one or more self-attention modules (Pages 10-11 discloses cross attention between input embeddings and latent embeddings then self attention blocks operating over the latent embeddings. ); and a decoder cross-attention processor to process an intermediate output of the neural network and the latent data representation to determine the grid of the scene (Page 10 discloses each dimension of the network output generated through cross attention query embedding over the set of latent embeddings.).
It would have been obvious to one of ordinary skill in the art at the time of effective filing to modify Kim in view of Emmons’s disclosure to include the above limitations in order to (A) improve scalability and efficiency (B) improve representation capacity (C) provide flexible decoding mechanism.
CONCLUSION
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Stephen P Coleman whose telephone number is (571)270-5931. The examiner can normally be reached Monday-Thursday 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Moyer can be reached at (571) 272-9523. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Stephen P. Coleman
Primary Examiner
Art Unit 2675
/STEPHEN P COLEMAN/Primary Examiner, Art Unit 2675