DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 15 July 2026 has been entered.
Summary
The Amendment filed on 15 July 2026 has been acknowledged.
Claims 1 – 2, 5, 8 – 9, 11. 13 – 14, 16, 19 and 21 are amended.
Claims 4 and 7 are cancelled.
Claims 22 – 23 are newly presented.
Currently, claims 1 – 3, 5 – 6, 8 – 9 and 11 – 23 are pending and considered as set forth.
Response to Arguments
Applicant’s arguments with respect to claims 1, 11 and 19 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1 –3, 5 – 6, 8 – 9 and 11 – 23 are rejected under 35 U.S.C. 103 as being unpatentable over Mirkovic et al. (Hereinafter Mirkovic)(US 2024/0190452 A1) in view of Arbabian et al. (Hereinafter Arbabian)(US 2022/0130109 A1).
As per claim 1, Mirkovic teaches the limitations of:
A method comprising:
generating, using one or more neural networks and based at least on first sensor data generated using one or more first sensors of one or more first sensor modalities, first data indicating one or more first locations associated with one or more first objects in an environment (See at least paragraph 33 and 39; The sensor system 111 may include one or more sensors that are coupled to and/or are included within the AV 102, as illustrated in FIG. 11. For example, such sensors may include, without limitation, a light detection and ranging (LiDAR) system, a radio detection and ranging (radar) system, a laser detection and ranging (LADAR) system, a sound navigation and ranging (sonar) system, one or more cameras (for example, visible spectrum cameras, infrared cameras, etc.), temperature sensors, position sensors (for example, a global positioning system (GPS), etc.), location sensors, fuel level sensors, motion sensors (for example, an inertial measurement unit (IMU), etc.), humidity sensors, occupancy sensors, or the like. The sensor data can include information that describes the location of objects within the surrounding environment of the AV 102, information about the environment itself, information about the motion of the AV 102, information about a route of the vehicle, or the like. … he perception system 202 includes sensors that capture information about moving actors and other objects that exist in the vehicle's environment or surroundings. Example sensors include cameras, LiDAR systems, and radar systems. The data captured by such sensors (such as a digital image, lidar point cloud data, or radar data) is known as perception data. Methods of identifying objects and assigning categorical labels to objects are well known in the art, and any suitable classification process may be used, such as those that make bounding box classifications for detected objects in a scene and use convolutional neural networks or other computer vision models.);
generating, using one or more algorithmic processing techniques and based at least on second sensor data generated using one or more second sensors of one or more second sensor modalities different from the one or more first sensor modalities, second data indicating one or more first attributes associated with one or more second objects in the environment (See at least paragraph 39; The perception system 202 includes sensors that capture information about moving actors and other objects that exist in the vehicle's environment or surroundings. Example sensors include cameras, LiDAR systems, and radar systems. The data captured by such sensors (such as a digital image, lidar point cloud data, or radar data) is known as perception data. The perception system may include one or more processors, along with a computer-readable memory with programming instructions and/or trained artificial intelligence models that, during a run of the vehicle, will process the captured data to identify objects and assign categorical labels and unique identifiers to each object detected in a scene. Categorical labels may include categories such as vehicle, bicyclist, pedestrian, building, and the like. Methods of identifying objects and assigning categorical labels to objects are well known in the art, and any suitable classification process may be used, such as those that make bounding box classifications for detected objects in a scene and use convolutional neural networks or other computer vision models.); and
performing one or more control operations that cause a machine to navigate within the environment based at least on the third data (See at least paragraph 41 and 64 – 65; In an AV, the vehicle's perception system 202, as well as the vehicle's forecasting system 203, will deliver data and information to the vehicle's motion planning system 204 and motion control system 205 so that the receiving systems may assess such data and initiate any number of reactive motions to such data. The motion planning system 204 and control system 205 include and/or share one or more processors and computer-readable programming instructions that are configured to process data received from the other systems, determine a trajectory for the vehicle, and output commands to vehicle hardware to move the vehicle according to the determined trajectory. Example actions that such commands may cause the vehicle hardware to take include causing the vehicle's brake control system to actuate, causing the vehicle's acceleration control subsystem to increase speed of the vehicle, or causing the vehicle's steering control subsystem to turn the vehicle.).
Mirkovic does not explicitly teach the limitation of:
determining a common object that is represented in both the first data and the second data;
fusing at least a portion of the first data and at least a portion of the second data to generate third data comprising at least one of:
an updated version of the first data indicating an updated location of the common object, the updated location determined based at least on an attribute, of the one or more attributes, that is associated with the common object; or
an updated version of the second data indicating an updated attribute of the common object, the updated attribute determined based at least on a location, of the one or more locations, that is associated with the common object.
Arbabian teaches the limitation of:
determining a common object that is represented in both the first data and the second data (See at least paragraph 66; the computing machine 412 is configured with executable instructions 416 stored in a storage memory 416 to implement a sensor unit track fusion module 436 to fuse radar tracks and corresponding camera tracks that track the same object within the sensor unit FOV 420. An object may be detected by one or more sensors within the sensor unit, e.g., the object may be sensed by only the radar unit 404, or by only the image unit 406, or by both the radar unit 404 and the image unit 406. An example sensor unit track fusion module 436 matches radar tracks and camera tracks that track the same object and fuses them into a single unified sensor unit track corresponding to the tracked object. The example sensor unit track fusion module 436 matches radar tracks and camera tracks based upon comparisons of information contained within the respective tracks. An example sensor unit track fusion module 436 matches radar tracks and camera tracks based upon comparisons of one or more of the tracks': radar classification and camera classifications, radar timestamp and camera timestamp information, and radar ROI and camera ROI. An example sensor unit track fusion module 436 fuses information from a matched radar track/camera track pair into a single sensor unit track that represents a single object represented by each member of the matched track/camera track pair.);
fusing at least a portion of the first data and at least a portion of the second data to generate third data (See at least paragraph 66) comprising at least one of:
an updated version of the first data indicating an updated location of the common object, the updated location determined based at least on an attribute, of the one or more attributes, that is associated with the common object; or
an updated version of the second data indicating an updated attribute of the common object, the updated attribute determined based at least on a location, of the one or more locations, that is associated with the common object (See at least paragraph 66 – 67; representing sensor unit fusion block 436 fusing multiple example radar sensor tracks 602.sub.1, 602.sub.2, 603.sub.3 and multiple example image sensor tracks 604.sub.1, 604.sub.2, and 604.sub.3 to produce multiple example fused sensor unit object tracks 606.sub.1, 606.sub.2, and 606.sub.3. The sensor unit track fusion module 436 can produce a plurality of sensor unit object tracks that each corresponds to one of a plurality of detected objects. Each sensor unit track comprises a data structure stored in a storage memory that includes but is not limited to the following information about an object tacked within a sensor unit's FOV: active status, classification, motion state estimates (velocity, acceleration, heading), one or more bounding boxes on respective sensors, timestamp data point, radar and camera ROIs, active status, and track ID).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to include determining a common object that is represented in both the first data and the second data; fusing at least a portion of the first data and at least a portion of the second data to generate third data comprising at least one of: an updated version of the first data indicating an updated location of the common object, the updated location determined based at least on an attribute, of the one or more attributes, that is associated with the common object; or an updated version of the second data indicating an updated attribute of the common object, the updated attribute determined based at least on a location, of the one or more locations, that is associated with the common object.as taught by Arbabian in the system of Mirkovic, since the claimed invention is merely a combination of old elements, and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
As per claim 2, Mirkovic teaches the limitations of:
wherein: the first data is instantaneous data indicating the one or more first locations associated with the one or more first objects in the environment surrounding the machine at an instance of time, and the second data is temporal data and the one or more first attributes are tracked over a period of time that at least partially precedes the instance of time (See at least paragraph 97).
As per claim 3, Mirkovic teaches the limitations of:
wherein: the first data further indicates one or more occluded portions of the environment, and the updated version of the first data further indicates whether the one or more occluded portions of the environment are occupied by at least one of the one or more first objects or at least one of the one or more second objects (See at least abstract and paragraph 57).
As per claim 5, the combination of Mirkovic and Arbabian teaches the limitations of:
wherein the first data is a dense occupancy representation of the environment from a top-down perspective, the dense occupancy representation including one or more points representing one or more samples obtained using the one or more first sensors at an instance of time, wherein one or more first points of the one or more points correspond to the one or more locations associated with the one or more first objects and one or more second points of the one or more points correspond to one or more unoccupied locations in the environment at the instance of time (Mirkovic, see at least paragraph 61, and Arbabian, see at least paragraph 67).
As per claim 6, the combination of Mirkovic and Arbabian teaches the limitations of:
wherein one or more values of the one or more points correspond to at least one of a height or a confidence associated with the one or more samples (Mirkovic, See at least paragraph 56 – 57).
As per claim 8, Mirkovic teaches the limitations of:
generating fourth data indicating one or more prior locations associated with the one or more first objects in the environment, the fourth data including one or more points representing one or more prior samples obtained using the one or more first sensors over a period of time and refined based at least on the one or more attributes, wherein the generating the third data is further based at least on the fourth data (See at least paragraph 41 and 64 – 65).
As per claim 9, the combination of Mirkovic and Arbabian teaches the limitations of:
wherein a first point of the one or more points included in the fourth data is indicative of a velocity associated with the common object (See at least paragraph 44 and 4 and Arbabian, see at least paragraph 676).
As per claim 17, Mirkovic teaches the limitations of:
wherein the one or more first sensors include one or more of: an image sensor; a radar sensor; an ultrasonic sensor; or a LiDAR sensor (See at least paragraph 33).
As per claim 18, Mirkovic teaches the limitations of:
wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational AI operations; a system implementing one or more large language models (LLMs); a system for generating synthetic data; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (See at least abstract).
As per claim 21, the combination of Mirkovic and Arbabian teaches the limitations of:
wherein the fused perception information is generated by fusing at least a portion of the occupancy representation and at least a portion of the attribute information to modify the occupancy representation based at least on the attribute information or to modify the attribute information based at least on the occupancy representation (Mirkovic, see at least paragraph 33 and Arbabian, see at least paragraph 66 – 67).
As per claim 22, the combination of Mirkovic and Arbabian teaches the limitations of:
wherein the third data comprises the updated version of the second data, and wherein the updated version of the second data indicates one or more second attributes for the common object comprising at least one of: a refined bounding shape; a refined location; a refined pose; a refined trajectory; or a refined classification (Arbabian, see at least paragraph 70 and 76).
As per claim 23, the combination of Mirkovic and Arbabian teaches the limitations of:
selecting the portion of the first data and the portion of the second data for the fusing based at least on the common object being represented in both the first data and the second data such that the portion of the first data comprises a location associated with the common object and the portion of the second data comprises an attribute associated with the common object (Arbabian, see at least paragraph 66 – 67 and 70).
Regarding claims 11 – 16 and 19 - 20:
Claims 11 – 16 and 19 - 20 are rejected using the same rationale, mutatis mutandis, applied to claims 1 – 9 and 17 - 18 above, respectively.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IG T AN whose telephone number is (571)270-5110. The examiner can normally be reached M - F: 10:00AM- 4:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aniss Chad can be reached at (571) 270-3832. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
IG T AN
Primary Examiner
Art Unit 3662
/IG T AN/Primary Examiner, Art Unit 3662
Prosecution Insights