DETAILED ACTION
This is the first Office action on the merits and is responsive to the papers filed 06/10/2025. Claims 1-20 are currently pending and examined below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 03/26/2026 and 09/01/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements have been considered by the examiner.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 4-5, 8-11, 13-14, 16-18 and 20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Nehmadi et al. (US 20220398851 A1; hereinafter Nehmadi).
Regarding claim 1, Nehmadi discloses:
At least one processor (Fig. 1A: system 10, [0054] “The system has different functional elements which are implemented in hardware or software to perform individual tasks”) comprising:
one or more circuits (Figs. 1A and 13: sensor fusion module 100) to
at a first time step, update a portion of a first instance of a segmented representation of an environment based at least on determining, using first sensor data obtained using two or more sensors of different sensor modalities ([0081]-[0083] “FIG. 13 is a block diagram of a sensor fusion module 100 that receives inputs form the different sensors described earlier and integrates the raw data from sensors to generate in a specific example of implementation a 3D RGB-D-V model.”), that a first location corresponding to a first hazard indication corresponds to the portion (Fig. 30 shows 2 frames of occupancy grid, each frame includes obstacle, static/dynamic radar target, previous frame may correspond to the claimed ‘first time step’”);
at a second time step, update the portion of a second instance of the segmented representation of the environment based at least on determining, using the first instance of the segmented representation after ego-motion compensation ([0072] “The ego-motion module 58 has an ego-motion estimation functional block 59, which receives at an input 60 a previously calculated 3D map from the sensor fusion operation”, [0073] “It is preferred to compare data frames of the 3D map, as in addition to the image data, the 3D map also conveys distance information for each point in relation to the host vehicle, thus allowing to more precisely compute displacement between points in successive data frames.”) and second sensor data obtained using the two or more sensors ([0074] “image data”), that a second location corresponding to a second hazard indication corresponds to the portion (Fig. 30 shows 2 frames of occupancy grid, each frame includes obstacle, static/dynamic radar target, later frame may correspond to the claimed ‘second time step’”); and
perform one or more planning, navigation, or control operations using the second instance of the segmented representation as updated ([0080] “The Bayesian filter functional block 88 uses a motion model 98 that describes spatial relationships between objects in the scene and the vehicle, allowing to create a representation of the vehicle in its driving environment. The Bayesian filter functional block 88 outputs localization information which conveys the representation of the vehicle in its driving environment including the objects of interest in the vicinity of the vehicles including road boundaries and landmarks.”).
Regarding claim 2, Nehmadi discloses:
wherein the one or more circuits are further to, based at least on the update at the second time step, compute a likelihood of hazard occupancy for the portion at the second time step using the first hazard indication and the second hazard indication ([0117] “The RGB-D-V outputs at 168 a description of the detected objects including a level of confidence”, Fig. 13: enhanced object including a level of confidence from raw data fusion 102 [Wingdings font/0xE0] detection merger 182 [Wingdings font/0xE0] tracker [Wingdings font/0xE0] occupancy grid 180).
Regarding claim 4, Nehmadi discloses:
wherein the second time step corresponds to a fusion timestep ([0147] “The system of FIG. 1A is configured such that different modalities operate asynchronously in their own time domains, and they are synchronized to the output time.”), and the second location corresponds to the second hazard indication at a sensor-capture time associated with the second sensor data ([0009] “each sensor module operating according to a respective sensor modality and according to a respective time domain, each sensor module including a sensor input for receiving sensor data from a sensor of the respective modality and a software-based perception entity processing the sensor data to detect objects of interest in the sensor data in the respective time domain. The system further includes a sensor fusion module including an input for receiving asynchronous detection results from the plurality of sensor modules and a processing entity for synchronizing the detection results and fusing the synchronized detection results into a unified perception solution”).
Regarding claim 5, Nehmadi discloses:
wherein the ego-motion compensation includes transforming hazard indicators in the first instance of the segmented representation into a coordinate frame associated with the second time step based at least on one or more ego-motion vectors associated with the ego-machine ([0062] “The ego-motion information allows accurately locating the vehicle on each frame and comprises a motion vector and trajectory of the host vehicle”, [0071] “The purpose of the ego-motion module 58 is to localize the vehicle on each frame of the 3D point cloud developed by fusing the camera, lidar and radar data. The ego-motion algorithms use as input consecutive data frames from the sensors, camera and lidar, among others to estimate motion between consecutive frames and thus derive a motion vector and a trajectory for the host car.”).
Regarding claim 8, Nehmadi discloses:
wherein the segmented representation includes an occupancy grid corresponding to the environment (Fig. 13: occupancy grid 180), and the portion corresponds to a cell of the occupancy grid (see occupancy grid in Fig. 30).
Regarding claim 9, Nehmadi discloses:
wherein the at least one processor is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine ([0054] “With specific reference to FIG. 1 , the block diagram illustrates a system 10 for performing environmental perception”);
a system for performing simulation operations;
a system for performing synthetic data generation;
a system for multi-dimensional collaborative content creation;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Regarding claim 10, Nehmadi discloses:
A method comprising:
determining that a location corresponding to a first hazard indication at a first time corresponds to an area in a segmented representation of an environment of an ego-machine at a second time ([0072] “The ego-motion module 58 has an ego-motion estimation functional block 59, which receives at an input 60 a previously calculated 3D map from the sensor fusion operation”, [0073] “It is preferred to compare data frames of the 3D map, as in addition to the image data, the 3D map also conveys distance information for each point in relation to the host vehicle, thus allowing to more precisely compute displacement between points in successive data frames.”);
responsive to the determination, computing a likelihood of hazard occupancy for the area in the segmented representation using the first hazard indication and a second hazard indication associated with the area at the second time ([0117] “The RGB-D-V outputs at 168 a description of the detected objects including a level of confidence”, Fig. 13: enhanced object including a level of confidence from raw data fusion 102 [Wingdings font/0xE0] detection merger 182 [Wingdings font/0xE0] tracker [Wingdings font/0xE0] occupancy grid 180); and
performing one or more operations for the ego-machine based at least on the likelihood ([0080] “The Bayesian filter functional block 88 uses a motion model 98 that describes spatial relationships between objects in the scene and the vehicle, allowing to create a representation of the vehicle in its driving environment. The Bayesian filter functional block 88 outputs localization information which conveys the representation of the vehicle in its driving environment including the objects of interest in the vicinity of the vehicles including road boundaries and landmarks.”).
Regarding claim 11, Nehmadi discloses:
further comprising:
at a first time step, updating the area of a first instance of the segmented representation based at least on determining that the location corresponds to the area of the first instance of the segmented representation ([0124] “the sensor fusion module 100 includes an occupancy grid functional block 180 which receives the geometric occupancy grid information from the raw data fusion functional block 102 and ego-motion information from the ego-motion merger functional block 64”, Fig. 30 shows 2 frames of occupancy grid, each frame includes obstacle, static/dynamic radar target, previous frame may correspond to the claimed ‘first time step’”), and
the determining that the location corresponds to the area at the second time includes, at a second time step, updating the area of a second instance of the segmented representation based at least on determining, using the first instance of the segmented representation after ego-motion compensation ([0072] “The ego-motion module 58 has an ego-motion estimation functional block 59, which receives at an input 60 a previously calculated 3D map from the sensor fusion operation”, [0073] “It is preferred to compare data frames of the 3D map, as in addition to the image data, the 3D map also conveys distance information for each point in relation to the host vehicle, thus allowing to more precisely compute displacement between points in successive data frames.”), that a second location corresponding to the second hazard indication corresponds to the area (Fig. 30 shows 2 frames of occupancy grid, each frame includes obstacle, static/dynamic radar target, later frame may correspond to the claimed ‘second time step’”).
Regarding claim 13, Nehmadi discloses:
wherein the second time corresponds to a fusion timestep ([0147] “The system of FIG. 1A is configured such that different modalities operate asynchronously in their own time domains, and they are synchronized to the output time.”), and the location corresponds to the first hazard indication at a sensor-capture time associated with the first time ([0009] “each sensor module operating according to a respective sensor modality and according to a respective time domain, each sensor module including a sensor input for receiving sensor data from a sensor of the respective modality and a software-based perception entity processing the sensor data to detect objects of interest in the sensor data in the respective time domain. The system further includes a sensor fusion module including an input for receiving asynchronous detection results from the plurality of sensor modules and a processing entity for synchronizing the detection results and fusing the synchronized detection results into a unified perception solution”).
Regarding claim 14, Nehmadi discloses:
wherein the determination includes transforming the location in a first instance of the segmented representation into a coordinate frame associated with the second time based at least on one or more ego-motion vectors associated with the ego-machine ([0062] “The ego-motion information allows accurately locating the vehicle on each frame and comprises a motion vector and trajectory of the host vehicle”, [0071] “The purpose of the ego-motion module 58 is to localize the vehicle on each frame of the 3D point cloud developed by fusing the camera, lidar and radar data. The ego-motion algorithms use as input consecutive data frames from the sensors, camera and lidar, among others to estimate motion between consecutive frames and thus derive a motion vector and a trajectory for the host car.”).
Regarding claim 16, Nehmadi discloses:
wherein the segmented representation includes an occupancy grid corresponding to the environment (Fig. 13: occupancy grid 180), and the area corresponds to a cell of the occupancy grid (see occupancy grid in Fig. 30).
Regarding claim 17, Nehmadi discloses:
A system (Fig. 1: system 10) comprising:
one or more processors to:
cause performance of one or more planning, navigation, or control operations of an ego-machine based at least on a hazard indicated in a representation of an environment ([0080] “The Bayesian filter functional block 88 uses a motion model 98 that describes spatial relationships between objects in the scene and the vehicle, allowing to create a representation of the vehicle in its driving environment. The Bayesian filter functional block 88 outputs localization information which conveys the representation of the vehicle in its driving environment including the objects of interest in the vicinity of the vehicles including road boundaries and landmarks.”),
the representation of the environment generated using a first sensor fusion output corresponding to a current time step and one or more second sensor fusion outputs corresponding to one or more prior time steps after compensating for motion of the ego-machine ([0072] “The ego-motion module 58 has an ego-motion estimation functional block 59, which receives at an input 60 a previously calculated 3D map from the sensor fusion operation”, [0073] “It is preferred to compare data frames of the 3D map, as in addition to the image data, the 3D map also conveys distance information for each point in relation to the host vehicle, thus allowing to more precisely compute displacement between points in successive data frames.”).
Regarding claim 18, Nehmadi discloses:
wherein the one or more processors are to compute a likelihood of hazard occupancy for the hazard using one or more hazard indications propagated to the representation of the environment for the current time step ([0117] “The RGB-D-V outputs at 168 a description of the detected objects including a level of confidence”, Fig. 13: enhanced object including a level of confidence from raw data fusion 102 [Wingdings font/0xE0] detection merger 182 [Wingdings font/0xE0] tracker [Wingdings font/0xE0] occupancy grid 180) based at least on the compensating for the motion of the ego-machine ([0072] “The ego-motion module 58 has an ego-motion estimation functional block 59, which receives at an input 60 a previously calculated 3D map from the sensor fusion operation”, [0073] “It is preferred to compare data frames of the 3D map, as in addition to the image data, the 3D map also conveys distance information for each point in relation to the host vehicle, thus allowing to more precisely compute displacement between points in successive data frames.”), and the performance is based at least on the likelihood of hazard occupancy ([0080] “The Bayesian filter functional block 88 uses a motion model 98 that describes spatial relationships between objects in the scene and the vehicle, allowing to create a representation of the vehicle in its driving environment. The Bayesian filter functional block 88 outputs localization information which conveys the representation of the vehicle in its driving environment including the objects of interest in the vicinity of the vehicles including road boundaries and landmarks.”).
Regarding claim 20, Nehmadi discloses:
wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine ([0054] “With specific reference to FIG. 1 , the block diagram illustrates a system 10 for performing environmental perception”);
a system for performing simulation operations;
a system for performing synthetic data generation;
a system for multi-dimensional collaborative content creation;
a system for performing deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3, 12 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Nehmadi in view of Slutsky et al. (US 20190353778 A1; hereinafter Slutsky).
Regarding claim 3, Nehmadi discloses the at least one processor of claim 1, but fails to disclose:
wherein the one or more circuits are further to, based at least on the update at the second time step, update a likelihood of hazard occupancy for the portion at the first time step to correspond to the second time step using the first hazard indication and the second hazard indication.
However, Slutsky discloses:
wherein the one or more circuits are further to, based at least on the update at the second time step, update a likelihood of hazard occupancy for the portion at the first time step to correspond to the second time step using the first hazard indication and the second hazard indication ([0044] “The negative ISM induces negative changes in the occupancy likelihood in cells of the occupancy grid for which there is a lack of detections in the corresponding direction of arrival (no-detection direction). Therefore, a probability value in a selected bin of the occupancy grid due to the presence of a detection in the direction corresponding to the bin during a previous time step can updated and reduced when no detections are occurring in the direction during more recent time steps”).
Nehmadi and Slutsky are considered to be analogous to the claimed invention because they are in the same field of occupancy grids for object detection confidence. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s occupancy grids to further incorporate Slutsky’s occupancy grids for the advantage of updating occupancy grid confidence value at the previous time step which results in safe and accurate vehicle navigation.
Regarding claim 12, Nehmadi discloses the method of claim 10, but fails to disclose:
wherein the computation of the likelihood includes updating the likelihood of hazard occupancy for the area at a first time step to correspond to a second time step.
However, Slutsky discloses:
wherein the computation of the likelihood includes updating the likelihood of hazard occupancy for the area at a first time step to correspond to a second time step ([0044] “The negative ISM induces negative changes in the occupancy likelihood in cells of the occupancy grid for which there is a lack of detections in the corresponding direction of arrival (no-detection direction). Therefore, a probability value in a selected bin of the occupancy grid due to the presence of a detection in the direction corresponding to the bin during a previous time step can updated and reduced when no detections are occurring in the direction during more recent time steps”).
Nehmadi and Slutsky are considered to be analogous to the claimed invention because they are in the same field of occupancy grids for object detection confidence. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s occupancy grids to further incorporate Slutsky’s occupancy grids for the advantage of updating occupancy grid confidence value at the previous time step which results in safe and accurate vehicle navigation.
Regarding claim 19, Nehmadi discloses the system of claim 17, but fails to disclose:
wherein the one or more processors are to update a likelihood of hazard occupancy for a portion of the representation associated with at least one time step of the one or more prior time steps to correspond to the current time step based at least on the compensating for the motion of the ego-machine, and the performance is based at least on the updated likelihood of hazard occupancy.
However, Slutsky discloses:
wherein the one or more processors are to update a likelihood of hazard occupancy for a portion of the representation associated with at least one time step of the one or more prior time steps to correspond to the current time step ([0044] “The negative ISM induces negative changes in the occupancy likelihood in cells of the occupancy grid for which there is a lack of detections in the corresponding direction of arrival (no-detection direction). Therefore, a probability value in a selected bin of the occupancy grid due to the presence of a detection in the direction corresponding to the bin during a previous time step can updated and reduced when no detections are occurring in the direction during more recent time steps”) based at least on the compensating for the motion of the ego-machine (Fig. 8: self-motion estimation module 808 [Wingdings font/0xE0] one or more detections 806 + resampled occupancy grid 824/updated static occupancy grid 826), and the performance is based at least on the updated likelihood of hazard occupancy ([0006] “the processor is configured to navigate the vehicle with respect to the object based on the mapping of the environment”).
Nehmadi and Slutsky are considered to be analogous to the claimed invention because they are in the same field of occupancy grids for object detection confidence. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s occupancy grids to further incorporate Slutsky’s occupancy grids for the advantage of updating occupancy grid confidence value at the previous time step which results in safe and accurate vehicle navigation.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Nehmadi in view of Buerkle et al. (US 20200225622 A1; hereinafter Buerkle).
Regarding claim 6, Nehmadi discloses the at least one processor of claim 1, but fails to disclose:
wherein the portion of the second instance represents a smaller area of the environment than the portion of the first instance based at least on the portion of the second instance corresponding to a closer location relative to the ego-machine than the portion of the first instance.
However, Buerkle discloses:
wherein the portion of the second instance represents a smaller area of the environment than the portion of the first instance based at least on the portion of the second instance corresponding to a closer location relative to the ego-machine than the portion of the first instance ([0069] “the occupancy grid may include a higher cell density/higher resolution/lower cell size in an area surrounding or near to the vehicle, and the occupancy grid may include a lower cell density/lower resolution/greater cell size in an area farther away from the vehicle”).
Nehmadi and Buerkle are considered to be analogous to the claimed invention because they are in the same field of occupancy grids for object detection confidence. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s occupancy grids to further incorporate Buerkle’s occupancy grids for the advantage of adjusting occupancy cell size which results in reducing required cells and improving processing efficiency.
Claims 7 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Nehmadi in view of Abt et al. (US 20200353942 A1; hereinafter Abt).
Regarding claim 7, Nehmadi discloses the at least one processor of claim 1, but fails to disclose:
wherein the update includes combining a first probability distribution of hazard occupancy for the first hazard indicator with a second probability distribution of hazard occupancy for the second hazard indicator to determine an aggregated likelihood of hazard occupancy for the portion of the second instance.
However, Abt discloses:
wherein the update includes combining a first probability distribution of hazard occupancy for the first hazard indicator with a second probability distribution of hazard occupancy for the second hazard indicator to determine an aggregated likelihood of hazard occupancy for the portion of the second instance (Fig. 4, [0047] occupancy probability 402, 404, and 406 are fused together).
Nehmadi and Abt are considered to be analogous to the claimed invention because they are in the same field of occupancy grids for object detection confidence. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s occupancy grids to further incorporate Abt’s occupancy grids for the advantage of aggregate occupancy probability which results in efficient and accurate object detection (Abt’s [0007]).
Regarding claim 15, Nehmadi discloses the method of claim 10, but fails to disclose:
wherein the computation of the likelihood includes combining a first probability distribution of hazard occupancy for the first hazard indicator with a second probability distribution of hazard occupancy for the second hazard indicator to determine an aggregated likelihood of hazard occupancy for the area.
However, Abt discloses:
wherein the computation of the likelihood includes combining a first probability distribution of hazard occupancy for the first hazard indicator with a second probability distribution of hazard occupancy for the second hazard indicator to determine an aggregated likelihood of hazard occupancy for the area (Fig. 4, [0047] occupancy probability 402, 404, and 406 are fused together).
Nehmadi and Abt are considered to be analogous to the claimed invention because they are in the same field of occupancy grids for object detection confidence. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Nehmadi’s occupancy grids to further incorporate Abt’s occupancy grids for the advantage of aggregate occupancy probability which results in efficient and accurate object detection (Abt’s [0007]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Trojahner (US 20230056589 A1) discloses a confidence level associated with the determination of a cell status can increase over time (e.g., by accumulating additional observations).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAYSUN WU whose telephone number is (571)272-1528. The examiner can normally be reached Monday-Friday 8AM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached on (571)272-7298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAYSUN WU/Examiner, Art Unit 3665
/DONALD J WALLACE/Primary Examiner, Art Unit 3665
