DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 8 is objected to because of the following informalities:
a.”… within the polygon being associated an area…” should be read as “…within the polygon being associated with an area…”
Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 9, 16, and 17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kuniaki (JP 7344743 B2).
Regarding claim 9, Kuniaki teaches a system comprising:
one or more processors to(Kuniaki, page 4 line 25, different microcomputers ):
generate a two-dimensional (2D) occupancy map associated with an environment (Kuniaki, page 10 line 22, in step S16, the occupancy grid map generation unit 16 projects the normal distribution of the covariance Σ in the three-dimensional space shown in the short-term accumulated distance measurement data 14 onto the road surface, and Calculate the occupancy value C .sub.i at x .sub.i . Then, the occupancy grid map generation unit 16 generates a two-dimensional short-term occupancy grid map 17 by performing a binarization process on the occupancy value C .sub.i. ), the 2D occupancy map indicating one or more first areas of the environment that are occupied and one or more second areas of the environment that are unoccupied(Kuniaki, page 5 line 17, the occupancy grid map is a map in which the presence of obstacles in a predetermined area is expressed as a set of grids on a road surface where obstacles such as structures and moving objects are present. In this occupied grid map, each grid is shown as an occupied grid occupied by an obstacle or an unoccupied grid not occupied by an obstacle);
determine, based at least on projecting one or more rays with regard to the 2D occupancy map, one or more points within the 2D occupancy map that are associated with the one or more first areas(Kuniaki, page 10 line 41, the unoccupied grid map generation unit 19 determines the presence or absence of an occupied grid in each of the ray projection paths. Kuniaki, page 12 line 17, the unoccupied grid map generation unit 19 virtually generates a 360° two-dimensional area centered on the distance measurement sensor 12 in the short-term occupancy grid map 17. 12, a ray 81 is projected while rotating. Kuniaki, page 10 line 33, in step S17, the unoccupied grid map generation unit 19 performs ray tracing processing on each of the short-term unoccupied grid map 17 and long-term unoccupied grid map 18 detected in step S16, and An occupancy grid map 21 is generated);
generate, based at least on the one or more points within the 2D occupancy map, one or more occluded areas of the environment (Kuniaki, page 14 line 33, the occupancy grid map generation unit 16 superimposes the occupancy states shown in each of the short-term sub-grid maps generated by the ranging data storage unit 13 for each grid on a two-dimensional plane, thereby generating short-term occupancy); and
cause, based at least on the one or more occluded areas of the environment, a machine to perform one or more operations(Kuniaki, page 5 line 49, the output unit 23 outputs the grid map generated by the evaluation unit 22 to an automatic driving system or the like outside the driving support system 100. For example, an automated driving system uses the received grid map to assist the vehicle in avoiding obstacles).
Regarding claim 16, Kuniaki teaches the system of claim 9(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16)), wherein the one or more processors are further to:
update the 2D occupancy map to indicate one or more third areas that are occluded(Kuniaki discloses changing map based on occupancy state to determine the final grid map, which corresponds to the updating of a map. Kuniaki, page 5 line 36 , in the long-term occupancy grid map 18, the evaluation unit 22 outputs, as a final grid map, a grid that has been evaluated as an unoccupied grid among the occupied grids in the long-term occupancy grid map 18 and has been changed to an unoccupied grid. output to section 23. In this way, in the long-term occupancy grid map 18, the occupancy grid caused by the past presence of a moving object can be deleted, so that the accuracy of determining the occupancy grid in the final grid map can be improved.),
the one or more third areas of the 2D occupancy map corresponding to the one or more occluded areas of the environment(Kuniaki, page 10 line 24, the occupancy grid map generation unit 16 generates a two-dimensional short-term occupancy grid map 17 by performing a binarization process on the occupancy value C .sub.i.),
wherein the machine is caused to perform the one or more operations based at least on the 2D occupancy map as updated (Kuniaki, page 5 line 49, the output unit 23 outputs the grid map generated by the evaluation unit 22 to an automatic driving system or the like outside the driving support system 100. For example, an automated driving system uses the received grid map to assist the vehicle in avoiding obstacles).
Regarding claim 17, Kuniaki teaches the system of claim 9 (Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16)), wherein the system is comprised in at least one of:
a control system for an autonomous or semi-autonomous machine(Kuniaki, page 5 line 49, the output unit 23 outputs the grid map generated by the evaluation unit 22 to an automatic driving system or the like outside the driving support system 100. For example, an automated driving system uses the received grid map to assist the vehicle in avoiding obstacles );
a perception system for an autonomous or semi-autonomous machine;
a system for performing one or more simulation operations;
a system for performing one or more digital twin operations;
a system for performing one or more light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing one or more wireless cellular transmissions using a wireless cellular network;
a system that provides one or more cloud gaming applications;
a system for performing one or more deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing one or more generative AI operations;
a system for performing one or more conversational AI operations;
a system for performing operations using one or more large language models (LLMs);
a system for performing operations using one or more vision language models (VLMs);
a system for performing operations using one or more multi-modal language models (MMLMs);
a system for performing operations using one or more vision-language-action (VLA) models;
a system for performing one or more conversational AI operations;
a system for performing one or more synthetic data generation operations;
a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
systems using or deploying one or more inference microservices;
systems that incorporate deploy one or more machine learning models in a service or microservice along with an OS-level virtualization package (e.g., a container);
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Claims 18 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Fasola (US 20230410423 A1).
Regarding claim 18, Fasola teaches one or more processors comprising processing circuitry(Fasola, paragraph 16, one or more microprocessors, of one or more computers) to:
determine, based at least on sensor data obtained using one or more sensors of a machine, one or more voxels within an environment that are occupied(Fasola, paragraph 59, It will be recognized that each frame of data comprising the LIDAR point cloud may be ray traced and processed as described in greater detail below to generate a 3D occupancy grid, as well as a corresponding 2D occupancy grid, for the autonomous vehicle for each frame and that, by comparing cells of two 3D occupancy grids created from successive frames of LIDAR point cloud data corresponding to the same location in space, a determination may be made as to whether motion has occurred (e.g., occupancy characterization of a voxel associated with a particular location in space transitions from occupied to freespace or from freespace to occupied). A speed with which the detected motion occurs may also be estimated based on the time that has elapsed between the two frames);
generate, based at least on the one or more voxels, an occupancy map indicating one or more two-dimensional (2D) areas within the environment that are occupied(Fasola, paragraph 59, each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location. Fasola, paragraph 72, a 2D occupancy grid is created from a 3D occupancy grid for a current frame of point cloud data. In one embodiment, the 2D motion grid may be created by assigning an occupancy characterization to each cell of the 2D grid that represents the collective occupancy characteristics of a column of the 3D occupancy grid corresponding to the cell of the 2D occupancy grid);
determine, based at least on performing ray tracing with regard to the occupancy map, one or more occluded areas of the environment(Fasola, paragraph 59, It will be recognized that each frame of data comprising the LIDAR point cloud may be ray traced and processed as described in greater detail below to generate a 3D occupancy grid, as well as a corresponding 2D occupancy grid, for the autonomous vehicle for each frame and that, by comparing cells of two 3D occupancy grids created from successive frames of LIDAR point cloud data corresponding to the same location in space, a determination may be made as to whether motion has occurred (e.g., occupancy characterization of a voxel associated with a particular location in space transitions from occupied to freespace or from freespace to occupied). A speed with which the detected motion occurs may also be estimated based on the time that has elapsed between the two frames ); and
cause a machine to perform one or more operations based at least on the one or more occluded areas(Fasola, paragraph 13, an output of the 3D motion grid system may include information regarding occupancy and motion of the various grid cells (which in the 3D case may also be referred to herein as “voxels”), which information may be advantageously used by downstream consumers (e.g., a collision imminent detection (CID) node) to identify objects and their direction and approximate speed of travel and to assist the autonomous vehicle in responding to such detected objects).
Regarding claim 20, Fasola teaches the one or more processors of claim 18(Fasola, paragraph 59, each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location), wherein the one or more processors are comprised in at least one of:
a control system for an autonomous or semi-autonomous machine(Fasola, paragraph 13, an output of the 3D motion grid system may include information regarding occupancy and motion of the various grid cells (which in the 3D case may also be referred to herein as “voxels”)…to identify objects and their direction and approximate speed of travel and to assist the autonomous vehicle in responding to such detected objects);
a perception system for an autonomous or semi-autonomous machine;
a system for performing one or more simulation operations;
a system for performing one or more digital twin operations;
a system for performing one or more light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing one or more wireless cellular transmissions using a wireless cellular network; a system that provides one or more cloud gaming applications;
a system for performing one or more deep learning operations;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing one or more generative AI operations;
a system for performing one or more conversational AI operations;
a system for performing operations using one or more large language models (LLMs);
a system for performing operations using one or more vision language models (VLMs);
a system for performing operations using one or more multi-modal language models (MMLMs);
a system for performing operations using one or more vision-language-action (VLA) models;
a system for performing one or more conversational AI operations;
a system for performing one or more synthetic data generation operations;
a system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
systems using or deploying one or more inference microservices;
systems that incorporate deploy one or more machine learning models in a service or microservice along with an OS-level virtualization package (e.g., a container);
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1,2, 6, 10, 12, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Kuniaki (JP 7344743 B2) in view of Burlina (US 20250076486 A1).
Regarding claim 1, Kuniaki teaches a method comprising:
generating, based at least on a three-dimensional (3D) occupancy map associated with an environment, a two-dimensional (2D) occupancy map associated with the environment (Kuniaki discloses an occupancy grid map generation unit that generates a two-dimensional short-term occupancy grid based on three-dimensional occupancy value. Kuniaki, page 10 line 22, in step S16, the occupancy grid map generation unit 16 projects the normal distribution of the covariance Σ in the three-dimensional space shown in the short-term accumulated distance measurement data 14 onto the road surface, and Calculate the occupancy value C .sub.i at x .sub.i . Then, the occupancy grid map generation unit 16 generates a two-dimensional short-term occupancy grid map 17 by performing a binarization process on the occupancy value C .sub.i. Kuniaki, page 14 line 7, after the ranging data showing a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16).);
determining, based at least on projecting rays with regard to the 2D occupancy map,
points within the 2D occupancy map that are associated with occupied locations within the environment(Kuniaki discloses its unoccupied grid map generation unit determines the presence or absence of an occupied grid. Kuniaki, page 10 line 41, the unoccupied grid map generation unit 19 determines the presence or absence of an occupied grid in each of the ray projection paths. Kuniaki, page 12 line 17, the unoccupied grid map generation unit 19 virtually generates a 360° two-dimensional area centered on the distance measurement sensor 12 in the short-term occupancy grid map 17. 12, a ray 81 is projected while rotating. Kuniaki, page 10 line 33, in step S17, the unoccupied grid map generation unit 19 performs ray tracing processing on each of the short-term unoccupied grid map 17 and long-term unoccupied grid map 18 detected in step S16, and An occupancy grid map 21 is generated.);
causing, based at least on the one or more occluded areas, a machine to perform one or more planning, control, or navigation operations (Kuniaki, page 5 line 49, the output unit 23 outputs the grid map generated by the evaluation unit 22 to an automatic driving system or the like outside the driving support system 100. For example, an automated driving system uses the received grid map to assist the vehicle in avoiding obstacles).
Kuniaki specifically fails to disclose generating a polygon based least on connecting the points within the 2D occupancy map; determining one or more occluded areas of the environment based at least on the polygon.
However, Burlina, which is in the same analogous art and that teaches about objection detection, discloses generating a polygon based least on connecting the points within the 2D occupancy map(Burlina discloses generating a bounding box for a detected object (occupancy) in an environment based on overlapping radar points (connected points).Furthermore, Burlina discusses the bounding box can be polygonal data. Burlina, paragraph 19, the object detection output data may include bounding shapes (e.g., 2D or 3D bounding boxes) and/or any combination of associated object attribute data (e.g., object classification, yaw, velocity, intent, etc.). In one example, a transformer model may generate output including an objectness head, a bounding box regression head, and a bounding box classification head, in which the output can be associated with or rendered onto a top-down representation and/or image domain representation of the environment. Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines. Burlina, paragraph 35, ML models within the object detection component 118 may be trained based on combined views of infrared data images and sets of overlapping radar points, and may output data indicative of object(s) detected within the combined sensor data view. In some examples, ML models within the object detection component 118 may be configured to receive combined infrared-radar sensor views and to output a set of bounding shapes 122 (e.g., bounding boxes, bounding contours, or other polygonal data));
determining one or more occluded areas of the environment based at least on the polygon (Burlina discusses the bounding shapes(polygons) may include detected objects, which corresponds to the occluded area. Burlina, paragraph 46, the bounding shapes 122 output from the infrared-radar fusion system 106 may include an object detection, object location, object size, object shape, object classification, object yaw, etc); and
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kuniaki with Burlina to generate a bounding box for detected objects in 2D environment based on overlapping radar points. By generating a bounding box (polygonal data) for the detected object, it is possible to output the shape and size representative of the detected object based on retrieved 2D data. (Burlina, paragraph 38, ML model generating an output indicative of presence of an object of the respective object class (and/or subclass), and a bounding shape 122 indicating the object size/extent and location within the environment, along with any number of object attributes and/or a probability or confidence score. In other examples, the ML model(s) used by the object detection component 118 may output a probability score at each pixel location of the input(s) representing a probability of the pixel location corresponding to each object class of the set of object classes (e.g., pedestrian, animal, etc.)).
Regarding claim 2, the combination of Kuniaki and Burlina teaches the method of claim 1(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16);Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines), further comprising generating, using one or more neural networks(Burlina, paragraph 19, the infrared-radar fusion system may include a machine-learned transformer model (or other deep neural network) trained to output combined sensor data views in the form of object detections, based on infrared and radar input data ) and based at least on fusing one or more first features associated with image data with one or more second features associated with RADAR data(Burlina, paragraph 11, the infrared-radar fusion system may determine and output object detections, which may include bounding shapes (e.g., bounding boxes or bounding contours) and/or other data representing objects detected or perceived based on the infrared and/or radar sensor data. ), the 3D occupancy map associated with the environment (Burlina, paragraph 16, an infrared-radar fusion system configured to generate combined (or fused) infrared-radar sensor views, and/or to perform object detections based on the fusion of infrared and radar data within the driving environment. Burlina, paragraph 50, Object detections 216 and 218 may include 2D or 3D extent data (e.g., size data) and a defined position within the environment, based on the combined infrared-radar data. Burlina, paragraph 98, the environment state may be a data structure that identifies object data (e.g., object position, area of environment occupied by object ).
Regarding claim 6, the combination of Kuniaki and Burlina teaches the method of claim 1(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16);Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines), wherein the determining the points within the 2D occupancy map that are associated with the occupied locations within the environment (Kuniaki, page 10 line 22, in step S16, the occupancy grid map generation unit 16 projects the normal distribution of the covariance Σ in the three-dimensional space shown in the short-term accumulated distance measurement data 14 onto the road surface, and Calculate the occupancy value C .sub.i at x .sub.i . Then, the occupancy grid map generation unit 16 generates a two-dimensional short-term occupancy grid map 17 by performing a binarization process on the occupancy value C .sub.i) comprises:
projecting the rays starting at a first location within the 2D occupancy map that is associated with the machine (Kuniaki , page 10 line 38, in the ray tracing process, in the occupancy grid map (short-term occupancy grid map 17 and long-term occupancy grid map 18), in a 360° area in the circumferential direction centered on the distance measurement sensor 12, the distance measurement sensor 12 is used as the starting point. A ray (line of sight), which is a virtual optical path, is sequentially rotated and projected. Kuniaki, page 12 line 17, the unoccupied grid map generation unit 19 virtually generates a 360° two-dimensional area centered on the distance measurement sensor 12 in the short-term occupancy grid map 17. 12, a ray 81 is projected while rotating. );
determining that the rays intersect with second locations within the 2D occupancy map that are indicated as being occluded, the second locations within the 2D occupancy map corresponding to the occupied locations within the environment(Burlina, paragraph 18, the infrared-radar fusion system may use ray casting and/or ray tracing to project the radar points onto the infrared data, by casting rays from the location of the radar sensor toward the 2D view of the infrared image, to determine the ray intersection points for each radar point relative to the 2D infrared image. Using a sensor data view including radar points projected onto infrared data, the infrared-radar fusion system may execute one or more trained models configured to detect, segment, classify, and/or track objects within the combined sensor data view. ); and
determining the points as being associated with the second locations(According to the present specification, the second location corresponds to an occupied location and points associated with it. Similarly, Burlina discloses radar points that are associated with detected objects. Burlina, paragraph 13, a first pipeline may determine object detections based on visible image data, a second independent pipeline may determine object detections based on lidar point cloud data, a third independent pipeline may determine object detections based on radar point cloud data, etc.).
Regarding claim 10, Kuniaki teaches the system of claim 9(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16)),
wherein the 2D occupancy map is generated based at least on the 3D occupancy map(Kuniaki discloses an occupancy grid map generation unit that generates a two-dimensional short-term occupancy grid based on three-dimensional occupancy value. Kuniaki, page 10 line 22, in step S16, the occupancy grid map generation unit 16 projects the normal distribution of the covariance Σ in the three-dimensional space shown in the short-term accumulated distance measurement data 14 onto the road surface, and Calculate the occupancy value C .sub.i at x .sub.i . Then, the occupancy grid map generation unit 16 generates a two-dimensional short-term occupancy grid map 17 by performing a binarization process on the occupancy value C .sub.i. Kuniaki, page 14 line 7, after the ranging data showing a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16)).
Kuniaki specifically fails to disclose a system wherein the one or more processors are further to:
generate, using one or more neural networks and based at least on fusing one or more first features associated with image data with one or more second features associated with RADAR data, a three-dimensional (3D) occupancy map associated with the environment.
However, Burlina, which is in the same analogous art and that teaches about objection detection, discloses a system wherein the one or more processors are further to:
generate, using one or more neural networks(Burlina, paragraph 19, the infrared-radar fusion system may include a machine-learned transformer model (or other deep neural network) trained to output combined sensor data views in the form of object detections, based on infrared and radar input data) and based at least on fusing one or more first features associated with image data with one or more second features associated with RADAR data(Burlina, paragraph 11, the infrared-radar fusion system may determine and output object detections, which may include bounding shapes (e.g., bounding boxes or bounding contours) and/or other data representing objects detected or perceived based on the infrared and/or radar sensor data.), a three-dimensional (3D) occupancy map associated with the environment(Burlina, paragraph 16, an infrared-radar fusion system configured to generate combined (or fused) infrared-radar sensor views, and/or to perform object detections based on the fusion of infrared and radar data within the driving environment. Burlina, paragraph 50, Object detections 216 and 218 may include 2D or 3D extent data (e.g., size data) and a defined position within the environment, based on the combined infrared-radar data. Burlina, paragraph 98, the environment state may be a data structure that identifies object data (e.g., object position, area of environment occupied by object).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kuniaki with Burlina to detect three-dimensional objects (occupancy) based on neural network and sensor data fusion. By fusing data different sensor data, it is possible to accurately detect and analyze heat-emitting objects within low-visibility areas. (Burlina, paragraph 16, by combining the infrared and radar data at an early stage in the object detection process, the infrared-radar fusion system can more accurately detect and analyze heat-emitting objects (e.g., animals, pedestrians, etc.) within low-visibility areas).
Regarding claim 12, Kuniaki teaches the system of claim 9 (Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16)), wherein:
the one or more points include a plurality of points(Burlina, paragraph 10, an infrared-radar fusion system may project radar points ); and
the one or more processors (Burlina, paragraph 90, processors )are further to:
determine a polygon by connecting the plurality of points(Burlina discloses generating a bounding box for a detected object (occupancy) in an environment based on overlapping radar points (connected points).Furthermore, Burlina discusses the bounding box can be polygonal data. Burlina, paragraph 19, the object detection output data may include bounding shapes (e.g., 2D or 3D bounding boxes) and/or any combination of associated object attribute data (e.g., object classification, yaw, velocity, intent, etc.). In one example, a transformer model may generate output including an objectness head, a bounding box regression head, and a bounding box classification head, in which the output can be associated with or rendered onto a top-down representation and/or image domain representation of the environment. Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines. Burlina, paragraph 35, ML models within the object detection component 118 may be trained based on combined views of infrared data images and sets of overlapping radar points, and may output data indicative of object(s) detected within the combined sensor data view. In some examples, ML models within the object detection component 118 may be configured to receive combined infrared-radar sensor views and to output a set of bounding shapes 122 (e.g., bounding boxes, bounding contours, or other polygonal data)); and determine the one or more occluded area of the environment based at least on the polygon(Burlina discusses the bounding shapes(polygons) may include detected objects, which corresponds to the occluded area. Burlina, paragraph 46, the bounding shapes 122 output from the infrared-radar fusion system 106 may include an object detection, object location, object size, object shape, object classification, object yaw, etc).
Regarding claim 13, Kuniaki teaches the system of claim 9(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16)), wherein the determination of the one or more points within the 2D occupancy map that are associated with the one or more first areas comprises:
projecting the one or more rays starting at a first location within the 2D occupancy map that is associated with the machine(Kuniaki , page 10 line 38, in the ray tracing process, in the occupancy grid map (short-term occupancy grid map 17 and long-term occupancy grid map 18), in a 360° area in the circumferential direction centered on the distance measurement sensor 12, the distance measurement sensor 12 is used as the starting point. A ray (line of sight), which is a virtual optical path, is sequentially rotated and projected. Kuniaki, page 12 line 17, the unoccupied grid map generation unit 19 virtually generates a 360° two-dimensional area centered on the distance measurement sensor 12 in the short-term occupancy grid map 17. 12, a ray 81 is projected while rotating); and
determining that the one or more rays intersect with one or more second locations within the 2D occupancy map that are associated with the one or more first areas of the environment(Burlina, paragraph 18, the infrared-radar fusion system may use ray casting and/or ray tracing to project the radar points onto the infrared data, by casting rays from the location of the radar sensor toward the 2D view of the infrared image, to determine the ray intersection points for each radar point relative to the 2D infrared image. Using a sensor data view including radar points projected onto infrared data, the infrared-radar fusion system may execute one or more trained models configured to detect, segment, classify, and/or track objects within the combined sensor data view. ); and
determining the one or more points as being associated with the one or more second locations(According to the present specification, the second location corresponds to an occupied location and points associated with it. Similarly, Burlina discloses radar points that are associated with detected objects. Burlina, paragraph 13, a first pipeline may determine object detections based on visible image data, a second independent pipeline may determine object detections based on lidar point cloud data, a third independent pipeline may determine object detections based on radar point cloud data, etc.).
Claims 3-5 are rejected under 35 U.S.C. 103 as being unpatentable over Kuniaki (JP 7344743 B2) in view of Burlina (US 20250076486 A1) in further view of Fasola (US 20230410423 A1).
Regarding claim 3, the combination of Kuniaki and Burlina teaches the method of claim 1(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16);Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines), generating the 2D occupancy map to indicate whether the 2D areas are occupied(Kuniaki, page 10 line 24, the occupancy grid map generation unit 16 generates a two-dimensional short-term occupancy grid map 17 by performing a binarization process on the occupancy value C .sub.i. Kuniaki, page 14 line 33, the occupancy grid map generation unit 16 superimposes the occupancy states shown in each of the short-term sub-grid maps generated by the ranging data storage unit 13 for each grid on a two-dimensional plane, thereby generating short-term occupancy ).
The combination of Kuniaki and Burlina specifically fails to disclose a method wherein:
the 3D occupancy map indicates whether voxels associated with the environment are occupied; the generating the 2D occupancy map comprises:
determining, based at least on whether the voxels are occupied, whether 2D areas of the environment are occupied; generating the 2D occupancy map to indicate whether the 2D areas are occupied;
However, Fasola, which is in the same analogous art and that teaches about three-dimensional (3D) motion grid, discloses a method wherein:
the 3D occupancy map indicates whether voxels associated with the environment are occupied(Fasola, paragraph 13, an output of the 3D motion grid system may include information regarding occupancy and motion of the various grid cells (which in the 3D case may also be referred to herein as “voxels”) ); and
the generating the 2D occupancy map comprises:
determining, based at least on whether the voxels are occupied, whether 2D areas of the environment are occupied (Fasola discloses each voxels corresponds to a 2D occupancy that indicate occupancy status of in a 2D grid. Fasola, paragraph 59, each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location. Fasola, paragraph 72, a 2D occupancy grid is created from a 3D occupancy grid for a current frame of point cloud data. In one embodiment, the 2D motion grid may be created by assigning an occupancy characterization to each cell of the 2D grid that represents the collective occupancy characteristics of a column of the 3D occupancy grid corresponding to the cell of the 2D occupancy grid); and
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kuniaki and Burlina with Fasola to determine 2D occupancy from occupancy data of voxels. By determining the 2D occupancy based on the voxels, it is possible to characterize each voxel as free space or occupied in a 2D occupancy grid.( Fasola, paragraph 72, if a first voxel in the corresponding column is characterized as ground and the remaining voxels in the corresponding column are characterized as freespace, then the cell of the 2D occupancy grid may accurately be characterized as freespace. Expanding on the previous example, if the uppermost voxel of the aforementioned column is characterized as overhang, then once again the cell of the 2D occupancy grid may accurately be characterized as freespace. In contrast, if instead of being characterized as freespace, a certain minimum number of the voxels disposed between the bottom-most and upper-most voxels in the column are characterized as occupied, then the corresponding cell of the 2D occupancy grid may most accurately be characterized as occupied).
Regarding claim 4, the combination of Kuniaki, Burlina, and Fasola teaches the method of claim 3(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16);Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines; Fasola, paragraph 59, each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location), wherein the determining whether a 2D area of the 2D areas is occupied comprises:
identifying a portion of the voxels that are associated with the 2D area(As discussed above, Fasola discloses each voxels are association with 2D occupancy, which corresponds to the identification of portion of voxels associated with the 2D area. Fasola, paragraph 59 Each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location);
determining whether one or more voxels of the portion of the voxels are occupied (Fasola, paragraph 59, Each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location. For example, a voxel may be indicated as being occupied, freespace, unknown (perhaps occluded), ground (corresponding to the actual surface on which the vehicle is driving if that surface is not otherwise occupied));
and determining whether the 2D area is occupied based at least on whether the one or more voxels are occupied(Fasola, paragraph 72, if…a certain minimum number of the voxels disposed between the bottom-most and upper-most voxels in the column are characterized as occupied, then the corresponding cell of the 2D occupancy grid may most accurately be characterized as occupied.).
Regarding claim 5, the combination of Kuniaki, Burlina, and Fasola teaches the method of claim 3(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16);Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines; Fasola, paragraph 59, each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location), wherein the determining whether a 2D area of the 2D areas is occupied comprises:
identifying a portion of the voxels that are associated with the 2D area (Fasola, paragraph 59 Each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location) and at least a threshold height above a navigation surface (Fasola discloses a particular height of a grid from the ground level, which correspond to the threshold height above a navigation surface. Fasola, paragraph 58, the grid 610 is defined as having a particular height from ground level 616 and “travels” with the autonomous vehicle 612. In an example embodiment, the height of the grid can be 2 m-4 m and in some embodiments is one of 1.8 m, 2.0 m, 2.2 m, 2.4 m, 2.6 m, 2.8 m, 3.0 m, 3.2 m, . . . 4.2 m, etc.). The height of the voxels may be selected based on the general height of obstacles and the relevant spatial region. A corresponding number of voxels will be used in the grid, depending on the height of the grid and the height of the voxels);
determining whether one or more voxels of the portion of the voxels are occupied (Fasola, paragraph 59, Each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location. For example, a voxel may be indicated as being occupied, freespace, unknown (perhaps occluded), ground (corresponding to the actual surface on which the vehicle is driving if that surface is not otherwise occupied));
and determining whether the 2D area is occupied based at least on whether the one or more voxels are occupied (Fasola, paragraph 72, if…a certain minimum number of the voxels disposed between the bottom-most and upper-most voxels in the column are characterized as occupied, then the corresponding cell of the 2D occupancy grid may most accurately be characterized as occupied.).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Kuniaki (JP 7344743 B2) in view of Burlina (US 20250076486 A1) in further view of Long (EP 4033324 A1).
Regarding claim 7, the combination of Kuniaki and Burlina teaches the method of claim 1(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16);Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines; Fasola, paragraph 59, each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location),
The combination of Kuniaki and Burlina specifically fails to disclose a method wherein the rays are projected using an angle that is associated with one or more dimensions of the 2D occupancy map.
However, Long, which is in the same analogous art and teaches about a method and apparatus for sensing obstacle information, discloses a method wherein the rays are projected using an angle that is associated with one or more dimensions of the 2D occupancy map (Long, paragraph 12, taking a projection center of a body of the mobile robot on the carrying surface as a starting point to set at least one ray at equal angles, and taking a two-dimensional point on the ray that is closest to the starting point as a contour of the obstacle to obtain a two-dimensional point of the contour of the obstacle. Long, paragraph 53, the two-dimensional point(s) on these rays and are closest to the projection center of the robot body on the carrying surface are taken as the obstacle boundary, that is, the contour of the obstacle. If there is no two-dimensional point on the rays, it means that there is no obstacle in the directions of the rays. The smaller the angle between the rays is, then the more two-dimensional points of the contour of the obstacle may be obtained, and the more accurate the determined contour of the obstacle is ).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kuniaki and Burlina with Long to project ray at angle to obtain contour of an obstacle. By applying smaller angle of projection between rays, it is possible to determine a more accurate two-dimensional contour of an obstacle. (Long, paragraph 53, the smaller the angle between the rays is, then the more two-dimensional points of the contour of the obstacle may be obtained, and the more accurate the determined contour of the obstacle is).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Kuniaki (JP 7344743 B2) in view of Burlina (US 20250076486 A1) in further view of Haifeng (GB 2584839 A).
Regarding claim 8, the combination of Kuniaki and Burlina teaches the method of claim 1(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16);Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines; Fasola, paragraph 59, each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location),
The combination of Kuniaki and Burlina specifically fails to disclose a method further comprising:
generating an occlusion map, a first region of the occlusion map that is within the polygon being associated an area of the environment that is not occluded and one or more second regions of the occlusion map that are outside of the polygon being associated with the one or more occluded areas of the environment,
wherein the causing the machine to perform the one or more planning, control, or navigation operations is based at least on the occlusion map.
However, Haifeng, which is in the same analogous art and that teaches about mapping of an environment, discloses a method further comprising:
generating an occlusion map, a first region of the occlusion map that is within the polygon being associated an area of the environment that is not occluded (Haifeng discloses a navigable area within a bounding box, which corresponds to non-occluded area within a polygon. Haifeng, page 21 line 7, a bounding box surrounding the navigable area 502 or the navigable area 502 with a buffer of a few pixels either side of the navigable area 502 (to account for any inaccuracies during rotation or warping in the navigation map 500) may be obtained. The portion of the navigation map 500 within the bounding box may then be processed to identify the largest rectangles that will fit within the bounding box (but outside the navigable area 502).) and one or more second regions of the occlusion map that are outside of the polygon being associated with the one or more occluded areas of the environment (Haifeng further discloses a non-navigable area outside a boundary, which corresponds to the occluded area of the environment. Haifeng, page 21 line 19, the navigation map 500 of Figures 5 to 7 includes a non-navigable area 504a which is disconnected from the non-navigable area 504b surrounding the navigable area 502. This area 504a may be referred to as a disconnected non-navigable area 504a. A non-navigable area such as this may correspond to a region of the environment that is occupied by a moveable object…figure 8, which shows the navigation map 500 of Figure 7 after removal of the representation of the object (corresponding to the disconnected non-navigable area 504a which is disconnected from the boundary 514 of the environment),
wherein the causing the machine to perform the one or more planning, control, or navigation operations is based at least on the occlusion map(Haifeng, page 6 line 21, robotic device comprising the computing system according to the fourth aspect, one or more actuators to enable the robotic device to interact with the environment, and an interaction engine comprising at least one processor to control the one or more actuators, wherein the interaction engine is configured to use the map of the environment to interact with the environment).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kuniaki and Burlina with Haifeng to obtain a bounding box (polygon) that is navigable (non-occluded) area. By obtaining the navigable area, it is possible to accurately represents an underlying layout of an environment for an autonomous navigation of a vehicle (robot). (Haifeng, page 22 line 1, the navigation map 500 therefore more accurately represents an underlying layout of the environment. In other cases, though, a representation of such an object may be retained in the navigation map 500. Haifeng, page 9 line 30, this navigation data. may be used to update or generate a map of the environment, which may be used by the robotic device for subsequent navigation (which may be improved compared to its original navigation, e.g. by covering a greater proportion of the environment, or by more accurately avoiding collisions such as with internal walls within the environment)).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Kuniaki (JP 7344743 B2) in view of Fasola (US 20230410423 A1).
Regarding claim 11, Kuniaki teaches the system of claim 9(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16)),
Kuniaki specifically fails to disclose a system wherein the generation of the 2D occupancy map comprises: obtaining data indicating whether voxels associated with the environment are occupied;
determining that a first portion of the voxels associated with the one or more first areas are occupied and a second portion of the voxels associated with the one or more second areas are unoccupied; and
generating the 2D occupancy map to indicate the one or more first areas as being occupied and the one or more second areas as being unoccupied.
However, Fasola, which is in the same analogous art and that teaches about three-dimensional (3D) motion grid, discloses a system wherein the generation of the 2D occupancy map comprises: obtaining data indicating whether voxels associated with the environment are occupied(Fasola, paragraph 13, an output of the 3D motion grid system may include information regarding occupancy and motion of the various grid cells (which in the 3D case may also be referred to herein as “voxels”));
determining that a first portion of the voxels associated with the one or more first areas are occupied(Fasola, paragraph 72, if…a certain minimum number of the voxels disposed between the bottom-most and upper-most voxels in the column are characterized as occupied, then the corresponding cell of the 2D occupancy grid may most accurately be characterized as occupied) and a second portion of the voxels associated with the one or more second areas are unoccupied(Fasola, paragraph 59, Each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location. For example, a voxel may be indicated as being occupied, freespace, unknown (perhaps occluded), ground (corresponding to the actual surface on which the vehicle is driving if that surface is not otherwise occupied)); and
generating the 2D occupancy map to indicate the one or more first areas as being occupied (Fasola, paragraph 72, if…a certain minimum number of the voxels disposed between the bottom-most and upper-most voxels in the column are characterized as occupied, then the corresponding cell of the 2D occupancy grid may most accurately be characterized as occupied)and the one or more second areas as being unoccupied (Fasola, paragraph 59, Each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location. For example, a voxel may be indicated as being occupied, freespace, unknown (perhaps occluded), ground (corresponding to the actual surface on which the vehicle is driving if that surface is not otherwise occupied)).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kuniaki with Fasola to determine 2D occupancy from occupancy data of voxels. By determining the 2D occupancy based on the voxels, it is possible to characterize each voxel as free space or occupied in a 2D occupancy grid.( Fasola, paragraph 72, if a first voxel in the corresponding column is characterized as ground and the remaining voxels in the corresponding column are characterized as freespace, then the cell of the 2D occupancy grid may accurately be characterized as freespace. Expanding on the previous example, if the uppermost voxel of the aforementioned column is characterized as overhang, then once again the cell of the 2D occupancy grid may accurately be characterized as freespace. In contrast, if instead of being characterized as freespace, a certain minimum number of the voxels disposed between the bottom-most and upper-most voxels in the column are characterized as occupied, then the corresponding cell of the 2D occupancy grid may most accurately be characterized as occupied).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Kuniaki (JP 7344743 B2) in view of Long (EP 4033324 A1).
Regarding claim 14, Kuniaki teaches the system of claim 9(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16)),
Kuniaki specifically fails to disclose a system wherein the one or more rays include a plurality of rays, and wherein the plurality of rays are projected using an angle that is associated with one or more dimensions associated with the 2D occupancy map.
However, Long, which is in the same analogous art and teaches about a method and apparatus for sensing obstacle information, discloses a system wherein the one or more rays include a plurality of rays, and wherein the plurality of rays are projected using an angle that is associated with one or more dimensions associated with the 2D occupancy map(Long, paragraph 12, taking a projection center of a body of the mobile robot on the carrying surface as a starting point to set at least one ray at equal angles, and taking a two-dimensional point on the ray that is closest to the starting point as a contour of the obstacle to obtain a two-dimensional point of the contour of the obstacle. Long, paragraph 53, the two-dimensional point(s) on these rays and are closest to the projection center of the robot body on the carrying surface are taken as the obstacle boundary, that is, the contour of the obstacle. If there is no two-dimensional point on the rays, it means that there is no obstacle in the directions of the rays. The smaller the angle between the rays is, then the more two-dimensional points of the contour of the obstacle may be obtained, and the more accurate the determined contour of the obstacle is ).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kuniaki with Long to project ray at angle to obtain contour of an obstacle. By applying smaller angle of projection between rays, it is possible to determine a more accurate two-dimensional contour of an obstacle. (Long, paragraph 53, the smaller the angle between the rays is, then the more two-dimensional points of the contour of the obstacle may be obtained, and the more accurate the determined contour of the obstacle is).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Kuniaki (JP 7344743 B2) in view of Haifeng (GB 2584839 A).
Regarding claim 15, Kuniaki teaches the system of claim 9(Kuniaki, page 14 line 7, a normal distribution of covariance in the three-dimensional space are superimposed in the three-dimensional space (S14), the normal distribution of the covariance Σ shown in the superimposed ranging data is A two-dimensional occupancy grid map was generated by projecting the distribution values in the height direction onto the road surface (S16)),
Kuniaki specifically fails to disclose a system wherein the one or more processors are further to:
generate an occlusion map, a first region of the occlusion map that is within the one or more points being associated an area of the environment that is not occluded and one or more second regions of the occlusion map that are outside of the one or more points being associated with the one or more occluded areas of the environment,
wherein the machine is caused to perform the one or more operations based at least on the occlusion map.
However, Haifeng, which is in the same analogous art and that teaches about mapping of an environment, discloses a system wherein the one or more processors are further to:
generate an occlusion map, a first region of the occlusion map that is within the one or more points being associated an area of the environment that is not occluded (Haifeng discloses a navigable area within a bounding box, which corresponds to non-occluded area within the one or more points associated with the area of the environment. Haifeng, page 21 line 7, a bounding box surrounding the navigable area 502 or the navigable area 502 with a buffer of a few pixels either side of the navigable area 502 (to account for any inaccuracies during rotation or warping in the navigation map 500) may be obtained. The portion of the navigation map 500 within the bounding box may then be processed to identify the largest rectangles that will fit within the bounding box (but outside the navigable area 502))and one or more second regions of the occlusion map that are outside of the one or more points being associated with the one or more occluded areas of the environment(Haifeng further discloses a non-navigable area outside a boundary, which corresponds to the occluded area of the environment. Haifeng, page 21 line 19, the navigation map 500 of Figures 5 to 7 includes a non-navigable area 504a which is disconnected from the non-navigable area 504b surrounding the navigable area 502. This area 504a may be referred to as a disconnected non-navigable area 504a. A non-navigable area such as this may correspond to a region of the environment that is occupied by a moveable object…figure 8, which shows the navigation map 500 of Figure 7 after removal of the representation of the object (corresponding to the disconnected non-navigable area 504a which is disconnected from the boundary 514 of the environment),
wherein the machine is caused to perform the one or more operations based at least on the occlusion map(Haifeng, page 6 line 21, robotic device comprising the computing system according to the fourth aspect, one or more actuators to enable the robotic device to interact with the environment, and an interaction engine comprising at least one processor to control the one or more actuators, wherein the interaction engine is configured to use the map of the environment to interact with the environment).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Kuniaki with Haifeng to obtain a bounding box (one or more points associated with the area of the environment) that is navigable (non-occluded) area. By obtaining the navigable area, it is possible to accurately represents an underlying layout of an environment for an autonomous navigation of a vehicle (robot). (Haifeng, page 22 line 1, the navigation map 500 therefore more accurately represents an underlying layout of the environment. In other cases, though, a representation of such an object may be retained in the navigation map 500. Haifeng, page 9 line 30, this navigation data. may be used to update or generate a map of the environment, which may be used by the robotic device for subsequent navigation (which may be improved compared to its original navigation, e.g. by covering a greater proportion of the environment, or by more accurately avoiding collisions such as with internal walls within the environment)).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Fasola (US 20230410423 A1) in view of Burlina (US 20250076486 A1).
Regarding claim 19, Fasola teaches the one or more processors of claim 18(Fasola, paragraph 59, each voxel 614 of the 2D occupancy grid 600 corresponds to a location relative to the autonomous vehicle and has associated therewith an indication as to the occupancy status of the corresponding location),
wherein the determination of the one or more occluded area comprises:
determining, based at least on the ray tracing, points along rays that are associated with the one or more 2D areas within the environment that are occupied(Fasola, paragraph 59, It will be recognized that each frame of data comprising the LIDAR point cloud may be ray traced and processed as described in greater detail below to generate a 3D occupancy grid, as well as a corresponding 2D occupancy grid, for the autonomous vehicle for each frame and that, by comparing cells of two 3D occupancy grids created from successive frames of LIDAR point cloud data corresponding to the same location in space, a determination may be made as to whether motion has occurred (e.g., occupancy characterization of a voxel associated with a particular location in space transitions from occupied to freespace or from freespace to occupied). A speed with which the detected motion occurs may also be estimated based on the time that has elapsed between the two frames);
Fasola specifically fails to disclose generating a polygon by connecting the points; and
determining the one or more occluded areas based at least on the polygon.
However, Burlina, which is in the same analogous art and that teaches about objection detection, discloses generating a polygon by connecting the points(Burlina discloses generating a bounding box for a detected object (occupancy) in an environment based on overlapping radar points (connected points).Furthermore, Burlina discusses the bounding box can be polygonal data. Burlina, paragraph 19, the object detection output data may include bounding shapes (e.g., 2D or 3D bounding boxes) and/or any combination of associated object attribute data (e.g., object classification, yaw, velocity, intent, etc.). In one example, a transformer model may generate output including an objectness head, a bounding box regression head, and a bounding box classification head, in which the output can be associated with or rendered onto a top-down representation and/or image domain representation of the environment. Burlina, paragraph 24, an early synchronization of the infrared-radar object detections with the object detections from other sensor modalities may refer to a synchronization that is performed immediately after the sets of object detections (e.g., bounding boxes) are generated by their respective pipelines. Burlina, paragraph 35, ML models within the object detection component 118 may be trained based on combined views of infrared data images and sets of overlapping radar points, and may output data indicative of object(s) detected within the combined sensor data view. In some examples, ML models within the object detection component 118 may be configured to receive combined infrared-radar sensor views and to output a set of bounding shapes 122 (e.g., bounding boxes, bounding contours, or other polygonal data)); and determining the one or more occluded areas based at least on the polygon(Burlina discusses the bounding shapes(polygons) may include detected objects, which corresponds to the occluded area. Burlina, paragraph 46, the bounding shapes 122 output from the infrared-radar fusion system 106 may include an object detection, object location, object size, object shape, object classification, object yaw, etc).
Therefore, it would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Fasola with Burlina to generate a bounding box for detected objects in 2D environment based on overlapping radar points. By generating a bounding box (polygonal data) for the detected object, it is possible to output the shape and size representative of the detected object based on retrieved 2D data. (Burlina, paragraph 38, ML model generating an output indicative of presence of an object of the respective object class (and/or subclass), and a bounding shape 122 indicating the object size/extent and location within the environment, along with any number of object attributes and/or a probability or confidence score. In other examples, the ML model(s) used by the object detection component 118 may output a probability score at each pixel location of the input(s) representing a probability of the pixel location corresponding to each object class of the set of object classes (e.g., pedestrian, animal, etc.)).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BESUFEKAD LEMMA TESSEMA whose telephone number is (571)272-6850. The examiner can normally be reached Monday - Friday 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hunter Lonsberry can be reached at 5712727298. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BESUFEKAD LEMMA TESSEMA/Examiner, Art Unit 3665
/HUNTER B LONSBERRY/Supervisory Patent Examiner, Art Unit 3665
