DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
In Claims 2-6, 9, 12, 15-18, “further to” should read “further configured to.”
In Claim 20, “plan a path” should read “planning a path.”
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Newcombe (U.S. Patent Publication 2012/0194516 A1) in view of Solenberg (U.S. Patent 11,233,937 B1).
In regard to Claim 1, Newcombe teaches one or more processors comprising processing circuitry to (see Paragraph 34 lines 3-5 teaching a 3D environment reconstruction system using one or more processors):
Obtain pose data representing a pose of an ego machine with respect to an environment (see Paragraph 7 lines 4-7, Paragraph 36 lines 8-12 teaching that the reconstruction model is built from pose data describing a camera location and orientation, and data indicating a distance from the camera to points within its environment);
Measure one or more distances between the ego machine and one or more surfaces of one or more objects in the environment (see Paragraph 7 lines 4-7, Paragraph 30 lines 1-10 teaching that the reconstruction model is built from data describing a camera location, and data indicating a distances from the camera to the surfaces of objects within its environment); and
Generate a three-dimensional (3D) data structure including a plurality of 3D elements, individual 3D elements of the 3D data structure including a value, computed using the pose data and the one or more distances, representing a distance of the ego machine to at least one surface of the one or more surfaces (see Abstract lines 1-7 teaching that the 3D environmental reconstruction is made of voxels and represents the camera’s location as well as distances to objects within its environment).
Newcombe fails to teach controlling movement of the ego machine in the environment based at least on the 3D data structure.
However, Solenberg teaches controlling movement of the ego machine in the environment based at least on the 3D data structure (see Column 2 lines 50-58, Column 3 lines 56-58, Column 17 lines 50-62 teaching an autonomous device that uses a processor to assess its environment as it traverses it, recreate the environment in a three-dimensional grid based on image data, detect obstacles in its path, and update its navigation plan accordingly).
Newcombe and Solenberg are both considered to be analogous to the claimed invention because they are in the same field of devices that capture map data of their environments as they traverse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a feature wherein the movement of the device is controlled based on a 3D representation of its environment as taught by Solenberg. Doing so could improve an autonomous vehicle by enabling it to continuously modify its trajectory based on its environment. This could increase the speed and reliability of the device.
In regard to Claim 2, Newcombe fails to teach wherein the processing circuitry is further configured to control the movement of the ego machine in the environment based at least on defining a path through the environment based at least on the 3D data structure.
However, Solenberg teaches wherein the processing circuitry is further configured to control the movement of the ego machine in the environment based at least on defining a path through the environment based at least on the 3D data structure (see Column 2 lines 50-58, Column 3 lines 56-58, Column 17 lines 50-62 teaching an autonomous device that uses a processor to assess its environment as it traverses it, recreate the environment in a three-dimensional grid based on image data, detect obstacles in its path, and update its navigation plan accordingly).
Newcombe and Solenberg are both considered to be analogous to the claimed invention because they are in the same field of devices that capture map data of their environments as they traverse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a feature wherein the movement of the device is controlled based on a 3D representation of its environment as taught by Solenberg. Doing so could improve an autonomous vehicle by enabling it to continuously modify its trajectory based on its environment. This could increase the speed and reliability of the device.
In regard to Claim 3, Newcombe further teaches wherein the processing circuitry is further to iteratively regenerate the 3D data structure responsive to the movement of the ego machine (see Paragraph 41 lines 4-11 teaching that the system uses an iterative process that detects frames that overlap when the capturing device has moved).
In regard to Claim 4, Newcombe further teaches wherein the processing circuitry is further configured to generate a volumetric mapping based at least on the one or more distances, wherein the 3D data structure comprises a 3D voxel grid generated based at least on the volumetric mapping (see Abstract lines 1-7 teaching that the 3D environmental reconstruction is made of voxels and represents the camera’s location as well as distances to objects within its environment).
In regard to Claim 5, Newcombe further teaches wherein the processing circuitry is further configured to determine free-space within the environment for operating the ego machine based at least on the 3D data structure (see Abstract lines 1-7, Figure 1, Paragraph 77 lines 1-6 teaching that the system reconstructs a 3D model of its environment, including objects within the environment and empty voxels that represent the space between them).
In regard to Claim 6, Newcombe further teaches wherein the processing circuitry is further configured to generate data for presenting a visual 3D reconstruction representing a view of the one or more surfaces relative to the ego machine (see Paragraph 69 teaching that to render a view of the model, a pose of a virtual camera defining the viewpoint for an image to be rendered is received).
In regard to Claim 7, Newcombe further teaches wherein the 3D data structure further includes one or more layers for generating the visual 3D reconstruction representing at least one of a surface structure, a texture, or a color associated with the one or more surfaces (see Paragraph 32 teaching that the capturing device contains a color camera that can capture images at visible light frequencies).
In regard to Claim 8, Newcombe further teaches wherein at least one of the surface structure, the texture, or the color associated with the one or more surfaces is based at least on raster-based images captured using at least one image sensor of the ego machine (see Paragraph 32, Paragraph 80 lines 19-22 teaching that the capturing device contains a color camera that can capture images at visible light frequencies, and that the model can be generated from a polygonal mesh using a rasterizing pipeline).
In regard to Claim 9, Newcombe further teaches wherein the processing circuitry is further configured to generate a polygonal mesh representation based at least on the 3D data structure, wherein the visual 3D reconstruction is based at least on the polygonal mesh representation (see Paragraph 80 teaching that the model can be represented via a polygon mesh, and a view of the model can be generated from the polygon mesh).
In regard to Claim 10, Newcombe further teaches wherein the one or more processors are comprised in at least one of:
A control system for an autonomous or semi-autonomous machine;
A perception system for an autonomous or semi-autonomous machine;
A system for performing simulation operations;
A system for generating synthetic data using machine learning;
A system for generating multi-dimensional assets using a collaborative content creation platform;
A system for performing light transport simulation;
A system for performing collaborative content creation for 3D assets;
A system for performing deep learning operations;
A system implemented using an edge device;
A system implemented using a robot;
A system for performing conversational AI operations;
A system for generating synthetic data (see Abstract lines 1-7 teaching that system generates a digital environment made of voxels and stores its data in a memory device);
A system incorporating one or more virtual machines (VMs);
A system implemented at least partially in a data center; or
A system implemented at least partially using cloud computing resources.
Claims 11, 13-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Newcombe (U.S. Patent Publication 2012/0194516 A1) in view of Solenberg (U.S. Patent 11,233,937 B1), in further view of Whittaker (U.S. Patent 7,069,124 B1).
In regard to Claim 11, Newcombe fails to teach planning a path through the environment based at least on a three-dimensional (3D) data structure comprising a plurality of 3D elements.
However, Solenberg teaches planning a path through the environment based at least on a three-dimensional (3D) data structure comprising a plurality of 3D elements (see Column 2 lines 50-58, Column 3 lines 56-58, Column 17 lines 50-62 teaching an autonomous device that uses a processor to assess its environment as it traverses it, recreate the environment in a three-dimensional grid based on image data, detect obstacles in its path, and update its navigation plan accordingly).
Newcombe and Solenberg are both considered to be analogous to the claimed invention because they are in the same field of devices that capture map data of their environments as they traverse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a feature wherein the movement of the device is controlled based on a 3D representation of its environment as taught by Solenberg. Doing so could improve an autonomous vehicle by enabling it to continuously modify its trajectory based on its environment. This could increase the speed and reliability of the device.
Newcombe further fails to teach wherein the 3D elements individually represent a cost to occupy a space within the environment, the plurality of 3D elements computed based at least on the pose data and the one or more distances.
However, Whittaker teaches wherein the 3D elements individually represent a cost to occupy a space within the environment, the plurality of 3D elements computed based at least on the pose data and the one or more distances (see Column 15 lines 50-56, Column 18 lines 28-33, Claim 15 teaching a robotic modeling system that uses 3D scans to measure distances from the robot to certain points within its environment, and determines pose data and cost values for individual locations).
Newcombe and Whittaker are both considered to be analogous to the claimed invention because they are in the same field of devices that capture map data of their environments as they traverse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a feature that uses distance data and pose data to determine cost values for locations around the device as taught by Whittaker. Doing so could improve an autonomous vehicle by enabling it to use sensor data to determine particular points that have a lower traversal cost than others. This could improve the device’s ability to determine optimal paths through complex terrain.
The rest of Claim 11 is substantially similar to Claim 1 (the bulk of both claims). Please see the rejection of Claim 1 above for analysis.
In regard to Claim 13, Newcombe further teaches wherein the one or more processors include one or more graphics processing units (GPUs) and a value of individual elements are at least in part computed in parallel using the one or more GPUs (see Paragraph 67 lines 3-8 teaching that the system can execute parallel program threads via GPU processors).
In regard to Claim 14, Newcombe fails to teach wherein the one or more processors execute a kernel comprising a pose estimator to compute the pose data based at least on image data.
However, Whittaker teaches wherein the one or more processors execute a kernel comprising a pose estimator to compute the pose data based at least on image data (see Column 10 lines 63-65, Column 15 lines 50-56, 63-67, Column 16 lines 1-7, Column 18 lines 28-33, Claim 15 teaching that the system uses 3D image scan data to compute the device’s footprints in different orientations with kernels, and pose the footprints in common, global coordinates).
Newcombe and Whittaker are both considered to be analogous to the claimed invention because they are in the same field of devices that capture map data of their environments as they traverse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a feature wherein kernels and image data are used to determine cost values for locations around the device as taught by Whittaker. Doing so could improve an autonomous vehicle by enabling it to use sensor data to determine particular points that have a lower traversal cost than others. This could improve the device’s ability to determine optimal paths through complex terrain.
In regard to Claim 15, Newcombe further teaches wherein the one or more processors are further configured to regenerate the 3D data structure based at least on the movement of the ego machine (see Paragraph 41 lines 4-11 teaching that the system uses an iterative process that detects frames that overlap when the capturing device has moved).
Claim 16 is substantially similar to Claim 5 (the bulk of both claims). Please see the rejection of Claim 5 above for analysis.
In regard to Claim 17, Newcombe further teaches wherein the one or more processors are further configured to generate data for presenting a visual 3D reconstruction representing a view of the one or more surfaces relative to the ego machine based at least on the 3D data structure (see Paragraph 7 lines 4-7, Paragraph 36 lines 8-12, Paragraph 69 teaching that to render a view of the model, a pose of a virtual camera defining the viewpoint for an image to be rendered is received, and that the reconstruction model is built from pose data describing a camera location and orientation, and data indicating a distance from the camera to points within its environment).
In regard to Claim 18, Newcombe fails to teach wherein the one or more processors are further to execute a path planning function that computes the path to avoids collisions with obstacles.
However, Solenberg teaches wherein the one or more processors are further to execute a path planning function that computes the path to avoids collisions with obstacles (see Column 2 lines 50-58, Column 3 lines 56-58, Column 17 lines 50-62 teaching an autonomous device that uses a processor to assess its environment as it traverses it, recreate the environment in a three-dimensional grid based on image data, detect obstacles in its path, and update its navigation plan accordingly).
Newcombe and Solenberg are both considered to be analogous to the claimed invention because they are in the same field of devices that capture map data of their environments as they traverse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a feature wherein the movement of the device is controlled based on a 3D representation of its environment as taught by Solenberg. Doing so could improve an autonomous vehicle by enabling it to continuously modify its trajectory based on its environment. This could increase the speed and reliability of the device.
Newcombe further fails to teach wherein the path is based at least on the cost to occupy the space within the environment associated with the individual elements of the 3D data structure.
However, Whittaker teaches wherein the path is based at least on the cost to occupy the space within the environment associated with the individual elements of the 3D data structure (see Column 15 lines 50-56, Column 18 lines 28-33, Claim 15 teaching a robotic modeling system that uses 3D scans to measure distances from the robot to certain points within its environment, and determines pose data and cost values for individual locations).
Newcombe and Whittaker are both considered to be analogous to the claimed invention because they are in the same field of devices that capture map data of their environments as they traverse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a feature that uses distance data and pose data to determine cost values for locations around the device as taught by Whittaker. Doing so could improve an autonomous vehicle by enabling it to use sensor data to determine particular points that have a lower traversal cost than others. This could improve the device’s ability to determine optimal paths through complex terrain.
Claim 19 is substantially similar to Claim 10 (the bulk of both claims). Please see the rejection of Claim 10 above for analysis.
In regard to Claim 20, Newcombe further teaches a method comprising analyzing an environment based at least on pose data of an ego machine and one or more distances between the ego machine and one or more surfaces of one or more objects in the environment (see Paragraph 7 lines 4-7, Paragraph 36 lines 8-12 teaching that the reconstruction model is built from pose data describing a camera location and orientation, and data indicating a distance from the camera to points within its environment).
Newcombe fails to teach planning a path through the environment based at least on a 3D data structure comprising a plurality of 3D elements; and
Controlling the movement of the ego machine in the environment based at least on the planned path.
However, Solenberg teaches planning a path through the environment based at least on a 3D data structure comprising a plurality of 3D elements; and
Controlling the movement of the ego machine in the environment based at least on the planned path (see Column 2 lines 50-58, Column 3 lines 56-58, Column 17 lines 50-62 teaching an autonomous device that uses a processor to assess its environment as it traverses it, recreate the environment in a three-dimensional grid based on image data, detect obstacles in its path, and update its navigation plan accordingly).
Newcombe and Solenberg are both considered to be analogous to the claimed invention because they are in the same field of devices that capture map data of their environments as they traverse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a feature wherein the movement of the device is controlled based on a 3D representation of its environment as taught by Solenberg. Doing so could improve an autonomous vehicle by enabling it to continuously modify its trajectory based on its environment. This could increase the speed and reliability of the device.
Newcombe further fails to teach a value of individual elements of the 3D data structure representing a cost to occupy a space.
However, Whittaker teaches a value of individual elements of the 3D data structure representing a cost to occupy a space (see Column 15 lines 50-56, Column 18 lines 28-33, Claim 15 teaching a robotic modeling system that uses 3D scans to measure distances from the robot to certain points within its environment, and determines pose data and cost values for individual locations).
Newcombe and Whittaker are both considered to be analogous to the claimed invention because they are in the same field of devices that capture map data of their environments as they traverse. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a feature that uses distance data and pose data to determine cost values for locations around the device as taught by Whittaker. Doing so could improve an autonomous vehicle by enabling it to use sensor data to determine particular points that have a lower traversal cost than others. This could improve the device’s ability to determine optimal paths through complex terrain.
Claim 12 is rejected under 35 U.S.C. 102(a)(1) as being anticipated by Newcombe (U.S. Patent Publication 2012/0194516 A1) in view of Solenberg (U.S. Patent 11,233,937 B1), in further view of Whittaker (U.S. Patent 7,069,124 B1), in further view of Lapin (U.S. Patent Publication 2021/0403034 A1).
In regard to Claim 12, Newcombe fails to teach wherein the one or more processors are further configured to compute the cost as representing a distance of the individual elements to a closest surface of the one or more surfaces.
However, Lapin teaches wherein the one or more processors are further configured to compute the cost as representing a distance of the individual elements to a closest surface of the one or more surfaces (see Paragraph 39 lines 1-8 teaching a vehicle trajectory planner that associates cost values with different parameters, such as the distance to a closest object).
Newcombe and Lapin are both considered to be analogous to the claimed invention because they are in the same field of systems that observe the environment around them as they move. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Newcombe’s invention to incorporate a cost value feature for distances to closest objects as taught by Lapin. Doing so could improve an autonomous vehicle system by comparing the cost value of moving towards the closest object in the environment with other factors that might be associated with moving in a different direction. This could increase the vehicle’s ability to traverse crowded environments.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAUL W ARELLANO whose telephone number is (571)270-0102. The examiner can normally be reached M-F 7:30-4:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado, can be reached on (571) 270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (in USA or Canada) or 571-272-1000.
/PAUL W ARELLANO/Examiner, Art Unit 3658
/Ramon A. Mercado/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights