DETAILED ACTION
This is the first office action regarding application number 18/987,228, filed December 19, 2024. This is a Non-Final Office Action on the merits, Claims 1-20 are currently pending and are addressed below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of applicants claim for domestic priority based on a provisional application filed on June 12, 2024.
Information Disclosure Statement
The information disclosure statement filed on 12/19/2024 is being considered by the examiner.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: Item 650, Item 850, Item 1030. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: Figure 2 item 260. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 4-6, 9-13, and 15 is/are rejected under 35 U.S.C. 102(a)(1) and (a)(2) as being anticipated by Vallespi (US-20170359561)
Regarding claim 1, Vallespi teaches one or more processors comprising processing circuitry to (Paragraph [0021], "Furthermore, one or more examples described herein may be implemented through the use of instructions that are executable by one or more processors.")
generate, based at least on a representation of stereo image data corresponding to an environment of an ego-machine (Paragraph [0029], "For example, camera interface 114 can connect to a video camera and/or stereoscopic camera 105 which continually generates image data of an environment of the vehicle 10. The stereo camera 105 can include a pair of imagers, each of which is mounted to a rigid housing structure that maintains the alignment of the imagers on a common plane when the vehicle is in motion.")
a surface disparity field representing estimated disparity values of a surface in the environment (Paragraph [0045], "Applying knowledge of the location and orientation of stereo camera 206 determined from sensor data 207, the disparity mapper 211 can use the 3D environment data from the current sub-map 238 to generate a baseline disparity image that represents distances from the stereo camera 206 to known features of the environment.")
and control one or more operations of the ego-machine based at least one the surface disparity field of the surface (Paragraph [0025], "The control system 100 can perform vehicle control actions (e.g., braking, steering, accelerating) and route planning using sensor information, as well as other inputs (e.g., transmissions from remote or local human operators, network communication from other vehicles, etc.).").
Regarding claim 4, Vallespi teaches the system as discussed above in claim 1, Vallespi further teaches wherein the one or more operations comprise controlling navigation of the ego-machine (Paragraph [0025], "The control system 100 can perform vehicle control actions (e.g., braking, steering, accelerating) and route planning using sensor information, as well as other inputs (e.g., transmissions from remote or local human operators, network communication from other vehicles, etc.).")
based on at least one of: a) determining that one or more obstacles are represented by one or more clusters of the surface disparity field that satisfy a designated threshold, or b) determining that one or more obstacles detected based at least on the surface disparity field appear in at least a threshold number of frames (Paragraph [0062], “In some implementations, the disparity mapper iterates through the pixels in the baseline disparity image and compares the pixel data in the baseline disparity image to its corresponding pixel in the same 2D location in one of the stereo camera images (typically the left image, but the right image can be used instead). For example, disparity data corresponding to a pixel in the upper left corner of the left stereo camera image is taken from the baseline disparity image. Assuming that no new feature or object is present in the scene that is not included in the 3D environment data, the disparity data should be roughly equal (within a reasonable margin of error to account for map inaccuracies) to the disparity between pixels in the left and right stereo images taken of the scene,” here the system is determining obstacles in the disparity field by determining if a difference between the disparity data and the 3D environment data is not within a margin of error).
Regarding claim 5, Vallespi teaches the system as discussed above in claim 1, Vallespi further teaches wherein the one or more operations comprise generating a representation of a navigable space (Paragraph [0045], “Applying knowledge of the location and orientation of stereo camera 206 determined from sensor data 207, the disparity mapper 211 can use the 3D environment data from the current sub-map 238 to generate a baseline disparity image that represents distances from the stereo camera 206 to known features of the environment.”)
based at least on radially casting two-dimensional (2D) rays in a representation of the surface disparity field from a reference point to one or more points corresponding to one or more disparity differences that are at least a designated threshold (Paragraph [0045], “These features generally include terrain features, buildings, and other static, non-moving objects such as signs and trees. In some implementations, the disparity mapper 211 can generate the baseline disparity image through a ray casting algorithm that renders the three-dimensional environment into a two-dimensional image.”) (Paragraph [0047], “The disparity mapper 211 can efficiently output a disparity map of the scene that represents the distances from the stereo camera 206 to features and objects in the scene, including both existing features from the current sub-map 238 and new features and objects present in the scene that are not part of the current sub-map 238 data.”).
Regarding claim 6, Vallespi teaches the system as discussed above in claim 1, Vallespi further teaches wherein the one or more operations comprise refining one or more estimated ego-motion transforms aligning LiDAR detections based at least on registering lifted representations of the surface disparity field from successive frames (Paragraph [0048], “Once corresponding pixels are found for each of the pixels in the left image, the disparity mapper 211 can output the generated disparity map (as image maps 218) for classifier 235 to use in classifying objects in the scene. In some aspects, an optical flow unit 212 can use the apparent motion of features in the field of view of the moving stereo camera 206 to supplement or replace the baseline disparity image generated from the 3D environment data. From either of the lenses of the stereo camera 206, a map of optical flow vectors can be calculated between a previous frame and a current frame. The optical flow unit 212 can use these vectors to improve the correspondence search algorithm. For example, given the motion vector of a pixel in the left image from the stereo camera 206, the motion vector of a corresponding pixel in the right image should be similar after accounting for the different perspective of the right lens of the stereo camera 206. Furthermore, image maps 218 can include images of optical flow vectors that classifier 235 can use to improve object classifications 213.,” here the system is refining the detected environment and motion of the vehicle by aligning the 3D environment data/LiDAR data and the surface disparity field from the stereo image cameras)
Regarding claim 9, Vallespi teaches the system as discussed above in claim 1, Vallespi further teaches wherein the one or more processors are comprised in at least one of:
a control system for an autonomous or semi-autonomous machine (Paragraph [0005], “FIG. 1 illustrates an example control system for operating an autonomous vehicle”);
a perception system for an autonomous or semi-autonomous machine (Paragraph [0027], “The AV 10 can be equipped with multiple types of sensors 101 and 103, which combine to provide a computerized perception of the space and environment surrounding the vehicle 10.”)
a system for performing simulation operations; a system for performing digital twin operations; a system for performing light transport simulation; a system for performing collaborative content creation for 3D assets; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using an edge device; a system implemented using a robot; a system for performing conversational Al operations; a system implementing one or more language models; a system implementing one or more large language models (LLMs); a system implementing one or more vision language models (VLMs); a system implementing one or more multi-modal language models; a system for generating synthetic data; a system for generating synthetic data using Al; a system for performing one or more generative Al operations; a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (EXAMINERS NOTE: Here the claim is reciting “at least one of” therefore the currently recited prior art only needs to recite one of the above limitations, the examiner has bolded the limitations that are currently being cited).
Regarding claim 10, Vallespi teaches a method comprising: controlling one or more operations of an ego-machine in an environment (Paragraph [0025], "The control system 100 can perform vehicle control actions (e.g., braking, steering, accelerating) and route planning using sensor information, as well as other inputs (e.g., transmissions from remote or local human operators, network communication from other vehicles, etc.).")
based at least on a ground disparity field representing estimated disparity values of a ground surface in the environment (Paragraph [0029], "For example, camera interface 114 can connect to a video camera and/or stereoscopic camera 105 which continually generates image data of an environment of the vehicle 10. The stereo camera 105 can include a pair of imagers, each of which is mounted to a rigid housing structure that maintains the alignment of the imagers on a common plane when the vehicle is in motion.") (Paragraph [0045], "Applying knowledge of the location and orientation of stereo camera 206 determined from sensor data 207, the disparity mapper 211 can use the 3D environment data from the current sub-map 238 to generate a baseline disparity image that represents distances from the stereo camera 206 to known features of the environment.").
Regarding claim 11, claim 11 is similar in scope to claim 4, and therefore is rejected under similar rationale.
Regarding claim 12, claim 12 is similar in scope to claim 5, and therefore is rejected under similar rationale.
Regarding claim 13, claim 13 is similar in scope to claim 6, and therefore is rejected under similar rationale.
Regarding claim 15, claim 15 is similar in scope to claim 9, and therefore is rejected under similar rationale.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 2-3, 7-8, 14, 16-17 and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Vallespi (US-20170359561) in view of Park (US 20220250624).
Regarding claim 2, Vallespi teaches the system as discussed above in claim 1, Vallespi further teaches wherein the one or more operations comprise detecting one or more obstacles (Paragraph [0038], “A detected event can correspond to a roadway condition or obstacle which, when detected, poses a potential hazard or threat of collision to the vehicle 10.”)
to a difference between lifted representations of the surface disparity field and a stereo disparity field corresponding to the representation of the stereo image data (Paragraph [0044], “The data processing system 210 can compare the sensor data 207 from the sensor array 205 with a current sub-map 238 from the sub-maps 231 to identify obstacles and potential road hazards in real time. In some aspects, a disparity mapper 211 and optical flow unit 212, which can be part of the data processing system 210, process the sensor data 207, images 208 from the stereo camera 206, and the current sub-map 238 to create image maps 218 (e.g., disparity maps and optical flow images). Classifier 235 can then provide object classifications 213—identifying obstacles and road hazards—to the AV control system 220”) (Paragraph [0047], “Assuming that no new feature or object is present in the scene that is not included in the current sub-map 238, the disparity data should be roughly equal (within a reasonable margin of error to account for map inaccuracies) to the disparity between pixels in the left and right stereo images taken of the scene.” here the system is determining a difference between the stereo image data disparity field in order to determine obstacles).
However Vallespi does not explicitly teach based at least on applying a range-dependent threshold height.
Park teaches systems and methods that detect hazards on a roadway by identifying discontinuities between pixels on a depth map including
detecting one or more obstacles based at least on applying a range-dependent threshold height (Paragraph [0024], “In general, a hazard's height may cause an occlusion of the roadway behind the hazard from a perspective of a camera. As such, when a hazard is present, there may be a discontinuity in disparity values indicative of a distance jump between a first pixel corresponding to a top of the hazard and a second pixel immediately above the first pixel,” here the system is using disparity values to determine the height of obstacles) (Paragraph [0025], “In determining the threshold disparity, a maximal distance to an object and/or a minimal pixel size may be considered. For the maximal distance, the threshold disparity must be small enough that a hazard of a predetermined height at a predetermined distance may be detected, which is to say that a disparity caused by the hazard occluding a portion of the roadway must be differentiable on the distance map. As such, a hazard may not be determined unless the disparity difference between neighboring pixels of a same column of pixels in the disparity and/or OF magnitude map are greater than the threshold disparity value,” here the system is using the disparity to determine the presence of an obstacle and the height of the object, the system is considering a maximal distance/range dependent and a predetermined/threshold height at a predetermined distance).
Vallespi and Park are analogous art as they are both generally related to systems and methods for controlling autonomous vehicles based on detected obstacles.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to include detecting one or more obstacles based at least on applying a range-dependent threshold height of Park in the system for detecting objects and controlling a vehicle of Vallespi with a reasonable expectation of success in order to adjust control strategies based on distance to improve the accuracy of the detection system in detecting small objects and allow the vehicle enough time to avoid the hazard (Paragraph [0002], “For example, an adequate hazard detection system must be robust to different types of hazards and include a high capacity to detect small hazards at a distance to allow an ego-vehicle enough time to avoid a hazard.”).
Regarding claim 3, Vallespi teaches the system as discussed above in claim 1, Vallespi further teaches wherein the one or more operations comprise detecting one or more obstacles (Paragraph [0038], “A detected event can correspond to a roadway condition or obstacle which, when detected, poses a potential hazard or threat of collision to the vehicle 10.”)
difference in disparity between the surface disparity field and a stereo disparity field corresponding to the representation of the stereo image data (Paragraph [0044], “The data processing system 210 can compare the sensor data 207 from the sensor array 205 with a current sub-map 238 from the sub-maps 231 to identify obstacles and potential road hazards in real time. In some aspects, a disparity mapper 211 and optical flow unit 212, which can be part of the data processing system 210, process the sensor data 207, images 208 from the stereo camera 206, and the current sub-map 238 to create image maps 218 (e.g., disparity maps and optical flow images). Classifier 235 can then provide object classifications 213—identifying obstacles and road hazards—to the AV control system 220”) (Paragraph [0047], “Assuming that no new feature or object is present in the scene that is not included in the current sub-map 238, the disparity data should be roughly equal (within a reasonable margin of error to account for map inaccuracies) to the disparity between pixels in the left and right stereo images taken of the scene.” here the system is determining a difference between the stereo image data disparity field in order to determine obstacles).
However Vallespi does not explicitly teach based at least on applying a range-dependent threshold height.
Park teaches systems and methods that detect hazards on a roadway by identifying discontinuities between pixels on a depth map including
detecting one or more obstacles based at least on applying a range-dependent threshold difference in disparity (Paragraph [0025], “In determining the threshold disparity, a maximal distance to an object and/or a minimal pixel size may be considered. For the maximal distance, the threshold disparity must be small enough that a hazard of a predetermined height at a predetermined distance may be detected, which is to say that a disparity caused by the hazard occluding a portion of the roadway must be differentiable on the distance map. As such, a hazard may not be determined unless the disparity difference between neighboring pixels of a same column of pixels in the disparity and/or OF magnitude map are greater than the threshold disparity value,” here in determining the threshold difference in disparity a maximum distance may be considered making the threshold difference range dependent).
Vallespi and Park are analogous art as they are both generally related to systems and methods for controlling autonomous vehicles based on detected obstacles.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to include detecting one or more obstacles based at least on applying a range-dependent threshold difference in disparity of Park in the system for detecting objects and controlling a vehicle of Vallespi with a reasonable expectation of success in order to adjust control strategies based on distance to improve the accuracy of the detection system in detecting small objects and allow the vehicle enough time to avoid the hazard (Paragraph [0002], “For example, an adequate hazard detection system must be robust to different types of hazards and include a high capacity to detect small hazards at a distance to allow an ego-vehicle enough time to avoid a hazard.”).
Regarding claim 7, Vallespi teaches the system as discussed above in claim 1, however Vallespi does not explicitly teach wherein the one or more operations comprise generating an estimated three-dimensional (3D) representation of a surface in the environment based at least on the surface disparity field.
Park teaches wherein the one or more operations comprise generating an estimated three-dimensional (3D) representation of a surface in the environment based at least on the surface disparity field (Paragraph [0024], “When the system identifies one or more hazard pixels, the system may determine that a hazard exists at the location (e.g., defined using 2D image space coordinates and/or 3D world space coordinates) of the one or more hazard pixels. The location of the hazard may then be mapped—e.g., using intrinsic and/or extrinsic parameters of the camera(s)—to a world space location and provided to one or more planning, control, or obstacle avoidance systems of the ego-machine,” here the system is generating a mapping/representation of a hazard to a 3D world space location and providing that 3D representation to the control systems of the vehicle).
Vallespi and Park are analogous art as they are both generally related to systems and methods for controlling autonomous vehicles based on detected obstacles.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to include wherein the one or more operations comprise generating an estimated three-dimensional (3D) representation of a surface in the environment based at least on the surface disparity field of Park in the system for detecting objects and controlling a vehicle of Vallespi with a reasonable expectation of success in order to adjust control strategies based on distance to improve the accuracy of the detection system in detecting small objects and allow the vehicle enough time to avoid the hazard (Paragraph [0002], “For example, an adequate hazard detection system must be robust to different types of hazards and include a high capacity to detect small hazards at a distance to allow an ego-vehicle enough time to avoid a hazard.”).
Regarding claim 8, Vallespi teaches the system as discussed above in claim 1, however Vallespi does not explicitly teach wherein the one or more operations comprise generating an estimated three-dimensional (3D) representation of a surface in the environment based at least on sampling, along one or more predicted trajectories of the ego-machine, one or more detections generated based on lifting the surface disparity field to 3D.
Park teaches wherein the one or more operations comprise generating an estimated three-dimensional (3D) representation of a surface in the environment based at least on sampling, along one or more predicted trajectories of the ego-machine, one or more detections generated based on lifting the surface disparity field to 3D (Paragraph [0024], “When the system identifies one or more hazard pixels, the system may determine that a hazard exists at the location (e.g., defined using 2D image space coordinates and/or 3D world space coordinates) of the one or more hazard pixels. The location of the hazard may then be mapped—e.g., using intrinsic and/or extrinsic parameters of the camera(s)—to a world space location and provided to one or more planning, control, or obstacle avoidance systems of the ego-machine,” here the system is generating a mapping/representation of a hazard to a 3D world space location and providing that 3D representation to the control systems of the vehicle, these hazards are detected based on sensor samples along a path ahead/predicted trajectory of the vehicle).
Vallespi and Park are analogous art as they are both generally related to systems and methods for controlling autonomous vehicles based on detected obstacles.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to include wherein the one or more operations comprise generating an estimated three-dimensional (3D) representation of a surface in the environment based at least on sampling, along one or more predicted trajectories of the ego-machine, one or more detections generated based on lifting the surface disparity field to 3D of Park in the system for detecting objects and controlling a vehicle of Vallespi with a reasonable expectation of success in order to adjust control strategies based on distance to improve the accuracy of the detection system in detecting small objects and allow the vehicle enough time to avoid the hazard (Paragraph [0002], “For example, an adequate hazard detection system must be robust to different types of hazards and include a high capacity to detect small hazards at a distance to allow an ego-vehicle enough time to avoid a hazard.”).
Regarding claim 14, claim 14 is similar in scope to claim 7, and therefore is rejected under similar rationale.
Regarding claim 16, Vallespi teaches a system comprising one or more processors to control one or more operations of an ego-machine (Paragraph [0025], "The control system 100 can perform vehicle control actions (e.g., braking, steering, accelerating) and route planning using sensor information, as well as other inputs (e.g., transmissions from remote or local human operators, network communication from other vehicles, etc.).")
based at least on a ground disparity field representing estimated disparity values of a ground surface (Paragraph [0029], "For example, camera interface 114 can connect to a video camera and/or stereoscopic camera 105 which continually generates image data of an environment of the vehicle 10. The stereo camera 105 can include a pair of imagers, each of which is mounted to a rigid housing structure that maintains the alignment of the imagers on a common plane when the vehicle is in motion.") (Paragraph [0045], "Applying knowledge of the location and orientation of stereo camera 206 determined from sensor data 207, the disparity mapper 211 can use the 3D environment data from the current sub-map 238 to generate a baseline disparity image that represents distances from the stereo camera 206 to known features of the environment.").
However Vallespi does not explicitly teach in a simulated environment within a simulation rendered using one or more light transport simulation algorithms.
Park teaches in a simulated environment within a simulation rendered using one or more light transport simulation algorithms (Paragraph [0026], “In the simulation, a virtual 3D environment may be generated that includes a roadway and one or more hazards. Some or all of the 3D environment may then be projected into a two dimensional (2D) space to generate a virtual 2D image. From the virtual 2D image, the system may compute OF for the 2D image and, based on the computed OF, the system may perform a simulation using a pair of simulated cameras—with given focal lengths and baseline—to identify a detectible distance for the one or more hazards. Accordingly, the system may determine an optimal disparity threshold based on the detectible distance for the one or more hazards. In some embodiments, random noise may be added to the simulation to simulate OF tracking errors, which may lead to a more accurate disparity threshold in the real-world”).
Vallespi and Park are analogous art as they are both generally related to systems and methods for controlling autonomous vehicles based on detected obstacles.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to include a simulated environment within a simulation rendered using one or more light transport simulation algorithms of Park in the system for detecting objects and controlling a vehicle of Vallespi with a reasonable expectation of success in order to adjust control strategies based on distance to improve the accuracy of the detection system in detecting small objects and allow the vehicle enough time to avoid the hazard (Paragraph [0002], “For example, an adequate hazard detection system must be robust to different types of hazards and include a high capacity to detect small hazards at a distance to allow an ego-vehicle enough time to avoid a hazard.”).
Regarding claim 17, the combination of Vallespi and Park teaches the system as described above in claim 16, Park further teaches wherein the simulation is generated, at least in part, using one or more content creation applications of a three-dimensional (3D) content collaboration platform for 3D assets (Paragraph [0091], “In some examples, the SoC(s) 604 may include a real-time ray-tracing hardware accelerator, such as described in U.S. patent application Ser. No. 16/101,232, filed on Aug. 10, 2018. The real-time ray-tracing hardware accelerator may be used to quickly and efficiently determine the positions and extents of objects (e.g., within a world model), to generate real-time visualization simulations, for RADAR signal interpretation, for sound propagation synthesis and/or analysis, for simulation of SONAR systems, for general wave propagation simulation, for comparison to LIDAR data for purposes of localization and/or other functions, and/or for other uses. In some embodiments, one or more tree traversal units (TTUs) may be used for executing one or more ray-tracing related operations.”).
Regarding claim 19, the combination of Vallespi and Park teaches the system as described above in claim 16, Vallespi further teaches wherein the one or more processors are further to generate the ground disparity field based at least on a representation of stereo image data representing the simulated environment (Paragraph [0029], "For example, camera interface 114 can connect to a video camera and/or stereoscopic camera 105 which continually generates image data of an environment of the vehicle 10. The stereo camera 105 can include a pair of imagers, each of which is mounted to a rigid housing structure that maintains the alignment of the imagers on a common plane when the vehicle is in motion.") (Paragraph [0045], "Applying knowledge of the location and orientation of stereo camera 206 determined from sensor data 207, the disparity mapper 211 can use the 3D environment data from the current sub-map 238 to generate a baseline disparity image that represents distances from the stereo camera 206 to known features of the environment.").
Regarding claim 20, the combination of Vallespi and Park teaches the system as described above in claim 16, Vallespi further teaches wherein at least one of the processors is implemented in at least one processing node of a plurality of processing nodes of a data center and accessible to one or more remote clients via at least one of an application programming interface (API), or an application plug-in (Paragraph [0026], “In some variations, the control system 100 can include other functionality, such as wireless communication capabilities, to send and/or receive wireless communications with one or more remote sources.”) (Paragraph [0020], “Some examples described herein can generally require the use of computing devices, including processing and memory resources. For example, one or more examples described herein may be implemented, in whole or in part, on computing devices such as servers, desktop computers, cellular or smartphones, personal digital assistants (e.g., PDAs), laptop computers, printers, digital picture frames, network equipment (e.g., routers) and tablet devices,” here the system can be implemented via a server remote from the vehicle).
Claim 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Vallespi (US-20170359561) in view of Park (US 20220250624) and further in view of Zhang (CN 118069277).
Regarding claim 18, the combination of Vallespi and Park teaches the system as described above in claim 16, however the combination does not explicitly teach wherein the simulated environment is represented in at least one content creation application of the one or more content creation applications using an OpenUSD format.
Zhang teaches image processing, providing a multi-mode virtual reality content control method, device, terminal device and medium including
wherein the simulated environment is represented in at least one content creation application of the one or more content creation applications using an OpenUSD format (Paragraph [0068], “For example, after obtaining the layer processing results, the use and disabling of the layer can be controlled through the OpenUSD Layer Disable/Enable attribute,” here the system is teaching rendering a simulated environment using content creation software including OpenUSD).
Vallespi, Park, and Zhang are analogous art as they are both generally related to systems and methods for using software to determine 3D environments.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the instant application to include wherein the simulated environment is represented in at least one content creation application of the one or more content creation applications using an OpenUSD format of Park in the system for detecting objects and controlling a vehicle of Vallespi with a reasonable expectation of success as this is a simple substitution of one known element for another to obtain predictable results, in this case the modeling software of Park is substituted for the OpenUSD software of Zhang to achieve the predictable result of modeling a simulated environment using a known method, as supported by KSR rationale (B). Additionally this combination provides the advantage of allowing the use of an open source software such as OpenUSD in order to reduce the cost of the system by avoiding costly licensing fees.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Jadhav (US-20240371035) teaches systems and methods using the combination of Lidar and camera image data to determine objects in the environment using methods such as a depth map indicating the estimated depth of each pixel directly, or a disparity map indicating the disparity between pixels. Wu (US-20230281843) teaches systems and methods for training a machine learning model configured to generate a predicted depth image, comprising receiving data representing training samples that include a plurality of image pairs, each image pair includes a target image and a reference image both capturing a particular scene from different orientations. Park (US 11657532) teaches surface profile estimation and bump detection may be performed based on a three-dimensional (3D) point cloud, the 3D point cloud may be filtered in view of a portion of an environment including drivable free-space, and within a threshold height to factor out other objects or obstacles other than a driving surface.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER FEES whose telephone number is (303)297-4343. The examiner can normally be reached Monday-Thursday 7:30 - 5:30 MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aniss Chad can be reached at (571) 270-3832. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER GEORGE FEES/ Primary Examiner, Art Unit 3662
Prosecution Insights