DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending in Instant Application.
Priority
Examiner acknowledges Applicant’s claim to priority benefits of U.S. Application No. 18/397,921, filed on December 27, 2023, which is a continuation of U.S. Application No. 16/938,706, filed on July 24, 2020 and issued as U.S. Patent. No. 12,051,206, which claims the benefit of U.S. Provisional Application No. 62/878,659, filed on July 25, 2019.
Information Disclosure Statement
The information disclosure statement(s) (IDS) submitted on 09/08/2025, 10/08/2025, 12/04/2025, 04/03/2026, and 06/10/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement(s) is/are being considered if signed and initialed by the Examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 7-9, and 17-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Saha et al. (USPGPub 2025/0390821). As per claim 1, Saha discloses a machine (see at least Figure 27; item 2702) comprising: one or more systems-on-a-chip (SoCs) individually comprising one or more central processing units (CPUs), one or more graphics processing units (GPUs), and one or more hardware accelerators (see at least paragraph 0232); and one or more sensors having fields of view or sensory fields external to the machine (see at least paragraph 0189; wherein UAS 2702, including a sensor 2804 (e.g., a high-resolution camera, etc.)), wherein the one or more SoCs (see at least paragraph 0232) are to: generate, in a single pass of a neural network and based at least on applying a representation of sensor data generated using the one or more sensors to the neural network (see at least paragraph 0123; wherein wind effect detection module 1604 is configured to receive the captured data from one or more sensors 1400 (e.g., the video sensor 1404b, etc.) and detect motion, the amount of motion, and/or the pattern of the motion of the vegetation (broadly, wind effect characteristics) (e.g., due to wind effect, etc.) using the deep learning model 1600), a first output representing a semantic segmentation and a second output representing an instance segmentation (see at least paragraph 0124; wherein the wind effect detection module 1604 is configured to, in connection with the detecting the wind effect characteristics, use machine learning that include an instance segmentation task, a semantic segmentation task); and cause performance of one or more control operations associated with the machine based at least on the first output and the second output (see at least paragraph 0099; wherein the UAS control module 1506 is generally configured to, in response to the determined condition, dispatch one or more dispatch and control one or more UAS (e.g., drones, etc.) to the location of the condition to surveil and/or treat the condition). As per claims 2 and 18, Saha discloses wherein the single pass of the neural network generates the first output representing the semantic segmentation (see at least paragraph 0124; wherein the wind effect detection module 1604 is configured to, in connection with the detecting the wind effect characteristics, use machine learning that include a semantic segmentation task) and a third output representing a depth map (see at least paragraph 0163; wherein the conductor clearance detection module 1618 is configured to estimate depth for each pixel of the image). As per claims 3 and 19, Saha discloses wherein the single pass of the neural network generates the first output representing the semantic segmentation of a navigable space (see at least paragraph 0124; wherein the wind effect detection module 1604 is configured to, in connection with the detecting the wind effect characteristics, use machine learning that include a semantic segmentation task) and a third output representing one or more distances to one or more instances of animate objects (see at least paragraph 0164; wherein the conductor clearance detection module 1618 may be configured to interpolate depth from their neighboring pixels that have depth value and belong to the same object (semantic segmentation maps generated by deep learning models can be used here)). As per claim 7, Saha discloses wherein the one or more hardware accelerators include at least one of a vision accelerator, a ray-tracing accelerator, an optical flow accelerator, or a deep learning accelerator (see at least paragraph 0124; wherein the wind effect detection module 1604 is configured to, in connection with the detecting the wind effect characteristics, use machine learning and computer visions algorithms that include an instance segmentation task, a semantic segmentation task). As per claim 8, Saha discloses wherein the machine includes a vehicle, a car, a truck, a robot, a warehouse vehicle, a drone, a watercraft, or an aircraft (see at least Figure 27; item 2702 - UAS). As per claims 9 and 20, Saha discloses wherein the machine includes or uses at least one of: a control system or a perception system; a system for performing simulation operations; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using a robot; a system for generating synthetic data; a system for generating synthetic data using Al; or a system implemented at least partially using cloud computing resources (see at least paragraph 0195; wherein UAS control module 1506). As per claim 17, Saha discloses a method comprising: generating, using a single forward pass of a neural network and based at least on applying a representation of sensor data generated using one or more sensors of a machine to the neural network (see at least paragraph 0123; wherein wind effect detection module 1604 is configured to receive the captured data from one or more sensors 1400 (e.g., the video sensor 1404b, etc.) and detect motion, the amount of motion, and/or the pattern of the motion of the vegetation (broadly, wind effect characteristics) (e.g., due to wind effect, etc.) using the deep learning model 1600), a semantic segmentation and an instance segmentation (see at least paragraph 0124; wherein the wind effect detection module 1604 is configured to, in connection with the detecting the wind effect characteristics, use machine learning that include an instance segmentation task, a semantic segmentation task); and causing performance of one or more control operations associated with the machine based at least on the semantic segmentation and the instance segmentation (see at least paragraph 0099; wherein the UAS control module 1506 is generally configured to, in response to the determined condition, dispatch one or more dispatch and control one or more UAS (e.g., drones, etc.) to the location of the condition to surveil and/or treat the condition).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 4-6 are rejected under 35 U.S.C. 103 as being unpatentable over Saha et al. (USPGPub 2025/0390821) in view of Goel et al. (USPGPub 2021/0216793). As per claim 4, Saha does not explicitly mention wherein the single pass of the neural network generates the first output representing the semantic segmentation of one or more static elements and the second output representing the instance segmentation of one or more animate objects. However Goel does disclose: wherein the single pass of the neural network generates the first output representing the semantic segmentation of one or more static elements (see at least paragraph 0049; wherein a semantic label generated by a sub-class ML model two levels of ML models down the structure from the general classification determined by the first ML model 202 may be represented as “signage:traffic:stop sign”) and the second output representing the instance segmentation of one or more animate objects (see at least paragraph 0031; wherein an instance segmentation feature map may comprise a mask that identifies pixels (and/or other discrete portions of the sensor data) that are associated with an object and/or additional data associated therewith as determined by the model). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Goel with the teachings as in Saha. The motivation for doing so would have been to increasing the accuracy of object detection and/or increasing the trainability of a machine-learning (ML) model, see Goel paragraph 0009. As per claim 5, Saha does not explicitly mention wherein the single pass of the neural network generates the first output representing the semantic segmentation of one or more static elements and the second output representing an assignment of one or more pixels to one or more instances of animate objects. However Goel does disclose: wherein the single pass of the neural network generates the first output representing the semantic segmentation of one or more static elements (see at least paragraph 0049; wherein a semantic label generated by a sub-class ML model two levels of ML models down the structure from the general classification determined by the first ML model 202 may be represented as “signage:traffic:stop sign”) and the second output representing an assignment of one or more pixels to one or more instances of animate objects (see at least paragraph 0031; wherein an instance segmentation feature map may comprise a mask that identifies pixels (and/or other discrete portions of the sensor data) that are associated with an object and/or additional data associated therewith as determined by the model). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Goel with the teachings as in Saha. The motivation for doing so would have been to increasing the accuracy of object detection and/or increasing the trainability of a machine-learning (ML) model, see Goel paragraph 0009. As per claim 6, Saha does not explicitly mention wherein the second output regresses or classifies one or more instances of animate objects. However Goel does disclose: wherein the second output regresses or classifies one or more instances of animate objects (see at least paragraph 0033; wherein the first ML model 202 may generate a classification and/or probability based at least in part on a portion of the sensor data 204 associated with an ROI and/or instance segmentation). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Goel with the teachings as in Saha. The motivation for doing so would have been to increasing the accuracy of object detection and/or increasing the trainability of a machine-learning (ML) model, see Goel paragraph 0009.
Claims 10-12 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Saha et al. (USPGPub 2025/0390821) in view of Iandola et al. (USPGPub 2024/0346816). As per claim 10, Saha discloses a system comprising: one or more processors to control, one or more operations of an ego-machine in an environment (see at least paragraph 0099; wherein the UAS control module 1506 is generally configured to, in response to the determined condition, dispatch one or more dispatch and control one or more UAS (e.g., drones, etc.) to the location of the condition to surveil and/or treat the condition) based at least on a semantic segmentation and an instance segmentation of the environment (see at least paragraph 0124; wherein the wind effect detection module 1604 is configured to, in connection with the detecting the wind effect characteristics, use machine learning that include an instance segmentation task, a semantic segmentation task), the semantic segmentation and the instance segmentation generated based at least on processing a representation of the environment using a single pass of a neural network (see at least paragraph 0123; wherein wind effect detection module 1604 is configured to receive the captured data from one or more sensors 1400 (e.g., the video sensor 1404b, etc.) and detect motion, the amount of motion, and/or the pattern of the motion of the vegetation (broadly, wind effect characteristics) (e.g., due to wind effect, etc.) using the deep learning model 1600). Saha does not explicitly mention within a simulation that is rendered using ray-tracing; and a simulated environment. However Iandola does disclose: within a simulation that is rendered using ray-tracing (see at least paragraph 0076; wherein The simulation can be implemented with raytracing); and a simulated environment (see at least abstract; wherein simulated environments). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Iandola with the teachings as in Saha. The motivation for doing so would have been to improve performance of computer models, simulate scenarios that are not included in existing training data, and/or train computer models that remove unwanted effects or occlusions from sensor data of the environment, see Iandola paragraph 0009. As per claim 11, Saha discloses wherein the single pass of the neural network generates the semantic segmentation (see at least paragraph 0124; wherein the wind effect detection module 1604 is configured to, in connection with the detecting the wind effect characteristics, use machine learning that include a semantic segmentation task) and a depth map representing the simulated environment (see at least paragraph 0163; wherein the conductor clearance detection module 1618 is configured to estimate depth for each pixel of the image). As per claim 12, Saha discloses wherein the single pass of the neural network generates the semantic segmentation of a navigable space of the simulated environment (see at least paragraph 0124; wherein the wind effect detection module 1604 is configured to, in connection with the detecting the wind effect characteristics, use machine learning that include a semantic segmentation task) and one or more distances to one or more instances of animate objects in the simulated environment (see at least paragraph 0164; wherein the conductor clearance detection module 1618 may be configured to interpolate depth from their neighboring pixels that have depth value and belong to the same object (semantic segmentation maps generated by deep learning models can be used here)).
As per claim 16, Saha discloses wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing deep learning operations; a system for performing remote operations; a system for performing real-time streaming; a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content; a system implemented using a robot; a system for generating synthetic data; a system for generating synthetic data using Al; or a system implemented at least partially using cloud computing resources (see at least paragraph 0195; wherein UAS control module 1506).
Claims 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over Saha et al. (USPGPub 2025/0390821), in view of Iandola et al. (USPGPub 2024/0346816), and further in view of Goel et al. (USPGPub 2021/0216793). As per claim 13, Saha and Iandola do not explicitly mention wherein the single pass of the neural network generates the semantic segmentation of one or more static elements in the simulated environment and the instance segmentation of one or more animate objects in the simulated environment. However Goel does disclose: wherein the single pass of the neural network generates the semantic segmentation of one or more static elements in the simulated environment (see at least paragraph 0049; wherein a semantic label generated by a sub-class ML model two levels of ML models down the structure from the general classification determined by the first ML model 202 may be represented as “signage:traffic:stop sign”) and the instance segmentation of one or more animate objects in the simulated environment (see at least paragraph 0031; wherein an instance segmentation feature map may comprise a mask that identifies pixels (and/or other discrete portions of the sensor data) that are associated with an object and/or additional data associated therewith as determined by the model). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Goel with the teachings as in Saha and Iandola. The motivation for doing so would have been to increasing the accuracy of object detection and/or increasing the trainability of a machine-learning (ML) model, see Goel paragraph 0009. As per claim 14, Saha and Iandola do not explicitly mention wherein the single pass of the neural network generates the semantic segmentation of one or more static elements in the simulated environment and the instance segmentation representing an assignment of one or more pixels to one or more instances of animate objects in the simulated environment. However Goel does disclose: wherein the single pass of the neural network generates the semantic segmentation of one or more static elements in the simulated environment (see at least paragraph 0049; wherein a semantic label generated by a sub-class ML model two levels of ML models down the structure from the general classification determined by the first ML model 202 may be represented as “signage:traffic:stop sign”) and the instance segmentation representing an assignment of one or more pixels to one or more instances of animate objects in the simulated environment (see at least paragraph 0031; wherein an instance segmentation feature map may comprise a mask that identifies pixels (and/or other discrete portions of the sensor data) that are associated with an object and/or additional data associated therewith as determined by the model). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Goel with the teachings as in Saha and Iandola. The motivation for doing so would have been to increasing the accuracy of object detection and/or increasing the trainability of a machine-learning (ML) model, see Goel paragraph 0009. As per claim 15, Saha and Iandola do not explicitly mention wherein the instance segmentation regresses or classifies one or more instances of animate objects. However Goel does disclose: wherein the instance segmentation regresses or classifies one or more instances of animate objects (see at least paragraph 0033; wherein the first ML model 202 may generate a classification and/or probability based at least in part on a portion of the sensor data 204 associated with an ROI and/or instance segmentation). Therefore it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Goel with the teachings as in Saha and Iandola. The motivation for doing so would have been to increasing the accuracy of object detection and/or increasing the trainability of a machine-learning (ML) model, see Goel paragraph 0009.
Relevant Art
The prior art made of record and not relied upon are considered pertinent to applicant’s disclosure: USPGPub 2020/0175286 – Provide vehicle sensor information for object detection may include capturing generating a feature map based on captured sensor information, associating with each pixel of the feature map a prior box having a set of two or more width priors and a set of two or more height priors, determining a confidence value of each height prior and each width prior, outputting an indication of a detected object based on a highest confidence height prior and a highest confidence width prior, and performing a vehicle operation based on the output indication of a detected object. Embodiments may include determining for each pixel of the feature map one or more prior boxes having a center value, a size value, and a set of orientation priors, determining a confidence value for each orientation prior, and outputting an indication of the orientation of a detected object based on the highest confidence orientation. USPGPub 2020/0031338 – Provide the autonomous vehicle includes several sensor systems that asynchronously generate sensor system outputs. A batch generator system, executed by a processor, identifies batches of sensor system outputs to provide to an object classifier system, wherein the batches of sensor system outputs are identified based upon timing estimates, wherein the timing estimates include first timing estimates and second timing estimates. The first timing estimates include estimates of amounts of time needed by the object classifier system to complete processing of batches of different sizes. The second timing estimates includes estimates of when sensor system outputs are expected to be received from the several sensor systems.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAHMOUD S ISMAIL whose telephone number is (571)272-1326. The examiner can normally be reached M - F: 8:00AM- 4:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jelani Smith can be reached at 571-270-3969. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MAHMOUD S ISMAIL/Primary Examiner, Art Unit 3662
