Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Note on non-compliant amendment of claim 19
The amendment to claim 19 filed 7/13/2026 was non-compliant for a lot of reasons, e.g. the original claim filed 1/23/2026 states “one or more processing unit to perform,” and the amended claim filed 7/13/2026 states “one or more processing units to:” – this is missing all of the usual formatting of a claim amendment. Because sending a non-compliant amendment notice doesn’t further prosecution, and because the amendment of claim 19 appears to copy the subject matter for amended claim 1, the non-compliant amendment of claim 19, filed 7/13/2026, will be examined in this office action.
Response to Arguments
For the 103 rejection, Applicant’s arguments with respect to claims 1-3, 6-10 and 19-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Election/Restrictions
Claims 4-5 and 11-18 are withdrawn from further consideration pursuant to 37 CFR 1.142(b) as being drawn to a nonelected invention, there being no allowable generic or linking claim. Election was made without traverse in the reply filed on 1/23/2026.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 6-10 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over US20220185267A1 to Beller et al and US20220261642A1 to Yoshida.
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over US20220185267A1 to Beller et al, US20220261642A1 to Yoshida, US20190238568A1 to Goswami et al and US20220405578A1 to Antonides et al.
Yoshida teaches claims 1 and 19. A method, comprising:
receiving, from a set of sensors deployed on a device having a visual driving system, a sensor data instance captured during operation of the device; (Beller fig. 2 step 202 receive sensor data, para 50, “the operation 202 can include the vehicle 102 receiving sensor data 204 representing the environment 100 which includes an occluded region 206.” Beller para 75 “In the illustrated example, the vehicle 502 is an autonomous vehicle; however, the vehicle 502 could be any other type of vehicle, such as a semi-autonomous vehicle, or any other system having at least an image capture device (e.g., a camera enabled smartphone).”)
generating, using a machine learning model of the visual driving system and based at least on the sensor data instance, one or more base outputs and one or more adversarial outputs that represent a likelihood that the sensor data instance is adversarial, wherein the one or more adversarial outputs indicate one or more regions in the sensor data instance that are adversarial, and wherein the machine learning model is trained to identify the one or more regions 208 can include determining, based at least in part on the sensor data, a probability of an object occupying the occlusion region. For example, the operation 208 can include the vehicle 102 implementing the model component 104 to determine a discretized representation 210 that comprises prediction probabilities indicating portions of the environment 100 likely to include an object. FIG. 2 shows the discretized representation 210 corresponding generally to the occluded region 206, though in other examples the discretized representation 210 may also represent regions that are not occluded.” Beller para 76 “the vehicle computing device 504 may store sensor data associated with actual location of an object at the end of the set of estimated states (e.g., end of the period of time) and may use this data as training data to train one or more models.” Beller para 141 “determining an adversarial behavior of the pedestrian predicted to occupy the cell…”)
determining that the sensor data instance is adversarial based on the one or more adversarial outputs; and (Beller para 51 “An operation 208 can include determining, based at least in part on the sensor data…” Beller para 141 “determining an adversarial behavior of the pedestrian predicted to occupy the cell…”)
sending, to one or more downstream components, an indication that the sensor data instance is adversarial to cause the one or more downstream components of the visual driving system to modify the operation of the device in response to the indication. (Beller fig. 2 control operation of the vehicle 216, para 56 “the operation 216 can include the vehicle computing device determining a level of caution (e.g., a caution flag and/or a sub-goal) along the candidate trajectory 124. In some examples, the vehicle computing device may control operation of the vehicle based at least in part on an output by the model to achieve different levels of acceleration, braking, and/or steering.”)
Beller doesn’t teach an adversarial object that is ground truth and perturbed.
However, Yoshida teaches identify the one or more regions based on perturbed ground truth sensor data generated by perturbing original ground truth sensor data; (Yoshida para 7 “However, the existence of adversarial examples is known. An adversarial example is data to which a small perturbation is added for the purpose of deriving a wrong determination result in a determination process using a deep learner.” Yoshida para 89 “the adversarial example detection unit 24 detects adversarial examples by determining for each input observation data whether the observation data is an adversarial example or not, based on the probabilistic margin calculated for each observation data (step S14).”)
Yoshida, the claims and Beller are systems that detect adversarial outputs. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to train on perturbed images because, “the existence of adversarial examples is known. An adversarial example is data to which a small perturbation is added for the purpose of deriving a wrong determination result in a determination process using a deep learner.” Yoshida para 7.
Yoshida teaches claim 2. The method of claim 1, further comprising:
generating, using the machine learning model and based at least on a (Yoshida fig. 3 input observation data s11. Yoshida para 87 “Next to step S11, the output distribution calculation unit 22 calculates the mean and variance of the output values by class…”)
determining, using the one or more (Yoshida para 89 “determining for each input observation data whether the observation data is an adversarial example or not, based on the probabilistic margin calculated for each observation data (step S14).”)
Yoshida doesn’t teach the downstream action
However, Goswami teaches sending a (Goswami fig. 10 does two different things in 1070 and 1080 based on an adversarial indication.)
Yoshida, Goswami and the claims all identify adversarial outputs. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to do something with the adversarial indicator because it’s wasteful to make an indication and do nothing with it. Also, because both references anticipate using these adversarial determinations in a car for safety and authentication. Yoshida para 5 and Goswami para 4.
Goswami and Yoshida don’t teach a second sensor and second determination.
However, Antonides teaches a second sensor, second base output and second indication. (Antonides para 46 “data captured within a first window in time by a first sensor may be combined with data captured by a second sensor within a second window in time to be included in the event driven fusion 110. The first window and the second window can be determined based on features of the first sensor and the second sensor as well as features of the surrounding environment. For example, an acoustic sensor of the sensor stack 105 may be able to detect the car 102 before a visual camera of the sensor stack 105 obtains visual data of the car 102.”)
Antonides, Yoshida and Goswami all detect adversarial data. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to use a second sensor because “[a]dvantageous implementations may further provide robustness against adversarial attacks.” Antonides para 35.
Yoshida teaches claim 3. The method of claim 2, wherein the determining the output type comprises matching a highest confidence included in the one or more (Yoshida para 67 “the mean of the output values calculated for the class with the highest likelihood to which the observation data corresponds is denoted as μa, and the variance of the output values calculated for that class is denoted as σa2.” The likelihood is the confidence.)
Yoshida doesn’t teach a second value.
However, Antonides teaches a second value. (Antonides para 46 “data captured within a first window in time by a first sensor may be combined with data captured by a second sensor within a second window in time to be included in the event driven fusion 110. The first window and the second window can be determined based on features of the first sensor and the second sensor as well as features of the surrounding environment. For example, an acoustic sensor of the sensor stack 105 may be able to detect the car 102 before a visual camera of the sensor stack 105 obtains visual data of the car 102.”)
Yoshida teaches claim 6. The method of claim 1, wherein the determining that the sensor data instance is adversarial comprises determining that at least one of the one or more adversarial outputs exceeds a threshold value. (Yoshida para 73 “whether the probabilistic margin M calculated by the adversarial example detection unit 24 is less than or equal to a predetermined threshold, and detect the observation data for which the probabilistic margin M is less than or equal to the threshold as an adversarial example. On the other hand, the adversarial example detection unit 24 may determine that the observation data for which the probabilistic margin M is greater than the threshold is normal observation data.”)
Yoshida teaches claim 7. The method of claim 1, wherein the determining that the sensor data instance is adversarial comprises determining that the one or more adversarial outputs include confidences that are higher than confidences associated with the one or more base outputs. (Yoshida para 73 “whether the probabilistic margin M calculated by the adversarial example detection unit 24 is less than or equal to a predetermined threshold…” The margin is a confidence.)
Yoshida teaches claim 8. The method of claim 1, wherein the sensor data instance comprises one or more images. (Yoshida para 78 “As described above, when the learning data is an image of a human face, an example of the preprocessing is to delete the background portion from the image stored as the learning data and to crop only the image of the portion corresponding to the face.”)
Yoshida teaches claim 9. The method of claim 1, wherein the one or more base outputs correspond to a set of classes associated with objects. (Yoshida para 87 “the output distribution calculation unit 22 calculates the mean and variance of the output values by class…”)
Yoshida teaches claims 10 and 20. The method of claim 1, wherein the one or more downstream components are included in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for 3D assets;
a system for performing deep learning operations;
a system implemented using an edge device;
a system for generating or presenting at least one of virtual reality content, augmented reality content, or mixed reality content;
a system implemented using a robot;
a system for performing conversational AI operations;
a system for generating synthetic data;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources. (Beller para 27 “The techniques discussed herein may improve a functioning of a vehicle computing device in a number of ways. Traditionally, in control planning for an autonomous vehicle…”)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Austin Hicks whose telephone number is (571)270-3377. The examiner can normally be reached Monday - Thursday 8-4 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AUSTIN HICKS/ Primary Examiner, Art Unit 2142
Prosecution Insights