DETAILED ACTION
This action is in response to the claims filed on Jan. 5th, 2023. A summary of this action:
Claims 1-20 have been presented for examination.
Claim(s) 1, 3-5, 7-8, 10-12, 14-15, 17-19 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Funke et al., US 12,415,510
Claim(s) 2, 9, and 16 is/are rejected under 35 U.S.C. 103 as being obvious over Funke et al., US 12,415,510 in view of Pedro, Ines. From Linear Regression to Neural Networks: Why and How Part 4 of the “Getting Started in Deep Learning” Series. Medium Article. Accessed via WayBack Machine with Archive date Jan 29th, 2022. URL: medium(dot)com/deep-learning-sessions-lisboa/neural-netwoks-419732d6afc0.
Claim(s) 6, 13, 20 is/are rejected under 35 U.S.C. 103 as being obvious over Funke et al., US 12,415,510 in view of Stocco, Andrea, Brian Pulfer, and Paolo Tonella. "Mind the gap! a study on the transferability of virtual versus physical-world testing of autonomous driving systems." IEEE Transactions on Software Engineering 49.4 (2022): 1928-1940.
This action is non-final
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
§ 101
No § 101 rejection as the claims are merely a collection of additional elements, but for claim 2 which adds an abstract idea of a math concept (MPEP § 2106.07), however the updating of the weights in view of ¶ 25 and Ex Parte Desjardins is a practical application.
The Examiner suggests however incorporating the subject matter of claim 2, and its parallels (claims 9 and 16), into the independent claims, as should an abstract idea be later recited in the independent claims by amendment, the subject matter of claim 2 would address any potential rejection. This is not a rejection or objection, but merely a suggestion for compact prosecution.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3-5, 7-8, 10-12, 14-15, 17-19 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Funke et al., US 12,415,510
Regarding Claim 1
Funke teaches:
An apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory, the at least one processor configured to: (Funke, abstract and cf. 3 and 6 and accompanying description)
receive road data comprising autonomous vehicle (AV) sensor data collected for a real-world environment navigated by an AV; measure one or more first performance metrics, wherein the one or more first performance metrics correspond with a performance of the AV in the real- world environment;
Funke, abstract: “Techniques are described herein for determining simulated vehicle positional errors and correlating the positional errors with vehicle features. Such techniques may include receiving log data comprising trajectories and position data for a real vehicle, and executing a log-based simulated vehicle based on the log data. A simulated vehicle may be controlled to follow a simulation trajectory in a simulated environment based on the trajectory of the real vehicle. A simulation system may determine a difference between the positions of the simulated vehicle and corresponding positions of the real vehicle in the real environment. The techniques may further include determining a vehicle state features correlated to the lateral and/or longitudinal position errors of the simulated vehicle, and determining, based on the vehicle state features, position error distributions and/or models that can be used to control subsequent driving simulations.”
To clarify on the real vehicle data including sensor data, see col. 8, description of # 104 incl: “…In some examples, the log data also may include 40 sensor data and/or data based on sensor data detected by the vehicle that captured the log data, such as data identifying characteristics of the environment in which a vehicle was operated, objects within the proximity of the vehicle, attributes or characteristics of the environment and objects ( e.g., 45 classifications, sizes, shapes, positions, trajectories, etc.).” – to clarify, the log data contains both “sensor data” and measured performance data of the AV (e.g. col. 8, description of # 104: “The log data may include, for any number of time points in the log data 25 spanning the period of time, various vehicle state data and/or data describing the environment at the particular time point. The vehicle state data stored in the log data at each time point may include the current position of the vehicle ( e.g., x-position and y-position), the current vehicle velocity, yaw, 30 steering angle, etc. The vehicle state data stored in the log data also may include the trajectory being followed by the vehicle at the particular time point (e.g., a trajectory generated by a planner component of the vehicle at each tick…”
generate simulation (SIM) data based on the road data, wherein the SIM data describes a simulated environment based on the real-world environment; measure one or more second performance metrics, wherein the one or more second performance metrics correspond with a performance of the AV in the simulated environment;
See Funke, abstract, as cited above, and step # 104, then see # 112 as discussed in col. 8: “At operation 112, the simulation system may execute a driving simulation, including controlling a simulated vehicle based at least in part on the log data received in operation 104. As shown in box 114, during the driving simulation, a simulated vehicle 116 representing the real vehicle 108 may be instantiated within a simulated environment based at least in part on the environment in which the real vehicle 108. captured the log data. During the simulation, the simulated vehicle 116 may be controlled to perform a trajectory 118 similar or identical to the trajectory 110 followed by the real vehicle 108 at the same corresponding time point… In some examples, each time step in the driving simulation may represent a point in the simulation at which the simulation system determines and records the current vehicle state (including position) of the simulated vehicle 116, determines the trajectory 118 for the simulated vehicle to follow from the current state, and initiates control commands for the simulated vehicle to follow (e.g., execute or track) the determined trajectory…” – i.e. its generating simulated data based on the log data (see fig. 1-2 to clarify this includes the road its driving on, as visually depicted and per accompanying descriptions) and measuring simulated performance characteristics
and train a machine-learning (ML) model to predict a correspondence between the performance of the AV in the real-world environment and the performance of the AV in the simulated environment.
Funke, abstract, as cited above, then see col. 3, ln. 20-50: “In various examples, a number of log-based driving simulations may be executed and analyzed to determine lateral and/or longitudinal positional errors between the simulated vehicle within the driving simulation and the corresponding vehicle from which the log data was captured. The aggregation of the positional differences may be used to model and/or determine probability distributions for positional error within log-based simulations. As described below in more detail, positional errors of simulated vehicles may be correlated with various vehicle feature states such as velocity, acceleration, steering angle, etc. For instance, ranges of low and high velocities may correlate with relatively large positional errors in log-based simulations, while middle-range velocity velocities may correlate with relatively small positional errors. Using the models and/or probability distributions for simulated vehicle positional errors, the simulation system may improve driving simulations using a number of techniques. For instance, the simulation system may adjust the impact regions ( e.g., safety buffers) of the simulated vehicles based on predicted positional errors and/or position confidence levels of the simulated vehicle during the simulation. Additionally or alternatively, the simulation system may revise one or more criteria for evaluating the success or failure of the simulation, and/or may determine a confidence value for the simulation result, based on the predicted positional errors and/or position confidence levels of the simulated vehicle.” - and col. 6, ln. 10-60 incl.: “…In some examples, the simulation system may determine position error distributions, models, and/or functions correlating the simulated vehicle positional errors and/or simulated vehicle position confidence values with one or more vehicle features…. Using the positional error distributions, models, and/or functions, the simulation system may determine, during the execution of a driving simulation, the predicted positional errors and/or confidence value(s) associated with the positions (e.g., lateral and longitudinal) of the simulated vehicle during the time period of the simulation. As described in more detail below, the predicted positional errors and/or position confidence values for a simulated vehicle may be different for the lateral and the longitudinal positional errors/confidences at the same time during a simulation Additionally, the predicted positional errors and/or position confidence values for the simulated vehicle may change throughout a single driving simulation, based on changes to the vehicle features/attributes ( e.g., speed, acceleration, steering angle, etc.) that may be correlated with the positional error” – see co. 9, starting at ln. 45, for description including # 124 and # 126, then # 130.
Then see in col. 12-13, the paragraph split between the columns: “…As noted above, after determining the determining positional error for a number of different 65 driving simulations, the simulation system 102 may aggregate and analyze the simulated vehicle positional error data, to determine correlations between one or more vehicle state features and data and the magnitude of the lateral and/or longitudinal positional errors. Once the correlations have been determined (e.g., using regression analysis), the correlations may be expressed as position error distributions, models, and/or functions determined by the simulation system 102, and may be used when performing subsequent driving simulations as discussed above.” – the using regression analysis on the differences/errors is an example of training an ML model, and in Funke it is later used to predict “errors”, when ML model is construed in view of instant disclosure ¶ 22: “ML model 210 may be a regression model”
To clarify, see description of # 408 in col. 21: “At operation 408, the simulation system 102 may determine differences (e.g., errors) between the positions of the simulated vehicle during the simulation, and the corresponding positions of the real vehicle from which the log data was captured…” and fig. 5A-5B
Regarding Claim 3
Funke teaches:
The apparatus of claim 1, wherein the at least one processor is further configured to:
provide new SIM data to the ML model; and receive, from the ML model, a prediction regarding how an AV would perform in a new simulated environment based on the new SIM data. (Funke, as was cited above, e.g. col. 6, ln. 45-60, see col. 22 description of # 410 to further clarify
also, see col. 10, description of # 103: “At operation 130, the simulation system 102 may execute one or more subsequent driving simulations based at least in 25 part on the position error distributions, functions, and/or models determined in operation 126. As noted above, the data analysis performed in operation 126 may correlate the simulated vehicle position errors with one or more vehicle state features, including but not limited to velocity, acceleration, and/or steering angle. After determining the correlations between vehicle state features and the positional error in the driving simulations, the simulation system 102 may use the resulting distributions, functions, and/or models representing the correlations to generate, execute, and/or 35 evaluate subsequent simulations. In this example, the subsequent driving simulations executed in operation 130 may include log-based simulations based on log data captured by real vehicles, or may include additional simulation types (e.g., non-log based), such as modified log-based simulations, synthetic simulations, log-based simulations using log data captured during simulations, etc.”
Regarding Claim 4
Funke teaches:
The apparatus of claim 3, wherein the new SIM data is not derived from AV sensor data. (Funke, col. 10, description of # 103: “At operation 130, the simulation system 102 may execute one or more subsequent driving simulations based at least in 25 part on the position error distributions, functions, and/or models determined in operation 126. As noted above, the data analysis performed in operation 126 may correlate the simulated vehicle position errors with one or more vehicle state features, including but not limited to velocity, acceleration, and/or steering angle. After determining the correlations between vehicle state features and the positional error in the driving simulations, the simulation system 102 may use the resulting distributions, functions, and/or models representing the correlations to generate, execute, and/or 35 evaluate subsequent simulations. In this example, the subsequent driving simulations executed in operation 130 may include log-based simulations based on log data captured by real vehicles, or may include additional simulation types (e.g., non-log based), such as modified log-based simulations, synthetic simulations, log-based simulations using log data captured during simulations, etc.”
Regarding Claim 5
Funke teaches:
The apparatus of claim 1, wherein the ML model is a regressor, a random forest, a convolutional neural network (CNN), or a combination thereof. (Funke, as cited above, teaches using a regressor/regression analysis)
Regarding Claim 7
Funke teaches:
The apparatus of claim 1, wherein the AV sensor data comprises Light Detection and Ranging (LiDAR) data, Radio Detection and Ranging (RADAR) data, camera data, or a combination thereof. (Funke, as cited above, teaches the sensor data is received – see Funke, description of # 606 in col. 23-24 to clarify this includes “lidar sensors, radar sensors”, “cameras”, etc.)
Regarding claims 8, 10-12, 14-15, 17-19
These claims are rejected under similar rationales as their parallel claims discussed above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 2, 9, and 16 is/are rejected under 35 U.S.C. 103 as being obvious over Funke et al., US 12,415,510 in view of Pedro, Ines. From Linear Regression to Neural Networks: Why and How Part 4 of the “Getting Started in Deep Learning” Series. Medium Article. Accessed via WayBack Machine with Archive date Jan 29th, 2022. URL: medium(dot)com/deep-learning-sessions-lisboa/neural-netwoks-419732d6afc0.
Regarding Claim 2
While Funke does not explicitly teach the following, Funke in view of Pedro teaches:
The apparatus of claim 1, wherein to train the ML model, the at least one processor is configured to:
calculate a loss function based on a difference between the one or more first performance metrics and the one or more second performance metrics; and update one or more weights of the ML model based on the loss function.
Funke, as discussed above, which used regression analysis to determine correlations based on the difference/error data (note, it was difference between the performance metrics, as discussed above) “using regression analysis” (Funke, col. 12-13, paragraph split between the columns)
As taken in view of Pedro, section Motivation, last paragraph, followed by section on “Why Feedforward Neural Networks” and “How Neural Networks Work”, then see section “Backward Pass: Learning”: “When we are training a machine learning model, the first step is to make a prediction, which translates to perform the forward pass in the case of a neural network. Then, the model adjusts its parameters in order to decrease its prediction error, with a gradient-based algorithm such as Gradient Descent. Let ℒ be loss function that computes the model’s error based on its prediction and the true output. The gradient descent update rule tells us how to adjust the weights of each layer [see equation which includes loss function]…As we can see, gradient computations made to update the parameters in deeper layers of the network are useful, using the chain rule, to compute other gradients of parameters of shallower layers. This efficient way of computing gradients is called backpropagation, and can be used in gradient descent.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings from Funke which used regression analysis for analyzing the resulting errors/differences with the teachings from Pedro on using a neural network with backpropagation instead of a regression analysis.
The motivation to combine would have been that “In the previous blog post, we implemented a particular machine learning model, called linear regression. This model assumes that… the output is linear on the inputs… the input variables are independent …which makes it inappropriate for a lot of real-world problems. Thus, we need a more powerful model that does not impose these kinds of assumptions and is able to solve more complex problems.” (Pedro, Motivation, last paragraph)
Regarding Claim 9 and 16
Claims 9 and 16 are rejected under similar rationales as claim 2 above.
Claim(s) 6, 13, 20 is/are rejected under 35 U.S.C. 103 as being obvious over Funke et al., US 12,415,510 in view of Stocco, Andrea, Brian Pulfer, and Paolo Tonella. "Mind the gap! a study on the transferability of virtual versus physical-world testing of autonomous driving systems." IEEE Transactions on Software Engineering 49.4 (2022): 1928-1940.
Regarding Claim 6
While Funke does not explicitly teach the following, Funke in view of Stocco teaches:
The apparatus of claim 1, wherein to generate the SIM data, the at least one processor is further configured to: generate one or more atmospheric effects for rendering in the simulated environment. (Funk, as cited above, for the AV simulation
As taken in view of Stocco, abstract, then see § 5.1, RQ2: “What is the relation between
system-level failures occurring in simulation and those observable in the real world? Can we expose the same real-world failures by running only the subset of tests that exhibit high telemetry (e.g., uncertainty) in the simulation? In this RQ we first compare the failures experienced during virtual vs physical system-level testing due to corrupted or adversarial settings. Ideally, good and faithful virtual environments should produce failures analogous to those experienced in the real world.” – then, cf. 1 for “Corruptions on real-world/virtual driving images” wherein this shows rendered atmospheric effects such as “fog” – see § 5.5.1 ¶ 2 to further clarify: “Concerning the former, we test our SDCs under the image corruptions by Hendrycks et al. [56], widely used to test DNNs that process imagery data. The paper proposes
19 corruptions belonging to five classes, namely noise (four types: gaussian, shot, impulse, speckle), blur (five types: gaussian, glass, defocus, motion, zoom), weather (four types: fog, frost, snow, rain), luminance (three types: contrast, brightness, saturate), and resolution reduction (three types: JPEG compression, pixelate, elastic transform). Each corruption has a severity level from 1 to 5, indicating intensifying perturbations. Figure 1 shows a few examples of simulated and real-world driving images….” – e.g. See table 2 and § 5.7.3
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings from Funke on a system using simulations to test AVs in a manner that that simulations “reflect real-world scenarios” (Funke, col. 1, ¶ 1) with the teachings from Stocco on AV simulations that include rendered weather atmospheric effects. The motivation to combine would have been that “Ideally, good and faithful virtual environments should produce failures analogous to those experienced in the real world” (Stocco, as cited above).
Regarding Claim 13 and 20
Claims 13 and 20 are rejected under a similar rationale as claim 6 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bewley, Alex, et al. "Learning to drive from simulation without real world labels." 2019 International conference on robotics and automation (ICRA). IEEE, 2019. Abstract and see page 5
Chebotar, Yevgen, et al. "Closing the sim-to-real loop: Adapting simulation randomization with real world experience." 2019 international conference on robotics and automation (ICRA). IEEE, 2019. Abstract and § 1 ¶ 2, then see § III.C
Donà, Riccardo, and Biagio Ciuffo. "Virtual testing of automated driving systems. A survey on validation methods." IEEE access 10 (2022): 24349-24367. Abstract and §§ I-II, and cf. 2. Pages 24352-24354
Ebert, Christof, et al. "Systematic testing for autonomous driving." ATZelectronics worldwide 16.3 (2021): 18-23. Pages 19-22
Fremont, Daniel J., et al. "Formal scenario-based testing of autonomous vehicles: From simulation to the real world." 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2020. Abstract and § I, last two paragraph.
Hanke, Timo, et al. "Generation and validation of virtual point cloud data for automated driving systems." 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2017. Abstract and § III.b
Haq, Fitash Ul, et al. "Comparing offline and online testing of deep neural networks: An autonomous car case study." 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST). IEEE, 2020. Abstract and § I last two paragraphs, cf. 4, and pages 6-8
Kadian, Abhishek, et al. "Sim2real predictivity: Does evaluation in simulation predict real-world performance?." IEEE Robotics and Automation Letters 5.4 (2020): 6670-6677. Pages 5-6
Wong, Kelvin, et al. "Testing the safety of self-driving vehicles by simulating perception and prediction." European Conference on Computer Vision. Cham: Springer International Publishing, 2020. Abstract and cf. 1. § 4.3
Xu, Chejian, et al. "Safebench: A benchmarking platform for safety evaluation of autonomous vehicles." Advances in Neural Information Processing Systems 35 (2022): 25667-25682. Abstract, page 2, cf. 1-2, §§ 3.3.1 and 3.4
Nygaard, US 10,713,148 Abstract and cf. 12
Taralova, US 10,832,093, Abstract, cf. 3 and col. 19-23.
Bai et al., US 11,213,946, abstract, cf. 1 and accompanying description, in particular # 134
Crego et al., US 11,648,962. Abstract and cf. 1 and accompanying description, in particular # 128, then cf. 2 and accompanying description, in particular # 218
Bagnell et al., US 12,444,247, abstract and cf. 6-7
Semple et al., US 12,552,396. Abstract and cf. 2 incl. # 230, then cf. 5 incl. # 510, see accompanying description
Lau et al., US 2023/0356733. Abstract and cf. 6 incl. 608 and accompanying description
Walther et al., US 10,599,546. Abstract and cf. 1, incl. accompanying description for # 118, cf. 4-7 and accompanying description to clarify.
Handa et al., US 2020/0306960. Abstract and cf. 7-8.
Nassar et al., US 2021/0294944. Abstract.
Whiteside et al., US 2023/0234613. Abstract and cf. 1B incl. # 252 and accompanying description.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID A. HOPKINS whose telephone number is (571)272-0537. The examiner can normally be reached Monday to Friday, 10AM to 7 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/David A Hopkins/Primary Examiner, Art Unit 2188
Prosecution Insights