elDETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 12, 28, and 41 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Regarding claims 12, 28, and 41, the broadest reasonable interpretation of the computer program product comprising computer readable executable code, when read in light of the specification and in view of one skilled in the art, includes software per se and signals per se. See MPEP §2106.03(I). Accordingly, claims 12, 28, and 41 are directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 5-7, 9, 10, 12-15, 21-24, 26, 28, and 29 is/are rejected under 35 U.S.C. 103 as being unpatentable over Meltz et al. (US 2021/0237772) (hereinafter Meltz) in view of Jing et al. (US 2021/0229668) (hereinafter Jing).
Regarding claims 1, 12, and 13, Meltz teaches a method, computer program product (ph. [0150], “non-transitory computer-readable storage medium”) and device comprising a memory and processor (ph. [0163], “Processing 172 may include processors, computers and/or memory”) to perform the method of calibrating a Digital Twin for an autonomous vehicle, comprising:
creating a model of the autonomous vehicle that defines one or more autonomous vehicle model parameters (ph. [0172], “A computerized simulation framework 270 of the simulated environment may include a computerized model, or models, of the simulated environment in which the vehicles move, of the simulated vehicles and/or of the simulated sensors. It may contain computer software simulation models of numerous fixed sensors numbered 1 to M, 255, 256… Note that simulated autonomous vehicles 257, 259 may include, in some cases, both simulation models of the vehicles, as well as simulation models of sensors of each vehicle”);
defining one or more simulation scenarios, wherein each simulation scenario defines at least one test maneuver of the autonomous vehicle (fig. 2, mission objectives 212, maneuver commands 228, scenario configuration 214; ph. [0177], “The AUTs may issue maneuver commands 228 to the modelled autonomous vehicles 257, 259 within the simulation framework 270”; ph. [0192], “Scenario generator 352 may send mission objectives 212 to AUT 220, and scenario configuration 214 to simulation framework 270.”);
executing the one or more simulation scenarios (fig. 4, step 440, generate and run sample verification test scenarios.; fig. 5, step 520, run sample scenarios; ph. [0176], “Once testing has begun, the simulation framework 270 and the AUTs may begin to interact, in a fashion similar to that in which the real world environment 100 interacts with the AUTs.”);
obtaining simulated data from one or more simulated sensors during the execution of the one or more simulation scenarios (fig. 2, 218 data from simulation (results); ph. [0275], “The set of results of each scenario run may in some cases be output 218 by simulation framework 270 to be stored, for example, in test results 374 in memory. These results may be also referred to herein as second results. In addition, in some cases the simulation framework 270 may also output 218, to memory 328, full log results 373 of the entire scenario.”);
comparing the data from one or more simulated sensors with a calibration data set, wherein the calibration data set includes data logged from one or more corresponding sensors on the autonomous vehicle during at least one real-world test maneuver corresponding to the one or more simulation scenarios (ph. [0299]-[0303], “ In some cases, the operational environment used for this generation may be somewhat modified, in order to meet the constraints of e.g. real-world testing performed on real roads in real cities and regions… This external-verification testing may use real-life equipment (autonomous vehicles configured with the relevant AUTs, sensors, management systems etc.), real travelling roads and terrain, possibly monitored by human test personnel. The external-verification test scenarios may thus in some cases be referred to herein also as real-world test verification scenarios for real-world verification testing… a statistical analysis may be performed, to compare the set of fourth results s to the set of second results, using for example, known statistical methods. The purpose of the comparison may be to determine whether the verification tests run on the simulated framework (run for example in step 440) and the external-verification tests (run for example in step 835), represent the same population with the required statistical significance… The external-verification test validity criteria may also be referred to herein as real-world test validity criteria.”).
Meltz does not explicitly teach applying a machine learning algorithm to the comparison to calibrate the one or more autonomous vehicle model parameters of the Digital Twin. However, Jing teaches applying a machine learning algorithm to the comparison to calibrate the one or more autonomous vehicle model parameters of the Digital Twin (fig. 4, model fit; ph. [0006], “an adaptive model of the autonomous vehicle that adaptively adjusts model parameters while the autonomous vehicle is in operation.”; ph. [0045], “the adaptive model uses model parameters that are modified during operation of the autonomous vehicle based on a Recursive Least Square method. In some embodiments, an Exponential Weighted Moving Average (EWMA) filter is applied to an error of the Recursive Least Square method for adaptation reset triggering.”). One of ordinary skill in the art before the effective filing date would have been motivated to modify Meltz in the manner taught by Jing in order to continuously improve the accuracy of vehicle model under changing conditions in order to increase the safety of operation of the autonomous vehicle (ph. [0015]).
Regarding claim 2, the Meltz/Jing combination teaches the method of claim 1. Jing further teaches the step of comparing further comprising: determining one or more error values based at least in part on the one or more simulated sensor data output and the data logged from the corresponding sensors on the autonomous vehicle (ph. [0034], “ek is the model fitting error at the kth step”).
Regarding claim 5, the Meltz/Jing combination teaches the method of claim 1. Meltz further teaches executing the one or more simulation scenarios comprises executing one simulation scenario, executing a batch of simulation scenarios sequentially, or executing a batch of simulation scenarios concurrently (fig. 4, step 440, generate and run sample verification test scenarios.; fig. 5, step 520, run sample scenarios; ph. [0176], “Once testing has begun, the simulation framework 270 and the AUTs may begin to interact, in a fashion similar to that in which the real world environment 100 interacts with the AUTs.”).
Regarding claim 6, the Meltz/Jing combination teaches the method of claim 1. Meltz further teaches executing the one or more simulation scenarios a plurality of times until a predetermined termination criteria is met (fig. 5, run sample scenarios 520 is executed repeatedly from the loop from 540 to 510 until it follows the No criteria at step 530).
Regarding claim 7, the Meltz/Jing combination teaches the method of claim 6. Jing further teaches identifying an optimal calibration of the Digital Twin as the autonomous vehicle model parameters relating to a minimised calibration score determined by the machine learning algorithm (ph. [0025], “In some embodiments, the least square fitting problem can be formulated into a constrained optimization problem to balance in between model response physical space rationality and fitting error minimization.”).
Regarding claim 9, the Meltz/Jing combination teaches the method of claim 1. Meltz further teaches the calibrated Digital Twin is used to tune one or more of at least one controller, navigation algorithms and/or guidance algorithms of the autonomous vehicle, and/or to optimise an autonomous vehicle shape (ph. [0140], “FIG. 2 illustrates a generalized example of components of an architecture for statistical testing of an autonomous vehicle navigation and control algorithm”; ph. [0274], “The verification test scenarios may be generated based on one or more sets of parameters of the computerized simulation framework 270 that are indicative of navigation scenarios. This may include, in some example cases, various parameters discussed with regard to FIG. 1a, e.g. number and types of autonomous vehicles, their mission objective, the road network 105, the number and types of sensors and their location, number and type of obstacles and their locations, traffic signals etc.”).
Regarding claim 10, the Meltz/Jing combination teaches the method of claim 1. Meltz further teaches the simulation scenarios are executed as event-based simulations (ph. [0222], “Similarly, based on time of day and day of week (e.g. rush hour), traffic patterns may change the typical speeds of vehicles. Similarly, in scenarios simulating bad weather and/or night hours, speeds will be distributed around a lower average speed than in daytime good-weather scenarios. As another example, in certain countries or regions, pedestrians tend to behave in a more or less unpredictable fashion, and thus the probability of a pedestrian bursting onto a street may be biased in a certain direction.”).
Regarding claim 14, 28, and 29, Meltz teaches a method of tuning one or more of at least one controller, a navigation algorithm and/or a guidance algorithm of an autonomous vehicle (ph. [0140], “FIG. 2 illustrates a generalized example of components of an architecture for statistical testing of an autonomous vehicle navigation and control algorithm”), computer program product (ph. [0150], “non-transitory computer-readable storage medium”) and device comprising a memory and processor (ph. [0163], “Processing 172 may include processors, computers and/or memory”), to perform the method comprising:
defining one or more simulation scenarios wherein each simulation scenario defines one or more components of a test manoeuver of the autonomous vehicle (ph. [0172], “Turning now to FIG. 2, it illustrates one generalized example of components of an architecture for statistical testing of an autonomous vehicle navigation and control algorithm, in accordance with certain embodiments of the presently disclosed subject matter.”; ph. [0220], “The output of such a step 420 may be one or more sets of parameters of the computerized simulation framework 270 that are indicative of navigation scenarios. This may include, in some example cases, various parameters discussed with regard to FIG. 1a, e.g. number and types of vehicles, number and type of obstacles and their locations, traffic signals etc.”);
defining one or more idealised control behaviours for the one or more simulation scenarios (fig. 2, maneuver 228);
executing the one or more simulation scenarios using a calibrated Digital Twin according to claim 1 (fig. 2, autonomous vehicle models 257, 259; fig. 4, step 440, generate and run sample verification test scenarios.; fig. 5, step 520, run sample scenarios; ph. [0176], “Once testing has begun, the simulation framework 270 and the AUTs may begin to interact, in a fashion similar to that in which the real world environment 100 interacts with the AUTs.”);
obtaining one or more simulated data from the Digital Twin, wherein the obtained data represents or corresponds to one or more simulated control behaviours of the calibrated Digital Twin during the execution of the one or more simulation scenarios (fig. 2, 218 data from simulation (results); ph. [0275], “The set of results of each scenario run may in some cases be output 218 by simulation framework 270 to be stored, for example, in test results 374 in memory. These results may be also referred to herein as second results. In addition, in some cases the simulation framework 270 may also output 218, to memory 328, full log results 373 of the entire scenario.”);
comparing the one or more simulated control behaviours with the defined one or more idealised control behaviours (ph. [0299]-[0303], “ In some cases, the operational environment used for this generation may be somewhat modified, in order to meet the constraints of e.g. real-world testing performed on real roads in real cities and regions… This external-verification testing may use real-life equipment (autonomous vehicles configured with the relevant AUTs, sensors, management systems etc.), real travelling roads and terrain, possibly monitored by human test personnel. The external-verification test scenarios may thus in some cases be referred to herein also as real-world test verification scenarios for real-world verification testing… a statistical analysis may be performed, to compare the set of fourth results s to the set of second results, using for example, known statistical methods. The purpose of the comparison may be to determine whether the verification tests run on the simulated framework (run for example in step 440) and the external-verification tests (run for example in step 835), represent the same population with the required statistical significance… The external-verification test validity criteria may also be referred to herein as real-world test validity criteria.”).
Meltz does not explicitly teach applying a machine learning algorithm to the comparison to tune one or more control parameters of the at least one controller, one or more parameters of the navigation algorithm and/or one or more parameters of the guidance algorithm of the autonomous vehicle. However, Jing teaches applying a machine learning algorithm to the comparison to tune one or more control parameters of the at least one controller, one or more parameters of the navigation algorithm and/or one or more parameters of the guidance algorithm of the autonomous vehicle (fig. 4, model fit; fig. 5, steps 501 and 503, determining using an adaptive mode of the autonomous vehicle an available engine torque for reducing a current speed of the autonomous vehicle to a lower speed and selecting a brake mode corresponding to the available engine brake torque; ph. [0006], “an adaptive model of the autonomous vehicle that adaptively adjusts model parameters while the autonomous vehicle is in operation.”; ph. [0045], “the adaptive model uses model parameters that are modified during operation of the autonomous vehicle based on a Recursive Least Square method. In some embodiments, an Exponential Weighted Moving Average (EWMA) filter is applied to an error of the Recursive Least Square method for adaptation reset triggering.”). One of ordinary skill in the art before the effective filing date would have been motivated to modify Meltz in the manner taught by Jing in order to continuously improve the accuracy of vehicle model under changing conditions in order to increase the safety of operation of the autonomous vehicle (ph. [0015]).
Regarding claim 15, the Meltz/Jing combination teaches the method of claim 14. Jing further teaches the step of comparing further comprising: determining one or more error values based at least in part on the simulated control behaviour and the defined idealised control behaviour. (ph. [0034], “ek is the model fitting error at the kth step”).
Regarding claim 21, the Meltz/Jing combination teaches the method of claim 14. Meltz further teaches executing the one or more simulation scenarios comprises executing one simulation scenario, executing a batch of simulation scenarios sequentially, or executing a batch of simulation scenarios concurrently (fig. 4, step 440, generate and run sample verification test scenarios.; fig. 5, step 520, run sample scenarios; ph. [0176], “Once testing has begun, the simulation framework 270 and the AUTs may begin to interact, in a fashion similar to that in which the real world environment 100 interacts with the AUTs.”).
Regarding claim 22, the Meltz/Jing combination teaches the method of claim 14. Meltz further teaches executing the one or more simulation scenarios a plurality of times until a predetermined termination criteria is met (fig. 5, run sample scenarios 520 is executed repeatedly from the loop from 540 to 510 until it follows the No criteria at step 530).
Regarding claim 23, the Meltz/Jing combination teaches the method of claim 22. Jing further teaches identifying an optimal tuning of the control parameters of the at least one controller, of the navigation parameters of the navigation algorithms, and/or of the guidance parameters of the guidance algorithms as the control parameters, the navigation parameters and/or the guidance parameters respectively relating to a minimised tuning score determined by the machine learning algorithm. (ph. [0025], “In some embodiments, the least square fitting problem can be formulated into a constrained optimization problem to balance in between model response physical space rationality and fitting error minimization.”).
Regarding claim 24, the Meltz/Jing combination teaches the method of claim 14. The combination further teaches the tuned at least one controller, the tuned navigation algorithm and/or the tuned guidance algorithm is implemented in an autonomous vehicle (Meltz, Abstract, “A computerized method of performing safety and functional verification of algorithms, for control of autonomous vehicles”; Jing, Abstract, “Methods, systems, and devices related to a method of controlling an autonomous vehicle, in particular, an autonomous diesel-engine truck are disclosed.”).
Regarding claim 26, the Meltz/Jing combination teaches the method of claim 14. Meltz further teaches the simulation scenarios are executed as event-based simulations (ph. [0222], “Similarly, based on time of day and day of week (e.g. rush hour), traffic patterns may change the typical speeds of vehicles. Similarly, in scenarios simulating bad weather and/or night hours, speeds will be distributed around a lower average speed than in daytime good-weather scenarios. As another example, in certain countries or regions, pedestrians tend to behave in a more or less unpredictable fashion, and thus the probability of a pedestrian bursting onto a street may be biased in a certain direction.”).
Claim(s) 11 and 27 is/are rejected under 35 U.S.C. 103 as being unpatentable over the Meltz/Jing combination as applied to claims 1 and 14 above, and further in view of Froehlich et al. (US 2022/0236698) (hereinafter Froehlich).
Regarding claims 11 and 27, the Meltz/Jing combination teaches the method of claim 1 and method of claim 14. The combination does not explicitly teach the machine learning algorithm is a Bayesian Optimisation. However, Froehlich teaches the machine learning algorithm is a Bayesian Optimisation (Abstract, “ascertaining a control strategy for a technical system using a Bayesian optimization method. The control strategy is created based on model parameters of a control model and is executable.”; ph. [0046], “In an exemplary embodiment, control unit 3 is used for the controlling of an at least partly autonomous robot, in particular an at least partly autonomous motor vehicle, as technical system 2.”). One of ordinary skill in the art before the effective filing date would have been motivated to modify the Meltz/Jing combination in the manner taught by Froehlich “in order to reduce the computational effort for the parameterization of control strategies.”(Froehlich, ph. [0001]).
Allowable Subject Matter
Claim 42 is allowed.
Claims 3, 4, 8, 16-20, 25, 30-40 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Russell M. Cummings, "Aerodynamics and Conceptual Design Studies on an Unmanned Combat Aerial Vehicle Configuration" teaches an integrated experimental and numerical approach to assess the stability and control prediction method capabilities, as well as the design and estimation of the control device effectiveness, for highly swept low observable unmanned combat aerial vehicle configurations.
Crego et al. (US Pat. 12,055,941) teaches a perception error model for fast simulation and estimation including “. In some examples, training the perception error model 242 may comprise receiving ground truth data associated with prediction data 240 extracted from log data (e.g., the log data may be generated from simulated or real-world operation of the autonomous vehicle) and determining a difference between the prediction data 240 and the ground truth data. For example, the prediction data 240 may be a prediction associated with time n+1 and the ground truth data may include the perception system output at time n+1 and/or label data generated by manual, semi-automatic, or automatic ground truth labelling. In some examples, the perception error model 242 may comprise an ML model… the perception error model may be a mixture density network (MDN) model trained via a feedforward neural network run on the input scenario-specific ground truth vector (e.g., determined by the k-means clustering) and the outputs of the neural network may be used as the parameters of the MDN that defines a probability density function of the error vector.”).
Pedersen (US 2022/0198107) teaches simulations for evaluating driving behaviors of autonomous vehicles.
Semple et la. (US Pat. 12,552,396) teaches object controller validation for Zoox autonomous vehicles.
Kavalar (US Pat. 12,530,508) teaches synthetic generation of simulation scenarios and probability-based simulation evaluation.
Wyrwas et al. (US 2021/0286924) teaches generating autonomous vehicle simulation data from logged data.
Puneeth (US 2024/0378926) teaches testing autonomous vehicles.
Redford et al. (US 2022/0289218) teaches performance testing for robotic systems.
Capell et al. (US Pat. 12,204,823) teaches generating perception scenarios for an autonomous vehicle from simulation data.
Kavalar (US Pat. 12,103,558) teaches probability calculation of encountering scenarios using emulator models including Bayesian optimization.
Murphy et al. (US 2022/0121786) teaches rapid aero modeling for computational experiments.
Kavalar (US Pat. 11,940,793) teaches vehicle component validation using adverse event simulation.
Ramanath et al. (US 2023/0315939) teaches validating a software-driven system based on real-world scenarios.
Van der Velden (US 2020/0401672) teaches fast method for computer-based simulation.
Froehlich et al. (US 2022/0236698) teaches determining model parameters for a control strategy for a technical system with the aid of a Bayesian optimization method.
Kaushik et al. (US 2020/0320176) teaches aerodynamically optimizing the geometry of vehicle bodies.
Khapane (US 2017/0212974) teaches simulating a vehicle driving through water.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN W WATHEN whose telephone number is (571)270-5570. The examiner can normally be reached M-F 9-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached at 571-272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
BRIAN W. WATHEN
Primary Examiner
Art Unit 2151
/BRIAN W WATHEN/ Primary Examiner, Art Unit 2151
Prosecution Insights