DETAILED ACTION
This action is in response to the claims filed on Dec. 2nd, 2022. A summary of this action:
Claims 1-20 have been presented for examination.
Claims 4 and 13 are objected to because of the following informalities:
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of a mental process without significantly more.
Claim(s) 1, 6-10, 15-19 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Chan et al., US 12,013,693
Claim(s) 2, 5, 11, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chan et al., US 12,013,693 in view of Seiya, Shunya, et al. "Deepware: An open-source toolkit for developing and evaluating learning-based and model-based autonomous driving models." IEEE Access 10 (2022): 105734-105743 taken in view of Breitenhuber, Guido. "Testing Expected Behavior of Integrated ROS Applications." Master’s Thesis. Alpen-Adria-Universität Klagenfurt. July 2022.
Claims 3-4, 12-13, and 20 are not rejected under § 102/103. The closest art is the relied upon art, however this combination of art does not fairly teach, without the use of impermissible hindsight, what is recited in full particularity in these dependent claims (specifically, that the identifying of the one or more processes is part of the troubleshooting and that it is based on excluding the process(es) from the modified simulation) – in contrast, see the rejection below for dependent claim 2 and its parallels and other dependents.
This action is non-final
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 4 and 13 are objected to because of the following informalities:
Representative dependent claim 4 recites (claim 13 rejected under similar rationale):
The method of claim 2, further comprising identifying whether the one or more processes or non-determinism in running the baseline simulation are a cause of the divergence of the baseline simulation, as part of troubleshooting the baseline simulation,…
See ¶ 49: “Divergence can also be due to the non-deterministic nature of the software stack in both the simulation template and the processes used in generated the real-world data. Non-determinism is the concept that the same software stack can be provided the same input and generate different output as it is run different times.”
At issue with the claim is that is points to “non-determinisms” as a separate and distinct element from the “one or more processes” rather than a characteristic of the process(es) themselves, re
The Examiner suggests amending these dependents claims to more expressly reflect what is disclosed, i.e. “identifying whether a non-determinism of the one or more processes …are a cause of the divergence…” or the like.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea of a mental process without significantly more.
Step 1
Claim 1 is directed towards the statutory category of a process.
Claim 11 is directed towards the statutory category of an apparatus.
Claim 19 is directed towards the statutory category of an article of manufacture.
Claims 11 and 19, and the dependents thereof, are rejected under a similar rationale as representative claim 1, and the dependents thereof.
Step 2A – Prong 1
The claims recite an abstract idea of a mental process. See MPEP § 2106.04(a)(2).
As an initial matter, see ¶¶ 11-13 for the thrust of the claimed advance which is conveyed as merely automating a previously manual process: “Improving simulation results to more effectively represent real world results is a rigorous process from both a time perspective and a resource perspective. Specifically, often a simulation can be rerun from end to end to identify a divergence between the simulation results and the real world results. Further, one or more experts have to manually inspect the results of each process in the stack from end to end to diagnose the cause of the divergence…Specifically, in the autonomous vehicle space and with reference to testing, due to both the large amount of data included in running a software stack for a test, and the sheer volume of tests that are run, it is very difficult to efficiently identify divergences between simulation results and real world results.” – i.e. “experts” [people] identify and diagnose the divergence between simulation results and real-world results. Doing this faster by using a computer is merely the speed inherent from the use of a computer to perform an abstract idea (MPEP § 2106.05(f)).
See MPEP § 2106.05(a)(I), for mere automating of manual processes is not an improvement to technology.
The mental process recited in claim 1 is:
identifying a divergence of the baseline simulation between the performed behaviors of the AV and simulated behaviors in the baseline simulation of the AV;
identifying a divergence of the modified simulation between the performed behaviors of the AV and simulated behaviors in the modified simulation of the AV;
and troubleshooting the baseline simulation based on a comparison between the divergence of the modified simulation and the divergence of the baseline simulation.
The above is a mental process when taken in view of ¶¶ 11-13 as discussed above, i.e. an engineer/software developer reviews the simulation results, and mentally evaluates/judges what is causing the divergence, and then mentally judges how to troubleshoot it.
Under the broadest reasonable interpretation, these limitations are process steps that cover mental processes including an observation, evaluation, judgment or opinion that could be performed in the human mind or with the aid of pencil and paper but for the recitation of a generic computer component. If a claim, under its broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components, then it falls within the "Mental Process" grouping of abstract ideas. A person would readily be able to perform this process either mentally or with the assistance of pen and paper. See MPEP § 2106.04(a)(2).
To clarify, see the USPTO 101 training examples, available at https://www.uspto.gov/patents/laws/examination-policy/subject-matter-eligibility. In particular, with respect to the physical aids, see example # 45, analysis of claim 1 under step 2A prong 1, including: “Note that even if most humans would use a physical aid (e.g., pen and paper, a slide rule, or a calculator) to help them complete the recited calculation, the use of such physical aid does not negate the mental nature of this limitation.”; also see example # 49, analysis of claim 1, under step 2A prong 1: “Moreover, the recited mathematical calculation is simple enough that it can be practically performed in the human mind. Even if most humans would use a physical aid, like a pen and paper or a calculator, to make such calculations, the use of a physical aid would not negate the mental nature of this limitation.”.
As such, the claims recite a mental process.
Step 2A, prong 2
The claimed invention does not recite any additional elements that integrate the judicial exception into a practical application. Refer to MPEP §2106.04(d).
The following limitations are merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f), including the “Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more”:
Claim 1 does not have any express recitation of a computer/processor (e.g. it is not expressly a computer-implemented method) – see In re Prater in MPEP § 2111
Claims 10 and 19 recite a generic computer and generic computer components as tool to automate an abstract idea.
Should claim 1 be amended to expressly recite such generic computer components/a generic computer, these would be rejected under a similar rationale as claims 10 and 19.
See ¶¶ 62-72 to clarify on the generic nature of these recitations
The following limitations are adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g):
accessing captured real-world data of performed behaviors of an autonomous vehicle (AV) in operation; running input data that is generated based on the real-world data through a simulation template as part of a baseline simulation of the performed behaviors of the AV; running the input data through a modified simulation template as part of a modified simulation of the performed behaviors of the AV; - mere data gathering for use in the mental process, see ¶¶ 11-13 as discussed above.
A claim that integrates a judicial exception into a practical application will apply, rely on, or use the judicial exception in a manner that imposes a meaningful limit on the judicial exception, such that the claim is more than a drafting effort designed to monopolize the judicial exception. See MPEP § 2106.04(d).
MPEP 2106.04(II)(A)(2) “…Instead, under Prong Two, a claim that recites a judicial exception is not directed to that judicial exception, if the claim as a whole integrates the recited judicial exception into a practical application of that exception. Prong Two thus distinguishes claims that are "directed to" the recited judicial exception from claims that are not "directed to" the recited judicial exception…Because a judicial exception is not eligible subject matter, Bilski, 561 U.S. at 601, 95 USPQ2d at 1005-06 (quoting Chakrabarty, 447 U.S. at 309, 206 USPQ at 197 (1980)), if there are no additional claim elements besides the judicial exception, or if the additional claim elements merely recite another judicial exception, that is insufficient to integrate the judicial exception into a practical application. See, e.g., RecogniCorp, LLC v. Nintendo Co., 855 F.3d 1322, 1327, 122 USPQ2d 1377 (Fed. Cir. 2017) ("Adding one abstract idea (math) to another abstract idea (encoding and decoding) does not render the claim non-abstract"); Genetic Techs. Ltd. v. Merial LLC, 818 F.3d 1369, 1376, 118 USPQ2d 1541, 1546 (Fed. Cir. 2016) (eligibility "cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself."). For a claim reciting a judicial exception to be eligible, the additional elements (if any) in the claim must "transform the nature of the claim" into a patent-eligible application of the judicial exception, Alice Corp., 573 U.S. at 217, 110 USPQ2d at 1981, either at Prong Two or in Step 2B” and MPEP § 2106(I): “Mayo, 566 U.S. at 80, 84, 101 USPQ2dat 1969, 1971 (noting that the Court in Diamond v. Diehr found “the overall process patent eligible because of the way the additional steps of the process integrated the equation into the process as a whole,”” – and see MPEP § 2106.05(e).
To further clarify, MPEP § 2106.04(II)(A)(1): “Alice Corp., 573 U.S. at 216, 110 USPQ2d at 1980 (citing Mayo, 566 US at 71, 101 USPQ2d at 1965). Yet, the Court has explained that ‘‘[a]t some level, all inventions embody, use, reflect, rest upon, or apply laws of nature, natural phenomena, or abstract ideas,’’ and has cautioned ‘‘to tread carefully in construing this exclusionary principle lest it swallow all of patent law” See also Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335, 118 USPQ2d 1684, 1688 (Fed. Cir. 2016) ("The ‘directed to’ inquiry, therefore, cannot simply ask whether the claims involve a patent-ineligible concept, because essentially every routinely patent-eligible claim involving physical products and actions involves a law of nature and/or natural phenomenon").”
As a point of clarity, RecogniCorp, LLC v. Nintendo Co., 855 F.3d 1322, 1327, 122 USPQ2d 1377 (Fed. Cir. 2017) ("Adding one abstract idea (math) to another abstract idea (encoding and decoding) does not render the claim non-abstract"); Genetic Techs. Ltd. v. Merial LLC, 818 F.3d 1369, 1376, 118 USPQ2d 1541, 1546 (Fed. Cir. 2016) (eligibility "cannot be furnished by the unpatentable law of nature (or natural phenomenon or abstract idea) itself." discussed in MPEP § 2106.04(II)(A)(2) as well as MPEP § 2106.04(I): “Synopsys, Inc. v. Mentor Graphics Corp., 839 F.3d 1138, 1151, 120 USPQ2d 1473, 1483 (Fed. Cir. 2016) ("a new abstract idea is still an abstract idea") (emphasis in original).
The claimed invention does not recite any additional elements that integrate the judicial exception into a practical application. Refer to MPEP §2106.04(d).
Step 2B
The claimed invention does not recite any additional elements/limitations that amount to significantly more.
The following limitations are merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f), including the “Use of a computer or other machinery in its ordinary capacity for economic or other tasks (e.g., to receive, store, or transmit data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not integrate a judicial exception into a practical application or provide significantly more”:
Claim 1 does not have any express recitation of a computer/processor (e.g. it is not expressly a computer-implemented method) – see In re Prater in MPEP § 2111
Claims 10 and 19 recite a generic computer and generic computer components as tool to automate an abstract idea.
Should claim 1 be amended to expressly recite such generic computer components/a generic computer, these would be rejected under a similar rationale as claims 10 and 19.
See ¶¶ 62-72 to clarify on the generic nature of these recitations
The following limitations are adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g):
accessing captured real-world data of performed behaviors of an autonomous vehicle (AV) in operation; running input data that is generated based on the real-world data through a simulation template as part of a baseline simulation of the performed behaviors of the AV; running the input data through a modified simulation template as part of a modified simulation of the performed behaviors of the AV; - mere data gathering for use in the mental process, see ¶¶ 11-13 as discussed above.
In addition, the above insignificant extra-solution activities are also considered as well-understood, routine, and conventional activities, as discussed in MPEP § 2106.05(d):
accessing captured real-world data of performed behaviors of an autonomous vehicle (AV) in operation; - this is considered similar to the example WURC activity as discussed in MPEP § 2106.05(d)(II) of: “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network);”
running input data that is generated based on the real-world data through a simulation template as part of a baseline simulation of the performed behaviors of the AV; running the input data through a modified simulation template as part of a modified simulation of the performed behaviors of the AV – see ¶¶ 11-13 as discussed above.
Additional WURC evidence for clarity:
Bergamini, Luca, et al. "Simnet: Learning reactive self-driving simulations from real-world observations." 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021. § I ¶ 3: “A common approach to mitigate some of these issues is log replay, where the movement of other traffic participants is replayed around the SDV in simulation as it happened when the log was collected. However, if the SDV’s new actions differ from those when the log was collected, the traffic participants don’t react to it, and thus the simulation becomes unrealistic and ineffective for validation. For example, even a slight braking during the log replay can result in an unrealistic collision with the trailing car due to non-reactivity. These unrealistic outcomes are a result of what is called simulation drift” and see § II last paragraph: “Notable examples of such driving simulators are SUMO [35] and CARLA [36].”
Brogle, Craig, et al. "Hardware-in-the-loop autonomous driving simulation without real-time constraints." IEEE Transactions on Intelligent Vehicles 4.3 (2019): 375-384. § II, then see § III incl. ¶¶ 1-2 incl. ”Driving simulation systems have long been a cornerstone in efforts to lower development costs for advanced driver assistance systems [8]–[10] and are used extensively by major automotive manufacturers [11]–[13]….”
Chen, Dian, Vladlen Koltun, and Philipp Krähenbühl. "Learning to drive from a world on rails." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021. Abstract and § I, incl. last paragraph.
Deter, Dean, et al. "Simulating the autonomous future: A look at virtual vehicle environments and how to validate simulation using public data sets." IEEE Signal Processing Magazine 38.1 (2020): 111-121. § II
Osiński, Błażej, et al. "CARLA Real Traffic Scenarios--novel training ground and benchmark for autonomous driving." arXiv preprint arXiv:2012.11329 (2020). § II.C to D.
Pena, Javier, et al. "Ad perdevkit: An autonomous driving perception development kit using carla simulator and ros." 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2022. § IV and § IV.D
Stević, Stevan, et al. "Development of ADAS perception applications in ROS and “Software-In-the-Loop” validation with CARLA simulator." Telfor Journal 12.1 (2020): 40-45. §§ II-III
Won, Minseok, and Shiho Kim. "Verification and validation utilizing carla simulator for autonomous driving development." International Conference on Simulation and Modeling Methodologies, Technologies and Applications. Cham: Springer International Publishing, 2022. § 1.1, then see § 2.3.
As such, the claims are directed towards a mental process without significantly more.
Regarding the dependent claims
Claim 2 is adding a mental step of a mental judgement – to clarify, see ¶¶ 44-45, ¶ 52, and ¶¶ 57-59, and fig. 2 and 4, i.e. the simulation template is simply the collection of programs/processes to later be run in the simulation, and this step is merely removing one of these data items, readily performable in the mind of an engineer, such as with pen and paper by writing down a list of programs to later be simulated, and crossing an item in the list off. July 2024 Fed. Register Notice: “Claims to “the use of an algorithm-generated content-based identifier to perform the claimed data-management functions,” which include limitations to “controlling access to data items,” “retrieving and delivering copies of data items,” and “marking copies of data items for deletion,” where the claims cover “a medley of mental processes that, taken together, amount only to a multistep mental process,” such that the steps can be practically performed in the human mind, PersonalWeb Techs. LLC v. Google LLC, 8 F.4th 1310, 1316-18 (Fed. Cir. 2021).”
Claim 3 is adding another mental step of a mental evaluation/observation, i.e. the person observes the simulation results from the simulations, mentally evaluates/judges there was a divergence (e.g. “differences in a trajectory of the AV”, ¶ 47) in the simulation results e.g. by comparing two sets of numbers and doing a simple subtraction in their own head or with physical aids, or by mental observation of the trajectories such as people doing routinely in their own mind when observing traffic on real worlds when driving (e.g. visually observing and judging another driver is likely intoxicated because they keep swerving all over the road, or reckless because they are driving 30 mph over the flow of traffic and darting in and out of various lanes, etc.), wherein the “based on” an exclusion is merely specifying that a program was excluded in the prior mere data gathering (see rejection of claim 2 above), and the based on the comparison is merely being based on the prior mental judgement.
Claim 4 is rejected under a similar rationale as claim 3. See ¶ 49 to clarify on the “non-determinism”, i.e. “the concept that the same software stack can be provided the same input and generate different output as it is run different times” – and this is merely to identify it from mental observations/evaluations/judgements from the previously gathered data.
Claim 5 – merely further limiting the mental process. People are readily equipped to mentally organize a set of programs into a grouping/org structure, e.g. fig. 2 and 4, and this is a routine practice by developers looking to avoid spaghetti code as its known in the art. To clarify, pen and paper would be a useful aid, e.g. a lead developer/software architect, before the coding even starts, sits down and mentally comes up with a code to be later written, e.g. a module for processing LiDAR data, another for processing camera data, another for route planning, etc. The person then organizes these to where they fit into an organization structure, e.g. which ones are for perceiving the environment, which ones are for planning the next motion(s) to be done, etc. An org chart on pen and paper would be a useful mental aid to this process, wherein once this is complete the lead dev is readily able to now delegate out the various modules to junior software developers to work on. Followed by another mental step of the removing (see rejection of claim 2 above)
Claim 6, this is merely further limiting the mental process to what data is to be observed, e.g. the trajectories in the sim results (such as observing these on a print-out map from the computer with the paths plotted of the vehicles, or on a display). See ¶ 47 and see the discussion of trajectories above
Claim 7 is merely further limiting the mental process akin to claim 6, i.e. its merely specifying what data is to previously be gathered for the mental observation/evaluation/judge of the divergence
Claim 8 – see the rejection of claim 2 above, wherein this claim does not even recite how the template (i.e. the list of programs to be simulated) is modified, but only requires modifying the list with any modification.
Claim 9 – see rejections of claims 2 and 8 above, i.e. this is merely specifying starting with an old list of programs, and creating a new list based on the old list.
Remaining dependent claims rejected under similar rationales as their parallel representative claims discussed above.
As such, the claims are directed towards a mental process without significantly more.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 6-10, 15-19 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Chan et al., US 12,013,693
Regarding Claim 1
Chan teaches:
A method comprising: (Chan, abstract and cf. 1 along with accompanying description)
accessing captured real-world data of performed behaviors of an autonomous vehicle (AV) in operation; (Chan, abstract, cf. 3 and 4 $ 402 – to clarify, see col. 2, ¶ 2, and col. 5-6 the paragraph split between the columns, i.e. it obtains “log data” from an AV such as “sensor data” and other such data – see col. 6-8 for the description of # 102 and its associated structure of the AV to further clarify; also see col. 14 for description of # 150 for the ground truth data obtained based on the log data [example of a processed input data based on the accessed log data])
running input data that is generated based on the real-world data through a simulation template as part of a baseline simulation of the performed behaviors of the AV; (Chan, as cited above, in particular # 150 the ground truth data as an example of input data generated based on the real-world data, as discussed in col. 14 ln. 35-65, then see the paragraph after those two paragraphs (the one split between col. 15-16), in particular: “In addition or alternatively, the object determination component 152 may simulate the operations of the vehicle 102 planned before the disengagement and/or based on the perception data and determine negative outcomes for the simulated operations when the other vehicles or objects move based on the ground truth data. In such a way, the object determination component 152 may determine the object that caused of the disengagement in a scenario where, for example, the operator disengaged the autonomous mode in response to observing that another vehicle swerved toward vehicle 102 without the autonomous operation components of the vehicle 102 perceiving or predicting the change…” – e.g. fig. 2 # 200 as discussed in col. 15 – to clarify, the “scenario” used for the baseline simulation is an example of a simulation template, i.e. it’s a template of the event(s) and associated objects that is to be simulated, to clarify cf. 4 # 402-418
to clarify on the ground truth data, see col. 3, ln. 1-15: “In some examples, the ground
truth data may include information about objects and other vehicles in the environment ( e.g. velocity, yaw, distance to the autonomous vehicle, etc.) with less or no uncertainty in comparison to the log data. Further, the ground truth data may include information about the objects and other vehicles in the environment for a period of time before and after the event ( e.g. the observed path of vehicles after the disengagement). In at least some examples, such ground truth data may be hand labeled log data indicating positions, trajectories, etc. of objects in sensor data.”
identifying a divergence of the baseline simulation between the performed behaviors of the AV and simulated behaviors in the baseline simulation of the AV; (Chan, as cited above, then cf. 4 # 420-424 as discussed in col. 20, line 5-25: “The component verification system may then simulate operations of the automated operations system based on the ground truth-based prediction output data and determine whether the automated operation system would have handled the scenario associated with the disengagement event correctly when given the ground truth-based prediction output data at 420. For example, if the cause of the disengagement event determined at 406 was avoided or not replicated when the planning was performed using the ground truth-based prediction output data, the component verification system may determine the scenario was handled correctly. If the component verification system determines the scenario was handled correctly, the process may continue to 422, where the component verification system may flag the perception 20 component for analysis and improvement. Otherwise, the process may continue to 424, where the component verification system may flag the prediction component for improvement.” –
then, cf. 5-6 and accompanying description incl.: “FIGS. 5 and 6 illustrate example processes that may be used to analyze a disengagement event attributed to a perception component to determine which aspects of the perception component may be improved to reduce the occurrence of similar disengagement events in the future.” – in particular, col. 21, lines 15-30: “At 514, the analysis component may output the state of the parameters used in the correct prediction and simulated handling and discontinue the analysis of the current disengagement event. In addition, the analysis component may 20 also indicate the deviation [example of a divergence] of the output parameters for the correct prediction and simulated handling from those in the log data or from the ground truth data. This may represent the amount of improvement in the determination of the changed parameters by the perception component toward the ground truth value needed to avoid the current disengagement event and to reduce the occurrence of similar disengagement events in the future. The deviation from the ground truth data may also be indicative of the degree of uncertainty in the parameter output by the perception com- 30 ponent which does not disrupt the predication component and the autonomous system as a whole.” - see line. 30-45 as well, incl.: “In some cases, the analysis component may determine an L2 distance to measure the difference between predicted trajectory and a ground truth trajectory of the object (e.g. which may be obtained by 40 looking at the future ground truth or logged position of the object in the log). The analysis component may then utilize backpropagation (that, in some examples, comprises keeping the weights of the neural network fixed) with the difference as a loss function to determine which outputs to vary.” And note # 504: “FDR INDIVIDUAL PARAMETERS DEVIATING FROM GROUND TRUTH DATA:”, i.e. these are the individual parameters from the real data that deviate from the parameters of the ground truth data, in then steps through them # 506, and simulates them, and determines the deviation between the simulated and log/ground truth data (and note # 516-518, i.e. its an iterative process until it convergences at the “Yes” to # 514)
Of particular note for clarity is note # 502 is “Set parameters to log data values” as discussed in col. 20: “At operation 502, the analysis component may set the current values of parameters to be input to a logic of a prediction component (e.g. a machine learned model of the prediction component 134) to the parameters of the log data associated with a disengagement event in an autonomous. vehicle. Such parameters may comprise, but are not limited to, object sizes, locations, velocities, classifications, raw sensor data, environmental parameters (time of day, geolocation, lane position, map data, weather data, etc.), hyperparameters of a machine learned model, and the like…”
running the input data through a modified simulation template as part of a modified simulation of the performed behaviors of the AV; identifying a divergence of the modified simulation between the performed behaviors of the AV and simulated behaviors in the modified simulation of the AV; (Chan, as cited above, in particular cf. 5-6, incl. col. 21 ln. 45-50: “The analysis component may then set the value of the current parameters to those of the stepped parameter whose respective simulation resulted in the best result. The process may then return to 504 for additional simulation.” – and cf. 5 for its iterative loop as discussed above
and troubleshooting the baseline simulation based on a comparison between the divergence of the modified simulation and the divergence of the baseline simulation. (Chan, as cited above, in particular cf. 5 as discussed above, wherein the troubleshooting it’s the amount of improvement for #514, i.e. col. 21 as cited above, i.e. it adjusting the “parameters” as discussed (examples in col. 20 ln. 45-55 in describing # 502
to clarify, see col. 2 last paragraph: “This disclosure is directed to systems and techniques that may provide for component verification, for example, regarding a cause of a failure in a complex system, such as in autonomous operation system that provides for navigating an autonomous vehicle within an environment. Such complex systems may comprise, for example, multiple systems, subsystems, and components in which a failure may not be indicative of a fault of any such system, subsystem, or component. Systems and techniques according to this disclosure may be configured to determine which component or subsystem caused and/or contributed to the failure. In some examples, systems and techniques according to this disclosure may be configured to determine whether the root of a failure lies with a perception component generating perception data with a degree of uncertainty or a prediction component utilizing the perception data, though any other component, system, or subsystem is contemplated” and col. 2 ln. 34-48: “In some examples, the component 35 verification system may determine which component of the autonomous operation system may be improved and how the component should be improved to reduce future occurrence of trigger events.”
And col. 2-3, paragraph split between the col.: “As mentioned above, 65 information derived from sensor data by the perception component may have a degree of uncertainty which may be the source of the disengagement or the perception component may output incorrect data ( e.g. errors or data that is different than ground truth).” And col. 4, ln. 9-21: “Differences in the previously determined prediction data and the ground truth based prediction output data may be indicative of a difference with the perception system ( e.g., had the perception output more correct data, the prediction outputs should not diverge as much). If the autonomous operation system would have correctly handled the scenario associated with the event using the ground truth-based prediction output data or the ground truth-based prediction output data matches the corresponding ground truth values, the component verification system may determine that the perception component of 20 the autonomous operation system may be improved to reduce the occurrence of future events.” – see the paragraph after this (col. 4 ln. 22-46) to further clarify, and to clarify on the uncertainity in the log data but not the ground truth, note col 14 line. 50-65, in particular: “The ground truth data may be determined in various ways. In some examples, the ground truth data may be determined in a similar manner to the perception data determined by the perception component 130 but with higher accuracy. In such 55 cases, the ground truth component may rely on the potentially greater processing resources of the computing devices 108 and the relaxed time constraints of offline operations to utilize, for example, higher accuracy but more resource intensive machine learned models to generate the ground 60 truth data.” – note in col. 20 description of # 502: “hyperparameters of a machine learned model,”
And col. 20., ln. 10-25: “For example, if the cause of the disengagement event determined at 406 was avoided or not replicated when the planning was performed using the ground truth-based prediction output data, the component verification system may determine the scenario was handled correctly. If the component verification system determines the scenario was handled correctly, the process may continue to 422, where the component verification system may flag the perception 20 component for analysis and improvement. Otherwise, the process may continue to 424, where the component verification system may flag the prediction component for improvement.” – in other words, it flags what component in the stack, e.g. the prediction (col. 11, see description of # 134-136) or the perception component (see col. 9-10, description of # 130) to clarify on these caused the divergence, and the once the component was identified it determines how to improve the component to troubleshoot it
Regarding Claim 6
Chan teaches:
The method of claim 1, wherein either or both:
the divergence of the baseline simulation is based on a control output of the baseline simulation and a control output of the performed behaviors of the AV; and the divergence of the modified simulation is based on a control output of the modified simulation and the control output of the performed behaviors of the AV. (Chan, as cited above – to clarify, see instant disclosure, ¶ 47: “…For example [as an example of a control output], divergence can be identified based on differences in a trajectory of the AV that is generated by running the software stacks to completion through the control process” – see Chan, col 21, as cited above, description of # 514-516 incl.: “In some cases, the analysis component may determine an L2 distance to measure the difference between predicted trajectory and a ground truth trajectory of the object (e.g. which may be obtained by 40 looking at the future ground truth or logged position of the object in the log).”
Regarding Claim 7
Chan teaches:
The method of claim 1, wherein either or both:
the divergence of the baseline simulation is based on an output of a subprocess of the baseline simulation and an output of the subprocess corresponding to the performed behaviors of the AV; and the divergence of the modified simulation is based on an output of the subprocess of the modified simulation and the output of the subprocess corresponding to the performed behaviors of the AV. (Chan, as cited above and discussed above, i.e. its flagging whether it which subprocess caused the failure, and then improving upon that (e.g. the perception or the prediction subprocesses)
Regarding Claim 8
Chan teaches:
The method of claim 1, further comprising modifying the baseline simulation template based on the comparison between the divergence of the modified simulation and the divergence of the baseline simulation as part of troubleshooting the baseline simulation. (Chan, as cited above, i.e. the final improvement is the adjusted parameters at # 514 that are output [the parameters associated with the scenario] for the correct handling – to clarify, note col. 21, ln. 45-50: “The analysis component may then set the value of the current parameters to those of the stepped parameter whose respective simulation resulted in the best result. The process may then return to 504 for additional simulation.” – i.e. each loop its modifying the baseline parameters by these adjustments and re-running the scenario with the adjusted parameters for “additional simulation”)
Regarding Claim 9
Chan teaches:
The method of claim 1, wherein the simulation template is a new version of a previous simulation template. (Chan, as cited above, i.e. the final improvement is the adjusted parameters at # 514 that are output [the parameters associated with the scenario] for the correct handling – to clarify, note col. 21, ln. 45-50: “The analysis component may then set the value of the current parameters to those of the stepped parameter whose respective simulation resulted in the best result. The process may then return to 504 for additional simulation.” – i.e. each loop its modifying the baseline parameters by these adjustments and re-running the scenario with the adjusted parameters for “additional simulation”)
Regarding Claims 10, 15-19
These are rejected under similar rationales as representative claim 1, and the parallel dependents, as discussed above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 5, 11, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chan et al., US 12,013,693 in view of Seiya, Shunya, et al. "Deepware: An open-source toolkit for developing and evaluating learning-based and model-based autonomous driving models." IEEE Access 10 (2022): 105734-105743 taken in view of Breitenhuber, Guido. "Testing Expected Behavior of Integrated ROS Applications." Master’s Thesis. Alpen-Adria-Universität Klagenfurt. July 2022.
Regarding Claim 2
While Chan does not explicitly teach the following, Chan in view of Seiya and Breitenhuber teaches:
The method of claim 1, wherein the simulation template is modified to create the modified simulation template by removing one or more processes from the simulation template. (See Chan as discussed above for claim 1, in particular note fig. 4 for which component is flagged for improvement (i.e. either the prediction (col. 11, see description of # 134-136) or the perception component (see col. 9-10, description of # 130, note in particular it includes a “machine learned component # 132”)) is flagged for later improvement, and as noted above this includes adjusting the “hyperparameters of a machine learned model” – col. 20, description of # 502 for what the parameters include, and note col. 14, ln. 54-60: “In such
55 cases, the ground truth component may rely on the potentially greater processing resources of the computing devices 108 and the relaxed time constraints of offline operations to utilize, for example, higher accuracy but more resource intensive machine learned models to generate the ground 60 truth data.”; also note col. 18, ln. 10-15: “The prediction component 134 may receive, as input to a machine learned model, a number of features about the object.” To col. 18, ln. 25-45: “In the case of the disengagement being attributed to the prediction component 134, the analysis component 156 may be configured to cause additional training of a machine learned model of the prediction component to be performed using the current disengagement and, optionally, other similar disengagements Additionally or alternatively, the analysis component 156 may be configured to flag the prediction component 134 to an operator of the component verification system 148 for improvement, for example, based on additional training using the current disengagement and, optionally, other similar disengagements. Once the component to which the failure was attributed has been improved or further trained, the component analysis may be repeated and/or the improved or retrained component may be output to one or more autonomous operations systems for further operations” – i.e. it identifies/flags which ML component (e.g. either the perception or the prediction) needs to be improved, e.g. by adjusting the hyperparameters, e.g. by additional training/re-training; also as a point of clarity, see col. 1-2 paragraph split between the columns: “…In some examples, systems and techniques according to this disclosure may be configured to determine whether the root of a failure lies with a perception component generating perception data with a degree of uncertainty or a prediction component utilizing the perception data, though any other component, system, or subsystem is contemplated” – which at least suggests checking the other components as well for flagging and improvement, e.g. the “localization component # 128 as discussed in col. 9, or the “planning component 136” as discussed in col. 11
But Chan does not explicitly teach removing a process, however this would have been obvious in view of Seiya and Breitenhuber
First, see Seiya, abstract: “In recent decades, many learning-based autonomous driving systems have been proposed, and researchers have also created toolkits for developing these systems. These toolkits allow developers to train their models easily, and then test them using simulators… As a solution, in this paper we introduce Deepware, an end-to-end toolkit for developing and evaluating learning-based autonomous driving models. Deepware includes the tools needed for collecting and evaluating datasets, training models, and evaluating models on simulators or in real-world environments using actual vehicles. Unlike existing toolkits, we used ROS as our platform, which is a set of software frameworks for robot software development widely used in autonomous driving systems as middleware, which allows cooperation with model-based systems. This approach also allows system modules to be shared when building models. In addition, it allows the comparison of learning-based and model-based methods under the same conditions.” – cf. 1 to clarify, and page ‘736 col. 1, ¶ 2: “Developers evaluate their systems in simulators and in real vehicles to test their models on-line. But in order to reduce development cost, the system should be able to share the same operating code during these two on-line evaluation steps. Related studies have only focused on evaluation using simulators however, so their systems will need to be rebuilt in order to perform evaluations using real vehicles” – then, see fig. 3 and accompanying description, and see § III subsection E: “By using ROS in Deepware, the toolkit can operate independently of the simulator, and information obtained using the simulator can be transferred to a real environment simply by connecting them, thus the cost of migration is low. When migrating a model from a simulator to a real-world environment without using this functionality, it is necessary to remove the module that obtained information from the simulator directly and reconfigure the model to function in a real environment. But migration can be performed easily using when using Deepware, thanks to ROS.” – i.e. this provides an “eas[y]” manner to retrain the machine learning model in the simulator, and then migrate it to the actual vehicle, wherein this “thanks to ROS”
As taken in further view of Breitenhuber, abstract: “We set out to elaborate on the building blocks required to create a developer-friendly framework for specifying test cases that can automatically verify the desired behavior of ROS applications and simplify automated testing. We realized key ideas in a prototype testing framework called IntegROS.NET. Our framework enables developers to specify the expected communication behavior of ROS applications in the form of test cases and execute them against a set of scenarios to verify the behavior specification at runtime automatically.” – then, see page 24, last few paragraphs: “At this point, we want to bring the testing strategies suggested by Fowler in relation to the testing
approaches of the ROS community presented in Subsection 2.2.3 and the previously explained
testing taxonomy of Subsection 2.2.1. The one thing that is consistent across all three sections
is unit testing. This term everywhere means to verify small units of isolated code. What Fowler
calls integration tests could be done with node unit tests in the ROS world. Both strategies try to verify that the interface of a microservice or ROS node respectively can technically communicate with other entities in the system. Component tests (Fowler) are integration tests (as per 2.2.1) for interacting units within one microservice. They use internal test doubles or stubs for units that would otherwise cross the microservice boundary (e.g. database requests or calls to other services). In ROS, this can also be realized with ROS node unit tests.”
Then, see § 3.4: “A necessity in many test scenarios described in Section 3.2 is the isolation of parts of the system using test doubles. As described before, this can be realized by replacing entire nodes [i.e. remove and replace] with test doubles. But this isolation can not only isolate the SUT from the rest of the application. They can also act as observation points or create “test friendly” indirect input for the SUT…. When we talk about isolating parts of the system, we do not want to isolate nodes. We want to isolate functional blocks of the ROS application and replace them with test doubles. Namely topics, services, and actions. In the simplest case, this leads to replacing entire nodes with test double nodes. But it is also valid (and common practice) to replace multiple nodes with one test double. For example, all nodes directly interacting with the robot’s hardware can be replaced by one simulator as test double. In a complex case, it also allows us to isolate only parts of a node. Installing test doubles is supported by ROS through launch configurations if whole nodes are replaced by a double.” – e.g. see § 3.4.3: “Like test fakes, test stubs are doubles that represent a working alternative to the real component. However, a test stub is a fake implementation used by the test to control indirect input to the SUT. Usually, a stub has very limited functionality, covering only the parts required by the test using it. When you use rosbag4 to playback previously recorded topics during testing, you are using a test stub. Here rosbag replaces the original publisher as test double. The double has very limited functionality. It can only replay the recorded messages every time the test runs. Another example would be a stub action server for a gripper. You can control the action server and test how the SUT behaves in different scenarios (e.g. grip object, miss object, loosely gripped object etc.). A test stub might be a cheap alternative to a fully functional simulation of the gripper.”
To clarify, what is taught by this combination is first a manner to improve the re-training/additional training process of Chan for the flagged component by the Deepware toolkit of Seiya the enables an easy way of re-training with the simulator then migrating it back to the real-world AV (see citations above) “thanks to ROS”, wherein Breitenhuber is then relied upon for further modification for removing the non-flagged process/component of Chan, e.g. when the prediction component is flagged for improvement, it would have been obvious to have removed the perception component of Chan and replace it with a “test stub” because this is a “cheap alternative to a fully function simulation” of the perception component (and vice-versa when the other component is flagged), wherein Chan does this by using a “rosbag” in ROS
to further clarify on the pertinence of Seiya in this combination, note Seiya, § III subsection B, last paragraph: “The Deepware toolkit also supports users who want to collect a new set of driving data. The user can employ rosbag, a data collection function of ROS, to collect sensor and recognition data from various simulators that support ROS, or to collect driving data from real-world driving environments…” and § III.A ¶ 2 which discusses the use of the “rosbag” for “a training dataset” as it “is the data format used by the Robot Operating System (ROS) for driving data”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings from Chan on the re-training/additional training feature of Chan with the teachings from Seiya on Deepware as discussed above wherein this “includes the tools needed for collecting and evaluating datasets, training models, and evaluating models on simulators or in real-world environments using actual vehicles” (Seiya, abstract). The motivation to combine would have been that “The contributions of the Deepware toolkit introduced in this paper are as follows Increased reusability: By avoiding the tight coupling of autonomous driving systems with a simulator, our method allows the easy migration of automated driving models into real vehicles, by using the same implementation used during simulated driving. Comparison with model-based systems: The proposed toolkit makes it possible to compare the driving performance of model-based and learning-based methods within the same environment. Middle-to-middle model support: The proposed toolkit includes a method which allows the use of model-based autonomous driving resources in a learning-based autonomous driving toolkit, including both end-to-end and middle-to-middle agent methods” (Seiya, § I, last paragraph) – also, see § III.E as discussed above.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings from Chan, as was modified by Seiya on a system which would flag a machine learning component (e.g. the perception or prediction ones) for additional training/retraining for improvement as modified with the teachings from Seiya on technique which enables the re-training/additional training to be done with an AV simulator, and then migrated back to the real-world AV as further modified by Breitenhuber for a method to replace dependencies in ROS (i.e. the non-flagged component(s) of Chan which would not be the SUT/system under test) with a test stub. The motivation to combine would have been that this would have been a ““cheap alternative to a fully function simulation” (Breitenhuber, § 3.4.3).
Regarding Claim 5
Chan teaches:
The method of claim 2, further comprising:
grouping processes in the simulation template into an organizational structure of software stacks run in controlling the AV; (Chan, cf. 1 # 128-#140 and accompanying description for the grouped structure of the stack, see accompanying description of these parts to further clarify)
and identifying the one or more processes to remove from the simulation template based on the organizational structure of the software stacks. (Chan, as cited above, i.e. c.f. # 128-140, as taken in combination with Seiya and Breitenhuber as discussed above for claim 2.
Regarding Claim 11 and 14
Rejected under a similar rationale as the parallel claims discussed above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Christakis, Maria, et al. "A general framework for dynamic stub injection." 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE, 2017. Abstract and § I
Mussa, Mohamed, and Ferhat Khendek. "Identification and selection of interaction test scenarios for integration testing." International Workshop on System Analysis and Modeling. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. Abstract and § III.
Handa et al., US 2020/0306960. Abstract and cf. 8-9 and accompanying description.
Stefan et al., US 2019 / 0322286. Abstract and ¶¶ 37-40
Korn et al., US 2019/0079849. Abstract and cf. 3-5 and accompanying description.
Semple et al., US 12,552,396. Abstract and cf. 2 and accompanying description, in particular note # 248. Cf. 5-6 and note the lower portion, and see accompanying description.
Kabirzadeh, US 12,545,292, abstract and cf. 5 and accompanying description.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID A. HOPKINS whose telephone number is (571)272-0537. The examiner can normally be reached Monday to Friday, 10AM to 7 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/David A Hopkins/ Primary Examiner, Art Unit 2188
Prosecution Insights