CTNF 18/178,413 CTNF 93438 DETAILED ACTION Claims 8 – 14 have been presented for examination. Claims 1 – 7 are cancelled. This office action is in response to submission of the application on 03/03/2023. Claim Rejections - 35 USC § 112 The following is a quotation of 35 U.S.C. 112(b): (b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention. 07-34-01 Claim 13 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. With regard to claim 13 , it is to an apparatus comprising various functions, and without any recited structure for performing said functions. Therefore, the BRI of this claim elements extends to all structural manners of performing the “receive” and “determine” and “carry out” and “combine” and “validate”, those known and unknown to the inventor, and amounts to pure functional claiming without boundaries. The “determine” and “carry out” and “combine” and “validate” are interpreted for examination purposes as the general purpose computer structure indicated in the specification for the performing the steps of claim 8 (see the instant application Page 26, Line 12 to Line 16 “One or more of the method steps described here in detail may be carried out (e.g. implemented) by a data processing unit by one or more specific functions which are performed by the data processing unit.”) . The “receive” is interpreted for examination purposes as data from generic vehicle sensors and interfaces (see the instant application Page 11, Line 24 – 28 “The sensor data or internal system states of the HAV at a suitable interface are typically recorded and criticality metrics may for example be subsequently calculated or simulation-based tests carried out.”) 07-30-01 The following is a quotation of the first paragraph of 35 U.S.C. 112(a): (a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention. 07-31-02 AIA Claim 13 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA), first paragraph, as failing to comply with the enablement requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to enable one skilled in the art to which it pertains, or with which it is most nearly connected, to make and/or use the invention. With regard to claim 13 , it is to an apparatus comprising various functions, and without any recited structure for performing said functions. Therefore, the BRI of this claim elements extends to all structural manners of performing the “receive” and “determine” and “carry out” and “combine” and “validate”, those known and unknown to the inventor, and amounts to pure functional claiming without boundaries. Therefore, the disclosure is only enabling over the structure manners actually disclosed, and is not enabling over the entire BRI of the claims . Claim Rejections - 35 USC § 101 07-04-01 AIA 07-04 35 U.S.C. 101 reads as follows: Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title. Claims 8 - 14 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without significantly more. Independent claim 8 recites at Step 1 a statutory category (i.e. a process) method for validating control software for a robotic device, comprising the following steps: determining scenarios in which events with at least one specified criticality have arisen in the field tests, and determining , for each determined scenario of the determined scenarios, a frequency with which the determined scenario including an event with at least the specified criticality occurs; determining , from the simulations, a collision rate for each of the determined scenarios; combining the determined collision rates into an average collision risk over all the determined scenarios, taking into account the determined frequencies; and validating the control software based on a comparison of the average collision risk with a safety criterion. At Step 2A, Prong I the recited limitations, alone or in combination, amount to steps that, under its broadest reasonable interpretation, cover performance of the limitations in the mind in combination with using a pen and paper (see MPEP 2106.04(a)(2)(III)) . For example, the “determining” covers analytical processes requiring no more than judgements and evaluations based on observations of field tests and/or simulations carried out. The ”combining” covers analytical processes requiring no more than judgements and evaluations based on previously determined quantities. The “validating” covers analytical processes requiring no more than “comparison” between known quantities, which itself requires no more than judgements and evaluations. Accordingly, the claim recites an abstract idea. At Step 2A, Prong II this judicial exception is not integrated into a practical application since the claimed invention further claims: carrying out field tests using the control software; carrying out simulations for each of the determined scenarios. The “carrying out field tests” amounts to insignificant data gathering since it is recited at a high-level of generality with regard to how the test itself is carried out beyond using the control software, and since the “determining scenarios” generically uses the results of the field tests. The “carrying out simulations” amounts to insignificant data gathering since it is recited at a high-level of generality with regard to how the simulation itself is carried out beyond being linked to the scenarios, and since the “determining … a collision rate” uses the results of the simulations in a highly-generic manner. The “carrying out simulations” also amounts to reciting the words “apply it” since it recites running simulations at a high-level of generality from the “determining scenarios”. The claim is directed to an abstract idea. At Step 2B the claim does not recite additional elements that, alone or in an ordered combination, are sufficient to amount to significantly more than the judicial exception. The recited “carrying out field tests” covers well-understood, routine, and conventional activity since they are not explicitly disclosed as requiring anything more than generic sensors and interfaces operating in combination with generic control software, and said disclosure being enabling to one of ordinary skill in the art (see the instant application Figure 1 and 2 and Page 11, Lines 22 – 28 “During this field test, the HAV drives fully autonomously ("closed loop"), but is monitored by a safety driver. The sensor data or internal system states of the HAV at a suitable interface are typically recorded and criticality metrics may for example be subsequently calculated or simulation-based tests carried out.”) (see MPEP 2164.01 “A patent need not teach, and preferably omits, what is well known in the art. In re Buchner, 929 F.2d 660, 661, 18 USPQ2d 1331, 1332 (Fed. Cir. 1991)”, and MPEP 2164.02). The “carrying out simulations” covers well-understood, routine, and conventional activity since they are not explicitly disclosed as requiring anything more than generic virtual environment simulation and desired models, and said disclosure being enabling to one of ordinary skill in the art (see the instant application Page 13, Lines 11 – 15 “Simulation-based HAV testing ("simulation"): HAV testing in a virtual environment, wherein the vehicle surroundings and the interaction of the HAV therewith are modeled by models (for example sensor models and models of road user behavior”) (see MPEP 2164.01 “A patent need not teach, and preferably omits, what is well known in the art. In re Buchner, 929 F.2d 660, 661, 18 USPQ2d 1331, 1332 (Fed. Cir. 1991)”, and MPEP 2164.02). Alternatively, the “carrying out simulations” amount to reciting the words “apply it”. Considering the additional elements in combination does not add anything more than when considering them individually since they do not directly interact with each other and merely provide data for subsequent steps. For at least these reasons, the claim is not patent eligible. Dependent claim 9 - 11 recite(s) at Step 1 the same statutory category as the parent claim(s), and further recite(s): Claim 9 wherein the criticality is specified in such a way that the scenarios include scenarios in which no collision has occurred; Claim 10 determining collision rates, and an average collision rate from the determined collision rates, for each degree of collision severity for at least one degree of collision severity and determining the collision risk from the average collision rate for each degree of collision severity; Claim 11 determining an average collision rate from the determined collision rates; determining an extrapolated collision rate across the determined scenarios from; results of the field tests by statistical extrapolation; and comparing the average collision rate with the determined extrapolated collision rate. At Step 2A, Prong I the recited limitations, alone or in combination, amount to steps that, under its broadest reasonable interpretation, cover performance of the limitations in the mind in combination with using a pen and paper (see MPEP 2106.04(a)(2)(III)) . For example, the “criticality is specified” further limits the parent claim “determining” and without precluding performance in the mind. The “determining” covers analytical processes requiring no more than judgements and evaluations based on observations of determined collision rates and/or scenarios. The ”comparing” covers analytical processes requiring no more than judgements and evaluations based on previously determined quantities. Accordingly, the claim recites an abstract idea. At Step 2A, Prong II this judicial exception is not integrated into a practical application since the claimed invention does not further recite any limitations. The claim is directed to an abstract idea. At Step 2B the claim does not recite additional elements that, alone or in an ordered combination, are sufficient to amount to significantly more than the judicial exception since there are no further recited limitations. For at least these reasons, the claim is not patent eligible. Dependent claim 12 recite(s) at Step 1 the same statutory category as the parent claim(s). Accordingly, the claim recites an abstract idea. At Step 2A, Prong II this judicial exception is not integrated into a practical application since the claimed invention further claims: Claim 12 wherein for each of the determined scenarios, a Monte Carlo simulation is carried out in which parameters of the determined scenario are randomly varied. The “simulation is carried out” amounts to reciting the words “apply it” since it recited running simulations at a high-level of generality from the “determined scenarios”. The claim is directed to an abstract idea. At Step 2B the claim does not recite additional elements that, alone or in an ordered combination, are sufficient to amount to significantly more than the judicial exception. The “simulation is carried out” amounts to reciting the words “apply it”. Considering the additional elements in combination does not add anything more than when considering them individually since they do not directly interact with each other. For at least these reasons, the claim is not patent eligible. Independent claim 13 recites at Step 1 a statutory category (i.e. a machine) validating apparatus configured to: determine scenarios in which events with at least one specified criticality have arisen in the field tests, and determine , for each determined scenario of the determined scenarios, a frequency with which the determined scenario including an event with at least the specified criticality occurs; determine , from the simulations, a collision rate for each of the determined scenarios; combine the determined collision rates into an average collision risk over all the determined scenarios, taking into account the determined frequencies; and validate the control software based on a comparison of the average collision risk with a safety criterion. At Step 2A, Prong I the recited limitations, alone or in combination, amount to steps that, under its broadest reasonable interpretation, cover performance of the limitations in the mind in combination with using a pen and paper (see MPEP 2106.04(a)(2)(III)) . For example, the “determine” covers analytical processes requiring no more than judgements and evaluations based on observations of field tests and/or simulations carried out. The ”combine” covers analytical processes requiring no more than judgements and evaluations based on previously determined quantities. The “validate” covers analytical processes requiring no more than “comparison” between known quantities, which itself requires no more than judgements and evaluations. Accordingly, the claim recites an abstract idea. At Step 2A, Prong II this judicial exception is not integrated into a practical application since the claimed invention further claims: the structure for performing the “receive” and “determine” and “carry out simulations” and “combine” and “validate” (see Claim Rejections - 35 USC § 112); receive field test data from field tests carried out using a control software for a robotic device; carry out simulations for each of the determined scenarios. The “structure” is generic such that it amounts to no more than mere application of the judicial exception using generic computer components and/or sensors with interfaces which does not amount to an improvement in computer or sensor functionality (see MPEP 2106.04(a)(I)) . The “receive field tests” amounts to insignificant data gathering since it is recited at a high-level of generality with regard to how the data is received. The “carry out simulations” amounts to insignificant data gathering since it is recited at a high-level of generality with regard to how the simulation itself is carried out beyond being linked to the scenarios, and since the “determine … a collision rate” uses the results of the simulations in a highly-generic manner. The “carry out simulations” also amounts to reciting the words “apply it” since it recites running simulations at a high-level of generality from the “determine scenarios”. The claim is directed to an abstract idea. At Step 2B the claim does not recite additional elements that, alone or in an ordered combination, are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the “structure” (see Claim Rejections - 35 USC § 112) amounts to no more than mere instructions to apply the judicial exception using generic computer components and/or sensors with interfaces. Mere instructions to apply an exception using a generic computer components and/or sensors with interfaces cannot provide an inventive concept. The recited “receive field test data” covers well-understood, routine, and conventional activity since it coves receiving data using any electronic means (see MPEP 2106.05(d)(II)(i) “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”). The ”carry out simulations” covers well-understood, routine, and conventional activity since they are not explicitly disclosed as requiring anything more than generic virtual environment simulation and desired models, and said disclosure being enabling to one of ordinary skill in the art (see the instant application Page 13, Lines 11 – 15 “Simulation-based HAV testing ("simulation"): HAV testing in a virtual environment, wherein the vehicle surroundings and the interaction of the HAV therewith are modeled by models (for example sensor models and models of road user behavior”) (see MPEP 2164.01 “A patent need not teach, and preferably omits, what is well known in the art. In re Buchner, 929 F.2d 660, 661, 18 USPQ2d 1331, 1332 (Fed. Cir. 1991)”, and MPEP 2164.02). Alternatively, the “carry out simulations” amount to reciting the words “apply it”. Considering the additional elements in combination does not add anything more than when considering them individually since they do not directly interact with each other and merely provide data for subsequent steps. For at least these reasons, the claim is not patent eligible. Independent claim 14 recites at Step 1 a statutory category (i.e. a manufacture) non-transitory computer-readable medium on which are stored commands, the commands, when executed by a processor, cause the processor to perform the following steps: determining scenarios in which events with at least one specified criticality have arisen in the field tests, and determining , for each determined scenario of the determined scenarios, a frequency with which the determined scenario including an event with at least the specified criticality occurs; determining , from the simulations, a collision rate for each of the determined scenarios; combining the determined collision rates into an average collision risk over all the determined scenarios, taking into account the determined frequencies; and validating the control software based on a comparison of the average collision risk with a safety criterion. At Step 2A, Prong I the recited limitations, alone or in combination, amount to steps that, under its broadest reasonable interpretation, cover performance of the limitations in the mind in combination with using a pen and paper (see MPEP 2106.04(a)(2)(III)) . For example, the “determining” covers analytical processes requiring no more than judgements and evaluations based on observations of field tests and/or simulations carried out. The ”combining” covers analytical processes requiring no more than judgements and evaluations based on previously determined quantities. The “validating” covers analytical processes requiring no more than “comparison” between known quantities, which itself requires no more than judgements and evaluations. Accordingly, the claim recites an abstract idea. At Step 2A, Prong II this judicial exception is not integrated into a practical application since the claimed invention further claims: receiving field test data from field tests carried out using a control software for a robotic device; carrying out simulations for each of the determined scenarios. The “receiving field tests” amounts to insignificant data gathering since it is recited at a high-level of generality with regard to how the data is received. The “carrying out simulations” amounts to insignificant data gathering since it is recited at a high-level of generality with regard to how the simulation itself is carried out beyond being linked to the scenarios, and since the “determining … a collision rate” uses the results of the simulations in a highly-generic manner. The “carry out simulations” also amounts to reciting the words “apply it” since it recites running simulations at a high-level of generality from the “determine scenarios”. The claim is directed to an abstract idea. At Step 2B the claim does not recite additional elements that, alone or in an ordered combination, are sufficient to amount to significantly more than the judicial exception. The recited “receiving field test data” covers well-understood, routine, and conventional activity since it coves receiving data using any electronic means (see MPEP 2106.05(d)(II)(i) “i. Receiving or transmitting data over a network, e.g., using the Internet to gather data”). The ”carrying out simulations” covers well-understood, routine, and conventional activity since they are not explicitly disclosed as requiring anything more than generic virtual environment simulation and desired models, and said disclosure being enabling to one of ordinary skill in the art (see the instant application Page 13, Lines 11 – 15 “Simulation-based HAV testing ("simulation"): HAV testing in a virtual environment, wherein the vehicle surroundings and the interaction of the HAV therewith are modeled by models (for example sensor models and models of road user behavior”) (see MPEP 2164.01 “A patent need not teach, and preferably omits, what is well known in the art. In re Buchner, 929 F.2d 660, 661, 18 USPQ2d 1331, 1332 (Fed. Cir. 1991)”, and MPEP 2164.02). Alternatively, the “carrying out simulations” amount to reciting the words “apply it”. Considering the additional elements in combination does not add anything more than when considering them individually since they do not directly interact with each other and merely provide data for subsequent steps. For at least these reasons, the claim is not patent eligible. Claim Rejections - 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries set forth in Graham v. John Deere Co. , 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: Determining the scope and contents of the prior art. Ascertaining the differences between the prior art and the claims at issue. Resolving the level of ordinary skill in the pertinent art. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claim s 8 – 10 and 12 - 14 are rejected under 35 U.S.C. 103 as being unpatentable over Olson et al. (US 12187322) (henceforth “Olson (322)”) in view of Azevedo et al. “Using Extreme Value Theory For the Prediction of Head-On Collisions During Passing Maneuvres” (henceforth “Azevedo”), and further in view of Feng et al. “Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology” (henceforth “Feng”). Olson (322) and Feng are analogous art because they solve the same problem of validating a vehicle control software risk of collision, and because they are from the same field of endeavor of vehicle control software design. Azevedo is from the same field of endeavor of vehicle safety evaluation . With regard to claim 8 , Olson (322) teaches a method for validating control software for a robotic device, comprising the following steps: (Olson (322) Col. 1, Line 67 – Col. 2, Line 4 “Simulations can be used to validate software (e.g., a vehicle controller) executed on vehicles ( e.g., autonomous vehicles) and gather safety metrics to ensure that the software is able to safely control such vehicles before being used in a real-world environment”) carrying out field tests using the control software; (Olson (322) Col. 4, Lines 15 – 19 real-world behavior is obtained from recording vehicles “In such an example, the computing device can be configured to introduce behavior consistent with real-world behavior or actions by other objects ( e.g., pedestrians, bikes, motorcycles, etc.) and vehicles that may be based off of stored log data previously recorded by a fleet of autonomous vehicles”) determining scenarios in which events have arisen in the field tests (Olson (322) Col. 10, Lines 39 – 41 “The remote computing device(s) may identify one or more scenarios based at least in part on the log data, which may also comprise defining a scenario”) carrying out simulations for each of the determined scenarios; determining, from the simulation, a collision data for each of the determined scenarios (Olson (322) Figure 2 scenarios are simulated and a collision data determined) validating the control software based on a comparison of an average collision risk with a safety criterion. (Olson (322) Col 2, Lines 50 – 54 collision data is compared against related standards including meeting thresholds “The collision data may be compared against metrics or standards to ensure adherence to or performance in excess of safety standards and thresholds before control systems or updates are implemented in real-world applications.”, and Col. 20, Lines 13 – 15 the collision data comprises an average severity (average collision risk)“The different severity scores for each of the different collision types may be aggregated by performing one or more statistical analyses, for example to determine an average, standard deviation, mode, or other such statistical information that may be used to aggregate the various severity scores”) Olson (322) does not appear to explicitly disclose: determining scenarios in which events with at least one specified criticality have arisen , and determining, for each determined scenario of the determined scenarios, a frequency with which the determined scenario including an event with at least the specified criticality occurs; that the determining, from the simulations, is a collision rate . However, Azevedo teaches: determining scenarios in which events with at least one specified criticality have arisen (Azevedo Page 270, Right tests are filtered according to TTC threshold to identify crash surrogates from a larger set of passing maneuvers “Furthermore, past studies concluded that with minimum TTC smaller than a low limit (typically, 1 to 1.5 s) are useful as crash surrogates … resulting in 463 maxima”) determining, for each determined scenario of the determined scenarios, a frequency with which the determined scenario including an event with at least the specified criticality occurs; (Azevedo Figure 2 relative frequency of all scenarios having the threshold TTC is instantly determined from the graph PNG media_image1.png 228 269 media_image1.png Greyscale ) determining, from the simulations, a collision rate for each of the determined scenarios; (Azevedo Page 271, Left proportion of crash surrogates resulting in actual collision are determined “In other words, the probability for a head-on collision assuming a near head-on collision in a passing maneuver is 9/463= 0.0194,”) It would have been obvious to one of ordinary skill in the art to combine the method simulating AV scenarios based on log data to validate controller disclosed by Olson (322) with selection of crash surrogate scenarios disclosed by Azevedo. One of ordinary skill in the art would have been motivated to make this modification in order to obtain more scenarios that are relevant to crash simulation (Azevedo Abstract “Crash data suffer from underreporting and frequently poor quality”). Olson (322) in view of Azevedo does not appear to explicitly disclose: combining the determined collision rates into an average collision risk over all the determined scenarios, taking into account the determined frequencies. However, Feng teaches: combining determined collision rates into an average collision risk over all determined scenarios, taking into account determined frequencies; (Feng Page 4, Right conditional rate of accident for each cut-in scenario is used in a weighted average to get the overall accident rate over all the scenarios, where this can be used in Olson (322) to compare to safety thresholds PNG media_image2.png 107 320 media_image2.png Greyscale ) It would have been obvious to one of ordinary skill in the art to combine the method simulating AV scenarios based on log data and criticality events disclosed by Olson (322) in view of Azevedo with the computation accident rate over all scenarios disclosed by Feng. One of ordinary skill in the art would have been motivated to make this modification in order to obtain more scenarios that are relevant to improve estimation of an overall crash rate (Feng Page 4, Right “To improve the estimation efficiency, the importance sampling technique was introduced by [14]”). With regard to claim 13 , it recites the same steps as claim 8 which is taught by Olson (322) in view of Azevedo, and further in view of Feng. Claim 13 further recites: a validating apparatus configured to receive field test data from field tests carried out using a control software for a robotic device; Olson (322) teaches: a validating apparatus configured to (see Claim Rejections - 35 USC § 112) (Olson (322) Col. 1, Line 67 – Col. 2, Line 4 “Simulations can be used to validate software (e.g., a vehicle controller) executed on vehicles ( e.g., autonomous vehicles) and gather safety metrics to ensure that the software is able to safely control such vehicles before being used in a real-world environment”, and Figure 7 computing device) receive field test data from field tests carried out using a control software for a robotic device (see Claim Rejections - 35 USC § 112) (Olson (322) Col. 4, Lines 15 – 19 real-world behavior is obtained from recording vehicles “In such an example, the computing device can be configured to introduce behavior consistent with real-world behavior or actions by other objects ( e.g., pedestrians, bikes, motorcycles, etc.) and vehicles that may be based off of stored log data previously recorded by a fleet of autonomous vehicles”) With regard to claim 14 , it recites the same steps as claim 8 which is taught by Olson (322) in view of Azevedo, and further in view of Feng. Claim 14 further recites: a non-transitory computer-readable medium on which are stored commands, the commands, when executed by a processor, cause the processor to perform the following steps: receiving field test data from field tests carried out using a control software for a robotic device. Olson (322) teaches: a non-transitory computer-readable medium on which are stored commands, the commands, when executed by a processor, cause the processor to perform the following steps: (Olson (322) Col. 1, Line 67 – Col. 2, Line 4 “Simulations can be used to validate software (e.g., a vehicle controller) executed on vehicles ( e.g., autonomous vehicles) and gather safety metrics to ensure that the software is able to safely control such vehicles before being used in a real-world environment”, and Figure 7 computing device comprising memory and processor) receiving field test data from field tests carried out using a control software for a robotic device. (Olson (322) Col. 4, Lines 15 – 19 real-world behavior is obtained from recording vehicles “In such an example, the computing device can be configured to introduce behavior consistent with real-world behavior or actions by other objects ( e.g., pedestrians, bikes, motorcycles, etc.) and vehicles that may be based off of stored log data previously recorded by a fleet of autonomous vehicles”) With regard to claim 9 , Olson (322) in view of Azevedo, and further in view of Feng teaches all the elements of the parent claim 8, and further teaches: wherein the criticality is specified in such a way that the scenarios include scenarios in which no collision has occurred. (Azevedo Page 271, Left “Out of the 463 near head-on collisions in the driving simulator (using the threshold of 1.5 s), 9 maneuvers ended with actual collisions”) It would have been obvious to one of ordinary skill in the art to combine the method simulating AV scenarios based on log data to validate controller disclosed by Olson (322) with selection of crash surrogate scenarios disclosed by Azevedo. One of ordinary skill in the art would have been motivated to make this modification in order to obtain more scenarios that are relevant to crash simulation (Azevedo Abstract “Crash data suffer from underreporting and frequently poor quality”). With regard to claim 10 , Olson (322) in view of Azevedo, and further in view of Feng teaches all the elements of the parent claim 8, and further teaches: determining collision rates, and an average collision rate from the determined collision rates, for each degree of collision severity for at least one degree of collision severity and determining the collision risk from the average collision rate for each degree of collision severity. (Feng Page 5, Right criticality of scenario depends on maneuver challenge (degree of severity) PNG media_image3.png 130 399 media_image3.png Greyscale , and Page 6, Right a subset of scenarios are deemed critical (at least one degree of collision severity), where the claim scope covers only one degree of collision severity PNG media_image4.png 216 400 media_image4.png Greyscale ) It would have been obvious to one of ordinary skill in the art to combine the method simulating AV scenarios based on log data and criticality events disclosed by Olson (322) in view of Azevedo with the computation accident rate over critical scenarios having maneuver challenge disclosed by Feng. One of ordinary skill in the art would have been motivated to make this modification in order to analyze more critical scenarios separately (Feng Page 5, Right). With regard to claim 12 , Olson (322) in view of Azevedo, and further in view of Feng teaches all the elements of the parent claim 8, and further teaches: wherein for each of the determined scenarios, a Monte Carlo simulation is carried out in which parameters of the determined scenario are randomly varied. (Feng Page 8, Left scenarios are sampled and simulated using Monte Carlo principles PNG media_image5.png 118 329 media_image5.png Greyscale ) It would have been obvious to one of ordinary skill in the art to combine the method simulating AV scenarios based on log data and criticality events disclosed by Olson (322) in view of Azevedo with the computation accident rate over all scenarios disclosed by Feng. One of ordinary skill in the art would have been motivated to make this modification in order to obtain more scenarios that are relevant to improve estimation of an overall crash rate (Feng Page 4, Right “To improve the estimation efficiency, the importance sampling technique was introduced by [14]”) . 07-21-aia AIA Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Olson (322) in view of Azevedo, and further in view of Feng, and further in view of Junietz, P. “Microscopic and Macroscopic Risk Metrics for the Safety Validation of Automated Driving” (henceforth “Junietz (Thesis)”) . Olson (322) and Feng and Junietz (Thesis) are analogous art because they solve the same problem of validating a vehicle control software risk of collision, and because they are from the same field of endeavor of vehicle control software design. Azevedo is from the same field of endeavor of vehicle safety evaluation . With regard to claim 11 , Olson (322) in view of Azevedo, and further in view of Feng teaches all the elements of the parent claim 8, and further teaches: determining an average collision rate from the determined collision rates; (Feng Page 4, Right conditional rate of accident for each cut-in scenario is used in a weighted average to get the overall accident rate over all the scenarios PNG media_image2.png 107 320 media_image2.png Greyscale ) Olson (322) in view of Azevedo, and further in view of Feng does not appear to explicitly disclose: determining an extrapolated collision rate across the determined scenarios from results of the field tests by statistical extrapolation; and comparing the average collision rate with the determined extrapolated collision rate. However, Junietz (Thesis) teaches: determining an extrapolated collision rate across the determined scenarios from results of the field tests by statistical extrapolation; and (Junietz (Thesis) Page 26 and Figure 6-3 collision rate can be extrapolated to larger equivalent field tests using statistical evaluation “One of the reasons why the required driving distance in real world testing is so high is the low frequency of accidents from which the SP is estimated … If, instead of accidents, critical scenes could be used as a basis for statistical evaluation, evaluation could be done earlier or faster, as the number of events per distance is higher.”) comparing an average collision rate with the determined extrapolated collision rate. (Junietz (Thesis) VIII control software could be evaluated for decreasing accidents compared to normal drivers (comparing) “The common expectation is that the introduction of automated driving will reduce the number of accidents at least long-term and per mileage. At the same time, it is obvious that the introduction will induce new risks to the live of modern society, as almost every new technology does.”, and Page 101 extrapolation is compared with average accident frequency PNG media_image6.png 174 809 media_image6.png Greyscale ) It would have been obvious to one of ordinary skill in the art to combine the method simulating AV scenarios based on log data and criticality events disclosed by Olson (322) in view of Azevedo, and further in view of Feng with the extrapolation and analysis of collision rates by bounding disclosed by Junietz (Thesis). One of ordinary skill in the art would have been motivated to make this modification in order to understand the true accident rate from limited data (Junietz (Thesis) Page 101). Examiner General Comments With regard to the prior art rejection(s), any cited portion of the relied upon reference(s), either by pointing to specific sections or as quotations, is intended to be interpreted in the context of the reference(s) as a whole as would be understood by one of ordinary skill in the art. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the applicant fully consider the references in their entirety as potentially teaching all or part of the claimed invention since the entire reference is considered to provide disclosure relating to the cited portions. Further, the claims and only the claims form the metes and bounds of the invention. Office personnel are to give the claims their broadest reasonable interpretation in light of the supporting disclosure. Unclaimed limitations appearing in the specification are not read into the claim. Prior art was referenced using terminology familiar to one of ordinary skill in the art. Such an approach is broad in concept and can be either explicit or implicit in meaning. Examiner’s notes are provided with the cited references to assist the applicant to better understand how the examiner interprets the applied prior art. Such comments are entirely consistent with the intent and spirit of compact prosecution . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Hruschka, C. (US 12503137) teaches the separation into collision probability and accident severity provides for the separation between preventive driving and emergency maneuvers. Kavalar, A. (US 11940793) teaches simulations may be based on log data developed from real world driving events to, for example, accurately model a likelihood that a scenario will occur during real-world driving.” Zhao et al. “Accelerated Evaluation of Automated Vehicles Safety in Lane-Change Scenarios Based on Importance Sampling Techniques” teaches a proposed accelerated evaluation approach for AV evaluation. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALFRED H. WECHSELBERGER whose telephone number is (571)272-8988. The examiner can normally be reached M - F, 10am to 6pm. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emerson Puente can be reached at 571-272-3652. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /ALFRED H B WECHSELBERGER/Examiner, Art Unit 2187 Application/Control Number: 18/178,413 Page 2 Art Unit: 2187 Application/Control Number: 18/178,413 Page 3 Art Unit: 2187 Application/Control Number: 18/178,413 Page 4 Art Unit: 2187 Application/Control Number: 18/178,413 Page 5 Art Unit: 2187 Application/Control Number: 18/178,413 Page 6 Art Unit: 2187 Application/Control Number: 18/178,413 Page 7 Art Unit: 2187 Application/Control Number: 18/178,413 Page 8 Art Unit: 2187 Application/Control Number: 18/178,413 Page 9 Art Unit: 2187 Application/Control Number: 18/178,413 Page 10 Art Unit: 2187 Application/Control Number: 18/178,413 Page 11 Art Unit: 2187 Application/Control Number: 18/178,413 Page 12 Art Unit: 2187 Application/Control Number: 18/178,413 Page 13 Art Unit: 2187 Application/Control Number: 18/178,413 Page 14 Art Unit: 2187 Application/Control Number: 18/178,413 Page 15 Art Unit: 2187 Application/Control Number: 18/178,413 Page 16 Art Unit: 2187 Application/Control Number: 18/178,413 Page 17 Art Unit: 2187 Application/Control Number: 18/178,413 Page 18 Art Unit: 2187 Application/Control Number: 18/178,413 Page 19 Art Unit: 2187 Application/Control Number: 18/178,413 Page 20 Art Unit: 2187 Application/Control Number: 18/178,413 Page 21 Art Unit: 2187 Application/Control Number: 18/178,413 Page 22 Art Unit: 2187 Application/Control Number: 18/178,413 Page 23 Art Unit: 2187 Application/Control Number: 18/178,413 Page 24 Art Unit: 2187 Application/Control Number: 18/178,413 Page 25 Art Unit: 2187 Application/Control Number: 18/178,413 Page 26 Art Unit: 2187
Prosecution Insights