Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 6/30/2026 has been entered.
Response to Amendment
The previous claim objections are withdrawn due to Applicant’s amendments.
Response to Arguments
Applicant’s arguments on pages 9-11 regarding the rejection under 35 U.S.C. 103 with respect to claims 1, 3, 6-8, 10-15 and 17-24 have been fully considered but are moot. New references Shkurti and Kehl have been incorporated to teach the newly presented limitations.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Shkurti et al. (US 20200089247 A1); hereinafter Shkurti in view of Kehl et al. (US 20190347515 A1); hereinafter Kehl in view of Feng et al. (Testing Scenario Library Generation for Connected and Automated Vehicles: An Adaptive Framework); hereinafter Feng
Claim 1 is rejected over Shkurti, Kehl and Feng.
Regarding claim 1, Shkurti teaches an autonomous vehicle (AV), comprising:
a vehicle; (Shkurti [Abstract]: “A method and apparatus for generating adversarial scenarios and training an autonomous driving agent for an autonomous vehicle, using one or more sets of parameters, each set of parameters defining a respective driving scenario.”)
a processor; and
a memory, where the memory contains an AV model capable of driving the vehicle without human input; (Shkurti [0016]: “the present disclosure describes a processing unit that includes: a processor; and a memory coupled to the processor, the memory storing machine-executable instructions of an autonomous driving agent for an autonomous vehicle”)
the AV model is iteratively trained on a set of edge case scenarios [drawn from clusters of edge case scenarios in the risk manifold,] (Shkurti [0007]: “the generation of adversarial scenarios is based on the history of the autonomous driving agent's past performance in respect of prior adversarial scenarios. In one example aspect, a method and system for generating adversarial scenarios and training an autonomous driving agent uses a scenario to improve performance of an autonomous driving agent; progressively changes selected parameters that define the scenario until the autonomous driving agent cannot satisfactorily perform in the scenario defined by the changed parameters”; and “[0059]: “Evaluation of the trained ADA 105 may then be carried out using scenario(s) that may be sampled from only the non-training scenarios, or that may be sampled from both the training scenarios as well as the non-training scenarios (optionally with weighting to adjust the ratio of training scenarios to non-training scenarios in the sample”)
Shkurti does not appear to explicitly teach where the AV model is trained on a plurality of edge case scenarios encoded in a risk manifold,
where the risk manifold encodes a distance metric between edge case scenarios in the risk manifold, and
drawn from clusters of edge case scenarios in the risk manifold,
However, Kehl teaches where the AV model is trained on a plurality of edge case scenarios encoded in a risk manifold, (Kehl [0026]: “Training of the N-dimensional manifold space and the object clusters may be performed for providing smooth representations of the object categories and corresponding poses. In aspects of the present disclosure, a smoothness of the representations of the object categories and corresponding poses enable estimating a category and a pose of similar objects that have not yet been seen by the system.”)
where the risk manifold encodes a distance metric between edge case scenarios in the risk manifold, and (Kehl [0016]: “FIG. 6 illustrates a 3D manifold space configured to separately define object clusters with aggregated object poses, in which the object clusters are separated by a predetermined distance,”)
drawn from clusters of edge case scenarios in the risk manifold, (Kehl [0016]: “FIG. 6 illustrates a 3D manifold space configured to separately define object clusters with aggregated object poses, in which the object clusters are separated by a predetermined distance,”)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the manifold learning of Kehl to improve confidence associated with identifying objects in scenarios. “The object identification process performed by the object detector 430 to identify an object and estimate an object pose in conjunction with the manifold network 420 may significantly improve a confidence associated with identifying unknown objects and their corresponding poses. (Kehl, [0049]). Shkurti and Kehl are analogous art because they both concern analyzing scenarios for autonomous vehicles.
Shkurti does not appear to explicitly teach where a distribution of edge case scenarios in the set is altered based on distance metrics at each iterative step to expand subspaces in which the AV model underperforms while using an unchanged version of the risk manifold as a reference.
However, Feng teaches where a distribution of edge case scenarios in the set is altered based on distance metrics at each iterative step to expand subspaces in which the AV model underperforms while using an unchanged version of the risk manifold as a reference. (Feng [page 5, B. Dissimilarity Function Estimation]: “be represented by the GP as
f
x
∼GP(m(x), k(x, x ‘)), where both x and x’ denote scenarios, m(x) denotes the mean function, and k(x, x’) denotes the covariance function.”; Note: This is the distance metric between two scenario points x and x’; [page 4, Algorithm 1]: Step 2.1: Obtain the estimation … Step 2.2: Update SM and library … Step 2.3: Decide next iteration of testing scenarios; [page 1]: “Underweight scenarios represent the critical scenarios that are ignored by the library, and overweight scenarios represent the uncritical scenarios that are included in the library. If we denote the scenario library generated by using the SM as “offline generated library”, and a customized library that includes all critical scenarios specifically designed for a CAV as “optimal library”, the differences between these two libraries include both underweight and overweight scenarios.”; Note: The underweight scenarios are where the CAV underperforms. The offline library is the unchanged version of a reference.)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the scenario library of Feng for efficient automated vehicle testing. “To compensate for the performance dissimilarities and leverage each test of the CAV, Bayesian optimization techniques are applied with classification-based Gaussian Process Regression and a newly designed acquisition function. Comparing with a pre-determined library, a CAV can be tested and evaluated in a more efficient manner with the customized library (Feng, Abstract).” Shkurti and Feng are analogous art because they both concern evaluating driving scenarios for autonomous vehicles.
Claim 6 is rejected over Shkurti, Kehl and Feng with the incorporation of claim 1.
Regarding claim 6, Shkurti teaches wherein the AV model is a perceptual subsystem. (Shkurti [0076]: “the ADA 105 may include several independent rules-based and/or learning-based functions and modules ( e.g. systems 120, 130, 140). Accordingly, in some examples, training and evaluation of ADA 105 may be focused on selectively training one or more individual sub-system agents of the ADA 105 and specific scenarios 318 may be focused for training specific individual sub-system agents. For example, scenarios could be generated that are targeted for specifically training a Lidar point cloud analysis sub-system agent of the state estimation system 120 to detect object boundaries.”)
Claims 3, 8, 13, 15, 20 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Shkurti, Kehl and Feng in view of Danna et al. (US 20210403036 A1); hereinafter Danna
Claim 3 is rejected over Shkurti, Kehl, Feng and Danna with the incorporation of claim 1.
Regarding claim 3, Shkurti does not appear to explicitly teach wherein the distance is a scalar valued dimensional reduction of data associated with edge case scenarios.
However, Danna teaches wherein the distance is a scalar valued dimensional reduction of data associated with edge case scenarios. (Danna [0012]: “In an embodiment, a machine learning model can be trained with an anchor representation comprising a first encoded image representing a scenario, a positive representation comprising a second encoded image representing a scenario that has a threshold level of similarity (distance) to the anchor representation, and a negative representation comprising a third encoded image representation of a scenario that does not have the threshold level of similarity to the anchor representation.”; Note: An encoded scenario is a compressed lower dimensional value)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the scenario encoding of Danna to improve the organization of different types of scenarios. “Thus, an improved approach that indexes or maintains scenario data of different types of scenarios that negates the need for the developers and searchers to keep up with the taxonomy structure is desired.” Shkurti and Danna are analogous art because they both concern evaluating scenarios on autonomous vehicles.
Claim 8 is rejected over Shkurti, Kehl, Feng and Danna.
Regarding claim 8, Shkurti teaches a system for training autonomous vehicles (AVs), comprising:
a processor; and
a memory, containing an AV training application that directs the processor to: (Shkurti [0016]: “the present disclosure describes a processing unit that includes: a processor; and a memory coupled to the processor, the memory storing machine-executable instructions of an autonomous driving agent for an autonomous vehicle”)
iteratively train the AV model on the hazard frames on a set of edge case scenarios [drawn from clusters of edge case scenarios in the manifold data structure,] (Shkurti [0007]: “the generation of adversarial scenarios is based on the history of the autonomous driving agent's past performance in respect of prior adversarial scenarios. In one example aspect, a method and system for generating adversarial scenarios and training an autonomous driving agent uses a scenario to improve performance of an autonomous driving agent; progressively changes selected parameters that define the scenario until the autonomous driving agent cannot satisfactorily perform in the scenario defined by the changed parameters”; and “[0059]: “Evaluation of the trained ADA 105 may then be carried out using scenario(s) that may be sampled from only the non-training scenarios, or that may be sampled from both the training scenarios as well as the non-training scenarios (optionally with weighting to adjust the ratio of training scenarios to non-training scenarios in the sample”)
Shkurti does not appear to explicitly teach obtain a manifold data structure storing a plurality of scenarios that an AV can encounter, and distance metrics indicating the distance between each scenario;
drawn from clusters of edge case scenarios in the risk manifold,
However, Kehl teaches obtain a manifold data structure storing a plurality of scenarios that an AV can encounter, and distance metrics indicating the distance between each scenario; (Kehl [0016]: “FIG. 6 illustrates a 3D manifold space configured to separately define object clusters with aggregated object poses, in which the object clusters are separated by a predetermined distance,”)
drawn from clusters of edge case scenarios in the risk manifold, (Kehl [0016]: “FIG. 6 illustrates a 3D manifold space configured to separately define object clusters with aggregated object poses, in which the object clusters are separated by a predetermined distance,”)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the manifold learning of Kehl to improve confidence associated with identifying objects in scenarios. “The object identification process performed by the object detector 430 to identify an object and estimate an object pose in conjunction with the manifold network 420 may significantly improve a confidence associated with identifying unknown objects and their corresponding poses. (Kehl, [0049]). Shkurti and Kehl are analogous art because they both concern analyzing scenarios for autonomous vehicles.
Shkurti does not appear to explicitly teach generate a list of edge case scenarios within the plurality of scenarios;
identify hazard frames within the edge case scenarios;
encode the hazard frames into one or more records interpretable by an AV model; and
However, Danna teaches generate a list of edge case scenarios within the plurality of scenarios; (Danna [0016]: “the one or more high-level primitives can be capable of being used to identify the one or more identified scenarios in lieu of including the low-level parameters in the search query.”)
identify hazard frames within the edge case scenarios; (Danna [0066]: “The example scenario 750 comprises a first vehicle 758 a, a second vehicle 760 a, and a cyclist 762 a navigating an intersection 752 controlled by at least one traffic light 754. Each of the vehicles 758 a, 760 a and the cyclist 762 a are moving (or predicted to move) along respective trajectories 758 b, 760 b, and 762 b. The cyclist 762 a is crossing a crosswalk 764.”)
encode the hazard frames into one or more records interpretable by an AV model; and (Danna [0004]: “the embedding of the at least one representation of the at least one example scenario can be generated within a vector space, and the embedding representing the at least one scenario can be included within the vector space.”)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the scenario encoding of Danna to improve the organization of different types of scenarios. “Thus, an improved approach that indexes or maintains scenario data of different types of scenarios that negates the need for the developers and searchers to keep up with the taxonomy structure is desired.” Shkurti and Danna are analogous art because they both concern evaluating scenarios on autonomous vehicles.
Shkurti does not appear to explicitly teach where a distribution of edge case scenarios the set is altered based on the distance metrics at each iterative step to expand subspaces in which the AV model underperforms while using an unchanged version of the manifold data structure as a reference; and
However, Feng teaches where a distribution of edge case scenarios the set is altered based on the distance metrics at each iterative step to expand subspaces in which the AV model underperforms while using an unchanged version of the manifold data structure as a reference; and (Feng [page 5, B. Dissimilarity Function Estimation]: “be represented by the GP as
f
x
∼GP(m(x), k(x, x ‘)), where both x and x’ denote scenarios, m(x) denotes the mean function, and k(x, x’) denotes the covariance function.”; Note: This is the distance metric between two scenario points x and x’; [page 4, Algorithm 1]: Step 2.1: Obtain the estimation … Step 2.2: Update SM and library … Step 2.3: Decide next iteration of testing scenarios; [page 1]: “Underweight scenarios represent the critical scenarios that are ignored by the library, and overweight scenarios represent the uncritical scenarios that are included in the library. If we denote the scenario library generated by using the SM as “offline generated library”, and a customized library that includes all critical scenarios specifically designed for a CAV as “optimal library”, the differences between these two libraries include both underweight and overweight scenarios.”; Note: The underweight scenarios are where the CAV underperforms. The offline library is the unchanged version of a reference.)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the scenario library of Feng for efficient automated vehicle testing. “To compensate for the performance dissimilarities and leverage each test of the CAV, Bayesian optimization techniques are applied with classification-based Gaussian Process Regression and a newly designed acquisition function. Comparing with a pre-determined library, a CAV can be tested and evaluated in a more efficient manner with the customized library (Feng, Abstract).” Shkurti and Feng are analogous art because they both concern evaluating driving scenarios for autonomous vehicles.
Claim 13 is rejected over Shkurti, Kehl, Feng and Danna with the incorporation of claim 1.
Regarding claim 13, Shkurti does not appear to explicitly teach wherein the AV model is a decision-making module; and
wherein a loss function used to train the AV model is modulated by the rate of adverse events experienced by an agent on a given set of scenarios.
However, Danna teaches wherein the AV model is a decision-making module; and (Danna [0108]: “Examples of data transmitted from the vehicle 1140 may include, e.g., telemetry and sensor data, determinations/decisions based on such data,”)
wherein a loss function used to train the AV model is modulated by the rate of adverse events experienced by an agent on a given set of scenarios. (Danna [0061]: “FIG. 5 illustrates an example diagram 500 of training a model based on a triplet loss technique, according to an embodiment of the present technology. The triplet loss technique can utilize sets of an anchor representation, a positive representation, and a negative representation as training data. The three representations in a set can be, respectively, an anchor encoded image 504, a positive encoded image 502, and a negative encoded image 506. In this example diagram 500, the anchor encoded image 504 is the example 300 of FIG. 3. “)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the scenario encoding of Danna to improve the organization of different types of scenarios. “Thus, an improved approach that indexes or maintains scenario data of different types of scenarios that negates the need for the developers and searchers to keep up with the taxonomy structure is desired.” Shkurti and Danna are analogous art because they both concern evaluating scenarios on autonomous vehicles.
Claim 15 is claim 8 in the form of a method and is rejected for the same reasons as claim 8 stated above.
Dependent claim 20 is claim 13 in the form of a method and is rejected for the same reasons as claim 13 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Dependent claim 24 is claim 22 in the form of a method and is rejected for the same reasons as claim 22 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Claims 7, 14 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Shkurti, Kehl and Feng in view of Wu et al. (A Pioneering Scalable Self-driving Car Simulation Platform); hereinafter Wu in view of Hospach et al. (Simulation of Falling Rain for Robustness Testing of Video-Based Surround Sensing Systems); hereinafter Hospach in view of Chao et al. (Autonomous Driving: Mapping and Behavior Planning for Crosswalks) Video-Based Surround Sensing Systems); hereinafter Chao and in further view of Rhinehart et al. (DEEP IMITATIVE MODELS FOR FLEXIBLE INFERENCE, PLANNING, AND CONTROL); hereinafter Rhinehart
Claim 7 is rejected over Shkurti, Kehl, Feng, Wu, Hospach, Chao and Rhinehart with the incorporation of claim 1.
Regarding claim 7, Shkurti does not teach wherein a subset of the plurality of edge case scenarios are artificially generated using a method selected from the group consisting of: applying a bandpass filter to sensor data; generating 2-D semi-opaque, semi-reflective, semi-occluding polygons into the scenario data at a position between a sensor source and an event; applying multiscale Gabor patterns to events within simulated scenarios; applying time-varying forces to moving entities within the scenarios; and applying fractal cracking to surfaces within the scenarios.
However, Wu teaches wherein a subset of the plurality of edge case scenarios are artificially generated using a method selected from the group consisting of: applying a bandpass filter to sensor data; (Wu [page 150, A. Lane Detection]: “The filtering process to distinguish the road pixels from the background plays a quite substantial part. Reference [13] just selects the scope of yellow and white color from the images by using LUV [14] and LaB image formats, thus separating the pixels of roads from those of the background. Alternatively, reference [12] uses Gabor filters, which is a best-known quadrature filters. Gabor filter is characterized by Gaussian-formed band pass filters. It is a suitable choice to accomplish functions demanding simultaneous measurement in both space and frequency domains [1]”)
applying multiscale Gabor patterns to events within simulated scenarios; (Wu [page 150, A. Lane Detection]: “The filtering process to distinguish the road pixels from the background plays a quite substantial part. Reference [13] just selects the scope of yellow and white color from the images by using LUV [14] and LaB image formats, thus separating the pixels of roads from those of the background. Alternatively, reference [12] uses Gabor filters, which is a best-known quadrature filters. Gabor filter is characterized by Gaussian-formed band pass filters. It is a suitable choice to accomplish functions demanding simultaneous measurement in both space and frequency domains [1]”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Shkurti iteratively training on scenarios with the Gabor filters of Wu to effectively aid autonomous vehicles in object detection in traffic (Wu, page 150, A. Lane Detection). Shkurti and Wu are analogous art because they both concern evaluating driving scenarios for autonomous vehicles.
Hospach teaches generating 2-D semi-opaque, semi-reflective, semi-occluding polygons into the scenario data at a position between a sensor source and an event; (Hospach [page 235]: “First, the background of the scene is blurred as if it was recorded with the settings of the simulation camera, according to the theory of circle of confusion above. Here, it is very important, that the depth-of-field of the source material is large enough, since already present blur cannot be undone. Second, the rain drops themselves are blurred depending on their depth during the simulation. In this manner, the OpenGL pinhole camera model for projective imaging can be extended as if the camera model had a lens with aperture and focus settings”; page 234 and “This value denotes the transparency value of a rain drop due to its motion blur. The base color of a drop, which in reality is scene and illumination dependent, is then weighted with α. Without knowing details on the scene illumination, the base color of a rain streak has been empirically set to a semi-transparent white with opacity factor 0.5. This has shown to produce good results. If more detail of scene illumination is known, the color value can be specified more accurate, though.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Shkurti iterative training on scenarios with the simulated rain drop shapes of Hospach to effectively test vision-based surround sensing systems in autonomous vehicles (Hospach, page 233, Abstract). Hari and Hospach are analogous arts because they both concern evaluating driving scenarios for autonomous vehicles.
Chao teaches applying time-varying forces to moving entities within the scenarios; and (Chao [page 40]: “The 100 meter dash scenario simulates a pedestrian sprinting across the crosswalk at 44 kph (time-varying forces). The autonomous vehicle travels at the speed limit of 30 kph toward the crosswalk. The sprinter is originally stationary and is triggered to cross the crosswalk when ego is 11 meters from the crosswalk stop line. 11 meters is chosen so that the pedestrian reaches the road when the vehicle is at the stop line.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Shkurti iteratively training on scenarios with the simulated time-varying forces to moving entities of pedestrians of Chao to safely navigate autonomous vehicles through unexpected scenarios (Chao, page 41). Shkurti and Chao are analogous arts because they both concern evaluating driving scenarios for autonomous vehicles.
Rhinehart teaches applying fractal cracking to surfaces within the scenarios. (Rhinehart [page 8, 4.2 Producing Unobserved Behaviors to Avoid Novel Obstacles]: “To further investigate our model’s flexibility to test-time objectives (question 3), we designed a pothole avoidance experiment. We simulated potholes (fractal cracking) in the environment by randomly inserting them in the cost map near waypoints.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Shkurti iteratively training on scenarios with the simulated potholes of Rhinehart to safely navigate autonomous vehicles through unexpected scenarios (Rhinehart, page 8, 4.2 Producing Unobserved Behaviors to Avoid Novel Obstacles). Shkurti and Rhinehart are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Dependent claim 14 is claim 7 in the form of a system and is rejected for the same reasons as claim 7 stated above. For the rejection of the limitations specifically pertaining to the system of claim 8, see the rejection of claim 8 above.
Dependent claim 21 is claim 14 in the form of a method and is rejected for the same reasons as claim 14 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Claims 10, 11, 12, 17, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Shkurti, Kehl, Feng and Danna in view of Hari et al. (US 20210389769 A1); hereinafter Hari
Claim 10 is rejected over Shkurti, Kehl, Feng, Danna and Hari with the incorporation of claim 8.
Shkurti does not appear to explicitly teach evaluate the AV model on scenarios in the plurality of scenarios; and input performance metrics indicating the performance of the AV model into the manifold data structure.
However, Hari teaches evaluate the AV model on scenarios in the plurality of scenarios; and input performance metrics indicating the performance of the AV model into the manifold data structure. (Hari [0064]: “The driving path computation 112 results may be input to a computer simulator 102. Simulation results are subject to scoring 120. The scenario may be scored based on possible driving policies and the effort each may expend to avoid unsafe conditions at various points in the simulated scenario time. The scoring 120 may provide an assessment of the performance of an AV. The scored scenarios are prioritized by a prioritizer 122 based on their difficulty and risk. Higher scoring scenarios may be applied to improve the utility of AV behavioral policies.”)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the autonomous vehicle system of Hari to improve AV training quality and time. “By efficiently providing a set of unsafe scenarios, automatically (using metrics, not subjective human intuition) characterized by difficulty of avoidance, the described techniques may improve AV training quality and time.” Shkurti and Hari are analogous art because they both concern evaluating scenarios for autonomous vehicles.
Claim 11 is rejected over Shkurti, Kehl, Feng, Danna and Hari with the incorporation of claim 8.
Regarding claim 11, Shkurti does not appear to explicitly teach wherein the AV training application further directs the processor to select a distribution of edge case scenarios from the [manifold] data structure based on the performance metrics for training the AV model in a second iteration of training.
However, Feng teaches wherein the AV training application further directs the processor to select a distribution of edge case scenarios from the [manifold] data structure based on the performance metrics for training the AV model in a second iteration of training. (Feng [page 5, B. Dissimilarity Function Estimation]: “be represented by the GP as
f
x
∼GP(m(x), k(x, x ‘)), where both x and x’ denote scenarios, m(x) denotes the mean function, and k(x, x’) denotes the covariance function.”; Note: This is the distance metric between two scenario points x and x’; [page 4, Algorithm 1]: Step 2.1: Obtain the estimation … Step 2.2: Update SM and library … Step 2.3: Decide next iteration of testing scenarios; [page 1]: “Underweight scenarios represent the critical scenarios that are ignored by the library, and overweight scenarios represent the uncritical scenarios that are included in the library. If we denote the scenario library generated by using the SM as “offline generated library”, and a customized library that includes all critical scenarios specifically designed for a CAV as “optimal library”, the differences between these two libraries include both underweight and overweight scenarios.”; Note: The underweight scenarios are where the CAV underperforms. The offline library is the unchanged version of a reference.)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the scenario library of Feng for efficient automated vehicle testing. “To compensate for the performance dissimilarities and leverage each test of the CAV, Bayesian optimization techniques are applied with classification-based Gaussian Process Regression and a newly designed acquisition function. Comparing with a pre-determined library, a CAV can be tested and evaluated in a more efficient manner with the customized library (Feng, Abstract).” Shkurti and Feng are analogous art because they both concern evaluation driving scenarios for autonomous vehicles.
Shkurti does not appear to explicitly teach manifold data structure
However, Kehl teaches [manifold] data structure (Kehl [0016]: “FIG. 6 illustrates a 3D manifold space configured to separately define object clusters with aggregated object poses, in which the object clusters are separated by a predetermined distance,”)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the manifold learning of Kehl to improve confidence associated with identifying objects in scenarios. “The object identification process performed by the object detector 430 to identify an object and estimate an object pose in conjunction with the manifold network 420 may significantly improve a confidence associated with identifying unknown objects and their corresponding poses. (Kehl, [0049]). Shkurti and Kehl are analogous art because they both concern analyzing scenarios for autonomous vehicles.
Claim 12 is rejected over Shkurti, Kehl, Feng, Danna and Hari with the incorporation of claim 8.
Regarding claim 12, Shkurti teaches wherein the AV model is a perceptual subsystem; and (Shkurti [0076]: “the ADA 105 may include several independent rules-based and/or learning-based functions and modules (e.g. systems 120, 130, 140). Accordingly, in some examples, training and evaluation of ADA 105 may be focused on selectively training one or more individual sub-system agents of the ADA 105 and specific scenarios 318 may be focused for training specific individual sub-system agents. For example, scenarios could be generated that are targeted for specifically training a Lidar point cloud analysis sub-system agent of the state estimation system 120 to detect object boundaries.”)
Shkurti does not appear to explicitly teach wherein a loss function used to train the AV model is modulated by an expectation of an adverse event within a given scenario.
However, Danna teaches wherein a loss function used to train the AV model is modulated by an expectation of an adverse event within a given scenario. (Danna [0061]: “FIG. 5 illustrates an example diagram 500 of training a model based on a triplet loss technique, according to an embodiment of the present technology. The triplet loss technique can utilize sets of an anchor representation, a positive representation, and a negative representation as training data. The three representations in a set can be, respectively, an anchor encoded image 504, a positive encoded image 502, and a negative encoded image 506. In this example diagram 500, the anchor encoded image 504 is the example 300 of FIG. 3.”)
It would have been obvious before the effective filing date to combine the iterative adversarial learning of Shkurti with the scenario encoding of Danna to improve the organization of different types of scenarios. “Thus, an improved approach that indexes or maintains scenario data of different types of scenarios that negates the need for the developers and searchers to keep up with the taxonomy structure is desired.” Shkurti and Danna are analogous art because they both concern evaluating scenarios on autonomous vehicles.
Dependent claim 17 is claim 10 in the form of a method and is rejected for the same reasons as claim 10 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Dependent claim 18 is claim 11 in the form of a method and is rejected for the same reasons as claim 11 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Dependent claim 19 is claim 12 in the form of a method and is rejected for the same reasons as claim 12 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Claim 22 is rejected under 35 U.S.C. 103 as being unpatentable over Shkurti, Kehl and Feng in view of Gupta et al. (Towards Safer Self-Driving Through Great PAIN (Physically Adversarial Intelligent Networks)); hereinafter Gupta
Claim 22 is rejected over Shkurti, Kehl, Feng and Gupta with the incorporation of claim 8.
Regarding claim 22, Shkurti does not appear to explicitly teach wherein the sets of edge case scenarios drawn from clusters of edge case scenarios are selected by an automated teacher which draws edge case scenarios according to a weighting
w
I
=
l
I
s
u
m
i
=
0
N
l
i
,
where i is a cluster of edge case scenarios unseen by the AV model during training, and l is the average loss over said cluster
However, Gupta teaches wherein the sets of edge case scenarios drawn from clusters of edge case scenarios are selected by an automated teacher which draws edge case scenarios according to a weighting
w
I
=
l
I
s
u
m
i
=
0
N
l
i
, where i is a cluster of edge case scenarios unseen by the AV model during training, and l is the average loss over said cluster (Gupta [page 11-12]: “The protagonist’s objective is to drive safely from a start to a goal location, while the adversary seeks to maximize the damage to the protagonist. This helps generate real-world, noisy data where the protagonist learns to anticipate and avoid impending direct collisions, while the adversary learns to cause increasingly-unpredictable collisions. The resultant data are better-representative of real-world scenarios than that provided by pre-programmed or randomized simulation alone.”; page 2, paragraph 3; and “PER [26] is an algorithm for sampling a batch of experiences from a memory
buffer to train a network. It improves the policy learned by DQN algorithms by increasing the replay probability of experiences that have a high impact on the learning process. These experiences may be rare but informative. The prediction error of the Q-learning algorithm is used to assign a priority value pi for each experience stored in a memory buffer, which generates a probability (23)”; Note: See page 12 of Gupta to see formula (23) used for normalization and the adversary acts like an automated teacher.)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the prioritized experience replay of Gupta to effectively deal with rare instances (Gupta, page 11, 5.2 Prioritized Experience Replay (PER)). Hari and Gupta are analogous art because they both concern simulating scenarios for autonomous vehicles.
Claims 23 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Shkurti, Kehl, Danna and Feng in view of Gupta.
Dependent claim 23 is claim 22 in the form of a system and is rejected for the same reasons as claim 22 stated above. For the rejection of the limitations specifically pertaining to the system of claim 8, see the rejection of claim 8 above.
Dependent claim 24 is claim 22 in the form of a method and is rejected for the same reasons as claim 22 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID H TRAN whose telephone number is (703)756-1525. The examiner can normally be reached M-F 9:30 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H TRAN/Examiner, Art Unit 2147
/VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights