Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments on pages 9-11 regarding the rejection under 35 U.S.C. 103 with respect to claims 1, 3, 6-8, 10-15 and 17-24 have been fully considered but are not persuasive.
Beginning on page 9, Applicant submits that Danna’s triplet is not reflective of a “risk manifold” because there are only three scenarios. However, the amount of scenarios is sufficient as the triplet’s three scenarios are still multiple different scenarios used.
Applicant’s arguments on pages 10-11 are moot because they do not apply to the new combination of references used to teach the amended claim.
Claim Objections
Claims 23 and 24 are objected to because of the following informalities:
In claim 23, line 1, “22. (New) The system for training AVs of claim 8,” should read “23. (New) The system for training AVs of claim 8,”.
In claim 24, line 1, “22. (New) The method for training AV models of claim 15,” should read “24. (New) The method for training AV models of claim 15,”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 6, 8, 10, 11, 15, 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Hari et al. (US20210389769A1, Adversarial scenarios for safety testing of autonomous vehicles); hereinafter Hari in view of Dana et al. (US20210403036A1, Systems and methods for encoding and searching scenario information); hereinafter Dana
Claim 1 is rejected over Hari and Dana.
Regarding claim 1, Hari teaches an autonomous vehicle (AV), comprising:
a vehicle (“elements of the autonomous vehicle simulation environment 100, and actions of the computer-implemented process 200, may be implemented or executed by such devices. Results may be applied to update the behavior logic (logic embodying behavioral policies) of autonomous vehicles.”; [0079]);
a processor; and (“The algorithms and techniques disclosed herein may be executed by computing devices utilizing one or more graphic processing unit (GPU) and/or general purpose data processor (e.g., a ‘central processing unit or CPU).”; [0076])
Hari does not appear to explicitly teach a memory, where the memory contains an AV model capable of driving the vehicle without human input;
where the AV model is trained on a plurality of edge case scenarios encoded in a risk manifold, where the risk manifold encodes a distance metric between edqe case scenarios in the risk manifold, and
the AV model is iteratively trained on a set of edqe case scenarios drawn from clusters of edge case scenarios in the risk manifold,
where a distribution of edge case scenarios in the set is altered based on distance metrics at each iterative step to expand subspaces in which the AV model underperforms.
However, Dana teaches a memory, where the memory contains an AV model capable of driving the vehicle without human input; (“As one example, the vehicle may have a computing system (e.g., one or more central processing units, graphical processing units, memory, storage, etc.) for controlling various operations of the vehicle, such as driving and navigating. “; [0002] and “Such vehicles, whether autonomously, semi-autonomously, or manually driven, may be capable of sensing their environment and navigating with little or no human input as appropriate.” [0040])
where the AV model is trained on a plurality of edge case scenarios encoded in a risk manifold, where the risk manifold encodes a distance metric between edqe case scenarios in the risk manifold, and (“In an embodiment, a machine learning model can be trained with an anchor representation comprising a first encoded image representing a scenario, a positive representation comprising a second encoded image representing a scenario that has a threshold level of similarity (distance) to the anchor representation, and a negative representation comprising a third encoded image representation of a scenario that does not have the threshold level of similarity to the anchor representation.”; [0012] and “Assume, for example, that three other scenarios 170, 180, 190 are known and maintained in a data store (data structure); [0046]; Note: The embedding is a vector representation of the scenario and behaves like a manifold.)
the AV model is iteratively trained on a set of edqe case scenarios drawn from clusters of edge case scenarios in the risk manifold, (“The triplet loss technique can utilize sets of an anchor representation, a positive representation, and a negative representation as training data. The three representations in a set can be, respectively, an anchor encoded image 504, a positive encoded image 502, and a negative encoded image 506. In this example diagram 500, the anchor encoded image 504 is the example 300 of FIG. 3. The triplet loss technique can generate respective embeddings 508 for each of the encoded images 502, 504, 506.”; [0061]; Note: Triplet loss is an iterative process applied across scenarios.)
where a distribution of edge case scenarios in the set is altered based on distance metrics at each iterative step to expand subspaces in which the AV model underperforms. (“The model can determine respective embeddings for the encoded images. For example, the model can arrange the encoded images within the low-dimensional vector space. Each of the encoded images can be associated with an embedding that gets adjusted, as training progresses, to better reflect its location in the low-dimensional vector space. The embeddings can be used to determine a measure of similarity based on a distance metric between an embedding and another embedding.”; [0053]; and “the scenarios can be used to further train (or refine) the model, run computer-based simulations of an autonomous navigation system, and evaluate various performance metrics of the autonomous navigation system”; [0046])
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the autonomous vehicle navigating without human input of Dana for autonomous vehicles to safely recognize and navigate through traffic hazards (Dana, [0040]). Hari and Dana are analogous arts because they both concern training autonomous vehicles with simulated traffic scenarios.
Claim 3 is rejected over Hari and Dana with the incorporation of claim 1.
Regarding claim 3, Hari does not teach wherein the distance is a scalar valued dimensional reduction of data associated with edge case scenarios.
However, Dana teaches wherein the distance is a scalar valued dimensional reduction of data associated with edge case scenarios. (“In an embodiment, a machine learning model can be trained with an anchor representation comprising a first encoded image representing a scenario, a positive representation comprising a second encoded image representing a scenario that has a threshold level of similarity (distance) to the anchor representation, and a negative representation comprising a third encoded image representation of a scenario that does not have the threshold level of similarity to the anchor representation.”; [0012]; Note: An encoded scenario is a compressed lower dimensional value)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the encoded scenarios of Dana for autonomous vehicles to safely recognize and navigate through traffic hazards (Dana, [0040]). Hari and Dana are analogous arts because they both concern training autonomous vehicles with simulated traffic scenarios.
Claim 6 is rejected over Hari and Dana with the incorporation of claim 1.
Regarding claim 6, Hari teaches wherein the AV model is a perceptual subsystem. (“In any example, when a determination is made, based on information from the path perceiver, the wait perceiver, the map perceiver, the obstacle perceiver, and/or another component of the perception component(s) 1604 (perceptual subsystem), that prevents the autonomous vehicle 1502 from proceeding through a certain situation, scenario, and/or environment, at least partial control may be transferred to the remote control system 1506.”; [0199]).
Claim 8 is rejected over Hari and Dana.
Regarding claim 8, Hari teaches a system for training autonomous vehicles (AVs), comprising:
a processor; and (“The algorithms and techniques disclosed herein may be executed by computing devices utilizing one or more graphic processing unit (GPU) and/or general purpose data processor (e.g., a ‘central processing unit or CPU).”; [0076])
generate a list of edge case scenarios within the plurality of scenarios; (“Once the unsafe scenarios are created (edge case scenarios), they may be characterized or categorized based on how difficult it is to avoid simulated accidents within the scenario,”; [0059])
identify hazard frames within the edge case scenarios; (“Once the unsafe scenarios are created (edge case scenarios), they may be characterized or categorized (identifying hazard frames) based on how difficult it is to avoid simulated accidents within the scenario,”; [0059])
train the AV model using the one or more records. (“Once the unsafe scenarios are created (edge case scenarios), they may be characterized or categorized based on how difficult it is to avoid simulated accidents within the scenario, and thus how useful the scenarios may be in training an AV for accident avoidance. It may not be helpful to train the AV using only safe scenarios, or using unsafe scenarios in which accidents are unavoidable or very difficult to avoid. By efficiently providing a set of unsafe scenarios, automatically (using metrics, not subjective human intuition) characterized by difficulty of avoidance, the described techniques may improve AV training quality and time.”; [0059])
Hari does not teach
a memory, containing an AV training application that directs the processor to:
obtain a manifold data structure storing a plurality of scenarios that an AV can encounter,
and distance metrics indicating the distance between each scenario;
encode the hazard frames into one or more records interpretable by an AV model; and
iteratively train the AV model on the hazard frames on a set of edge case scenarios drawn from clusters of edge case scenarios in the manifold data structure,
where a distribution of edge case scenarios the set is altered based on the distance metrics at each iterative step to expand subspaces in which the AV model underperforms; and
a vehicle configured to obtain the trained AV model, where the AV model drives the vehicle without human input.
However, Dana teaches a memory, containing an AV training application that directs the processor to: (“As one example, the vehicle may have a computing system (e.g., one or more central processing units, graphical processing units, memory, storage, etc.) for controlling various operations of the vehicle, such as driving and navigating. “; [0002] and “Such vehicles, whether autonomously, semi-autonomously, or manually driven, may be capable of sensing their environment and navigating with little or no human input as appropriate.” [0040])
obtain a manifold data structure storing a plurality of scenarios that an AV can encounter, and distance metrics indicating the distance between each scenario; (“In an embodiment, a machine learning model can be trained with an anchor representation comprising a first encoded image representing a scenario, a positive representation comprising a second encoded image representing a scenario that has a threshold level of similarity (distance) to the anchor representation, and a negative representation comprising a third encoded image representation of a scenario that does not have the threshold level of similarity to the anchor representation.”; [0012] and “Assume, for example, that three other scenarios 170, 180, 190 are known and maintained in a data store (data structure); [0046])
encode the hazard frames into one or more records interpretable by an AV model; and (“In an embodiment, a machine learning model can be trained with an anchor representation comprising a first encoded image representing a scenario, a positive representation comprising a second encoded image representing a scenario that has a threshold level of similarity to the anchor representation,”; [0012])
iteratively train the AV model on the hazard frames on a set of edge case scenarios drawn from clusters of edge case scenarios in the manifold data structure, (“The triplet loss technique can utilize sets of an anchor representation, a positive representation, and a negative representation as training data. The three representations in a set can be, respectively, an anchor encoded image 504, a positive encoded image 502, and a negative encoded image 506. In this example diagram 500, the anchor encoded image 504 is the example 300 of FIG. 3. The triplet loss technique can generate respective embeddings 508 for each of the encoded images 502, 504, 506.”; [0061]; Note: Triplet loss is an iterative process applied across scenarios.)
where a distribution of edge case scenarios the set is altered based on the distance metrics at each iterative step to expand subspaces in which the AV model underperforms; and (“The model can determine respective embeddings for the encoded images. For example, the model can arrange the encoded images within the low-dimensional vector space. Each of the encoded images can be associated with an embedding that gets adjusted, as training progresses, to better reflect its location in the low-dimensional vector space. The embeddings can be used to determine a measure of similarity based on a distance metric between an embedding and another embedding.”; [0053]; and “the scenarios can be used to further train (or refine) the model, run computer-based simulations of an autonomous navigation system, and evaluate various performance metrics of the autonomous navigation system”; [0046])
a vehicle configured to obtain the trained AV model, where the AV model drives the vehicle without human input. (“As one example, the vehicle may have a computing system (e.g., one or more central processing units, graphical processing units, memory, storage, etc.) for controlling various operations of the vehicle, such as driving and navigating. “; [0002] and “Such vehicles, whether autonomously, semi-autonomously, or manually driven, may be capable of sensing their environment and navigating with little or no human input as appropriate.” [0040])
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the encoded scenarios of Dana for autonomous vehicles to safely recognize and navigate through traffic hazards (Dana, [0040]). Hari and Dana are analogous arts because they both concern training autonomous vehicles with simulated traffic scenarios.
Claim 10 is rejected over Hari and Dana with the incorporation of claim 8.
Regarding claim 10, Hari teaches evaluate the AV model on scenarios in the plurality of scenarios; and input performance metrics indicating the performance of the AV model into the data structure. (“The driving path computation 112 results may be input to a computer simulator 102. Simulation results are subject to scoring 120. The scenario may be scored based on possible driving policies and the effort each may expend to avoid unsafe conditions at various points in the simulated scenario time. The scoring 120 may provide an assessment of the performance of an AV. The scored scenarios are prioritized by a prioritizer (data structure) 122 based on their difficulty and risk. Higher scoring scenarios may be applied to improve the utility of AV behavioral policies.”; [0064])
Claim 11 is rejected over Hari and Dana with the incorporation of claim 8.
Regarding claim 11, Hari does not teach select a distribution of edge case scenarios from the data structure based on the performance metrics for training the AV model in a second iteration of training.
However, Dana teaches select a distribution of edge case scenarios from the data structure based on the performance metrics for training the AV model in a second iteration of training. (“Simulation results are subject to scoring 120. The scenario may be scored based on possible driving policies and the effort each may expend to avoid unsafe conditions at various points in the simulated scenario time. The scoring 120 may provide an assessment of the performance of an AV. The scored scenarios are prioritized by a prioritizer 122 based on their difficulty and risk. Higher scoring scenarios may be applied (retraining) to improve the utility of AV behavioral policies.”; [0064] and “For example, the scenarios can be used to further train (or refine) the model, run computer-based simulations of an autonomous navigation system, and evaluate various performance metrics of the autonomous navigation system.”; [0046])
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the autonomous vehicle performance assessment of Dana for retraining of autonomous vehicles to safely recognize and navigate through traffic hazards (Dana, [0040]). Hari and Dana are analogous arts because they both concern training autonomous vehicles with simulated traffic scenarios.
Claim 15 is claim 8 in the form of a method and is rejected for the same reasons as claim 8 stated above.
Dependent claim 17 is claim 10 in the form of a method and is rejected for the same reasons as claim 10 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Dependent claim 18 is claim 11 in the form of a method and is rejected for the same reasons as claim 11 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Claims 7, 14 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Hari and Dana in further view of Wu et al. (A Pioneering Scalable Self-driving Car Simulation Platform); hereinafter Wu in view of Hospach et al. (Simulation of Falling Rain for Robustness Testing of Video-Based Surround Sensing Systems); hereinafter Hospach in view of Chao et al. (Autonomous Driving: Mapping and Behavior Planning for Crosswalks) Video-Based Surround Sensing Systems); hereinafter Chao and in further view of Rhinehart et al. (DEEP IMITATIVE MODELS FOR FLEXIBLE INFERENCE, PLANNING, AND CONTROL); hereinafter Rhinehart
Claim 7 is rejected over Hari, Dana, Wu, Hospach, Chao and Rhinehart with the incorporation of claim 1.
Regarding claim 7, Hari does not teach wherein a subset of the plurality of edge case scenarios are artificially generated using a method selected from the group consisting of: applying a bandpass filter to sensor data; generating 2-D semi-opaque, semi-reflective, semi-occluding polygons into the scenario data at a position between a sensor source and an event; applying multiscale Gabor patterns to events within simulated scenarios; applying time-varying forces to moving entities within the scenarios; and applying fractal cracking to surfaces within the scenarios.
However, Wu teaches wherein a subset of the plurality of edge case scenarios are artificially generated using a method selected from the group consisting of: applying a bandpass filter to sensor data; (“The filtering process to distinguish the road pixels from the background plays a quite substantial part. Reference [13] just selects the scope of yellow and white color from the images by using LUV [14] and LaB image formats, thus separating the pixels of roads from those of the background. Alternatively, reference [12] uses Gabor filters, which is a best-known quadrature filters. Gabor filter is characterized by Gaussian-formed band pass filters. It is a suitable choice to accomplish functions demanding simultaneous measurement in both space and frequency domains [1]”; page 150, A. Lane Detection)
applying multiscale Gabor patterns to events within simulated scenarios; (“The filtering process to distinguish the road pixels from the background plays a quite substantial part. Reference [13] just selects the scope of yellow and white color from the images by using LUV [14] and LaB image formats, thus separating the pixels of roads from those of the background. Alternatively, reference [12] uses Gabor filters, which is a best-known quadrature filters. Gabor filter is characterized by Gaussian-formed band pass filters. It is a suitable choice to accomplish functions demanding simultaneous measurement in both space and frequency domains [1]”; page 150, A. Lane Detection)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the Gabor filters of Wu to effectively aid autonomous vehicles in object detection in traffic (Wu, page 150, A. Lane Detection). Hari and Wu are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Hospach teaches generating 2-D semi-opaque, semi-reflective, semi-occluding polygons into the scenario data at a position between a sensor source and an event; (“First, the background of the scene is blurred as if it was recorded with the settings of the simulation camera, according to the theory of circle of confusion above. Here, it is very important, that the depth-of-field of the source material is large enough, since already present blur cannot be undone. Second, the rain drops themselves are blurred depending on their depth during the simulation. In this manner, the OpenGL pinhole camera model for projective imaging can be extended as if the camera model had a lens with aperture and focus settings”; page 234 and “This value denotes the transparency value of a rain drop due to its motion blur. The base color of a drop, which in reality is scene and illumination dependent, is then weighted with α. Without knowing details on the scene illumination, the base color of a rain streak has been empirically set to a semi-transparent white with opacity factor 0.5. This has shown to produce good results. If more detail of scene illumination is known, the color value can be specified more accurate, though.”; page 235)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the simulated rain drop shapes of Hospach to effectively test vision-based surround sensing systems in autonomous vehicles (Hospach, page 233, Abstract). Hari and Hospach are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Chao teaches applying time-varying forces to moving entities within the scenarios; and (“The 100 meter dash scenario simulates a pedestrian sprinting across the crosswalk at 44 kph (time-varying forces). The autonomous vehicle travels at the speed limit of 30 kph toward the crosswalk. The sprinter is originally stationary and is triggered to cross the crosswalk when ego is 11 meters from the crosswalk stop line. 11 meters is chosen so that the pedestrian reaches the road when the vehicle is at the stop line.”; page 40)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the simulated time-varying forces to moving entities of pedestrians of Chao to safely navigate autonomous vehicles through unexpected scenarios (Chao, page 41). Hari and Chao are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Rhinehart teaches applying fractal cracking to surfaces within the scenarios. (“To further investigate our model’s flexibility to test-time objectives (question 3), we designed a pothole avoidance experiment. We simulated potholes (fractal cracking) in the environment by randomly inserting them in the cost map near waypoints.”; page 8, 4.2 Producing Unobserved Behaviors to Avoid Novel Obstacles)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the simulated potholes of Rhinehart to safely navigate autonomous vehicles through unexpected scenarios (Rhinehart, page 8, 4.2 Producing Unobserved Behaviors to Avoid Novel Obstacles). Hari and Rhinehart are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Claim 14 is rejected over Hari, Dana, Wu, Hospach, Chao and Rhinehart with the incorporation of claim 8.
Regarding claim 14, Hari does not teach wherein a subset of the plurality of edge case scenarios are artificially generated using a method selected from the group consisting of: applying a bandpass filter to sensor data; generating 2-D semi-opaque, semi-reflective, semi-occluding polygons into the scenario data at a position between a sensor source and an event; applying multiscale Gabor patterns to events within simulated scenarios; applying time-varying forces to moving entities within the scenarios; and applying fractal cracking to surfaces within the scenarios.
However, Wu teaches wherein a subset of the plurality of edge case scenarios are artificially generated using a method selected from the group consisting of: applying a bandpass filter to sensor data; (“The filtering process to distinguish the road pixels from the background plays a quite substantial part. Reference [13] just selects the scope of yellow and white color from the images by using LUV [14] and LaB image formats, thus separating the pixels of roads from those of the background. Alternatively, reference [12] uses Gabor filters, which is a best-known quadrature filters. Gabor filter is characterized by Gaussian-formed band pass filters. It is a suitable choice to accomplish functions demanding simultaneous measurement in both space and frequency domains [1]”; page 150, A. Lane Detection)
applying multiscale Gabor patterns to events within simulated scenarios; (“The filtering process to distinguish the road pixels from the background plays a quite substantial part. Reference [13] just selects the scope of yellow and white color from the images by using LUV [14] and LaB image formats, thus separating the pixels of roads from those of the background. Alternatively, reference [12] uses Gabor filters, which is a best-known quadrature filters. Gabor filter is characterized by Gaussian-formed band pass filters. It is a suitable choice to accomplish functions demanding simultaneous measurement in both space and frequency domains [1]”; page 150, A. Lane Detection)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the Gabor filters of Wu to effectively aid autonomous vehicles in object detection in traffic (Wu, page 150, A. Lane Detection). Hari and Wu are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Hospach teaches generating 2-D semi-opaque, semi-reflective, semi-occluding polygons into the scenario data at a position between a sensor source and an event; (“First, the background of the scene is blurred as if it was recorded with the settings of the simulation camera, according to the theory of circle of confusion above. Here, it is very important, that the depth-of-field of the source material is large enough, since already present blur cannot be undone. Second, the rain drops themselves are blurred depending on their depth during the simulation. In this manner, the OpenGL pinhole camera model for projective imaging can be extended as if the camera model had a lens with aperture and focus settings”; page 234 and “This value denotes the transparency value of a rain drop due to its motion blur. The base color of a drop, which in reality is scene and illumination dependent, is then weighted with α. Without knowing details on the scene illumination, the base color of a rain streak has been empirically set to a semi-transparent white with opacity factor 0.5. This has shown to produce good results. If more detail of scene illumination is known, the color value can be specified more accurate, though.”; page 235)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the simulated rain drop shapes of Hospach to effectively test vision-based surround sensing systems in autonomous vehicles (Hospach, page 233, Abstract). Hari and Hospach are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Chao teaches applying time-varying forces to moving entities within the scenarios; and (“The 100 meter dash scenario simulates a pedestrian sprinting across the crosswalk at 44 kph (time-varying forces). The autonomous vehicle travels at the speed limit of 30 kph toward the crosswalk. The sprinter is originally stationary and is triggered to cross the crosswalk when ego is 11 meters from the crosswalk stop line. 11 meters is chosen so that the pedestrian reaches the road when the vehicle is at the stop line.”; page 40)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the simulated time-varying forces to moving entities of pedestrians of Chao to safely navigate autonomous vehicles through unexpected scenarios (Chao, page 41). Hari and Chao are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Rhinehart teaches applying fractal cracking to surfaces within the scenarios. (“To further investigate our model’s flexibility to test-time objectives (question 3), we designed a pothole avoidance experiment. We simulated potholes (fractal cracking) in the environment by randomly inserting them in the cost map near waypoints.”; page 8, 4.2 Producing Unobserved Behaviors to Avoid Novel Obstacles)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the simulated potholes of Rhinehart to safely navigate autonomous vehicles through unexpected scenarios (Rhinehart, page 8, 4.2 Producing Unobserved Behaviors to Avoid Novel Obstacles). Hari and Rhinehart are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Dependent claim 21 is claim 14 in the form of a method and is rejected for the same reasons as claim 14 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Claims 12, 13, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Hari and Dana in further view of Ramamoorthy et al. (US 20210339772A1, Driving Scenarios for Autonomous Vehicles); hereinafter Ramamoorthy
Claim 12 is rejected over Hari, Dana and Ramamoorthy with the incorporation of claim 8.
Regarding claim 12, Hari teaches wherein the AV model is a perceptual subsystem; and (“In any example, when a determination is made, based on information from the path perceiver, the wait perceiver, the map perceiver, the obstacle perceiver, and/or another component of the perception component(s) 1604 (perceptual subsystem), that prevents the autonomous vehicle 1502 from proceeding through a certain situation, scenario, and/or environment, at least partial control may be transferred to the remote control system 1506.”; [0199])
Hari does not teach wherein a loss function used to train the AV model is modulated by an expectation of an adverse event within a given scenario.
However, Ramamoorthy teaches wherein a loss function used to train the AV model is modulated by an expectation of an adverse event within a given scenario. (“The anomalous driving behaviour (adverse event) observed can also be used to train the scenario generator to construct new, more life-like, scenarios for training, such that the scenarios generated are artificial and do not use the collected data directly, but do contain actors performing anomalous driving behaviours similar to those observed. This may, for example, be through the use of generative adversarial networks (GANs).”; [0115] and “A GAN comprises two networks, a first of which (the generator) generates driving scenarios and the second of which (the classifier) classifies the real and the generated driving scenarios in relation to the set of training data as “real”, i.e. belonging to the training set, or “artificial” (generated), i.e. not belonging to the training set. The adversarial aspect is that the generator is incentivised (via a suitably-defined loss function) to try to “beat” the classifier by generating driving scenarios that the classifier classifies, incorrectly, as “real”, whereas the classifier is incentivised to try to beat the generator by classifying the driving scenarios accurately as real or artificial. As the networks are trained, the generator is pushed to get better and better at generating realistic driving scenarios capable of fooling the increasing accurate classifier, such that, by the end of the process, the generator is capable of generating highly driving scenarios, i.e. which are hard to distinguish from the training examples. The networks are incentivised via suitably defined loss functions applied to their respective outputs.”; [0116])
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the loss function of Ramamoorthy to safely navigate autonomous vehicles through unexpected scenarios (Ramamoorthy, [0116]). Hari and Ramamoorthy are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Claim 13 is rejected over Hari, Dana and Ramamoorthy with the incorporation of claim 8.
Regarding claim 13, Hari teaches wherein the AV model is a decision-making module; and (“The machine learning model(s) 2204 may include any type of machine learning model(s), such as machine learning models using linear regression, logistic regression, decision trees,”; [0272])
Hari does not teach wherein a loss function used to train the AV model is modulated by the rate of adverse events experienced by an agent on a given set of scenarios.
However, Ramamoorthy teaches wherein a loss function used to train the AV model is modulated by the rate of adverse events experienced by an agent on a given set of scenarios. (“wherein the reinforcement learning component is configured to execute a policy search algorithm to select action policies for attempting in the driving scenario simulations, with the objective of maximizing a cumulative reward assigned to the series of ego vehicle actions (decision-making), and thereby determine an optimal action policy for performing the predefined manoeuvre in an encountered driving context, the cumulative reward is defined so as to penalize (i) actions which are determined to be unsafe and (ii) actions which are determined not to progress the predefined manoeuvre.”; [0071] and “A GAN comprises two networks, a first of which (the generator) generates driving scenarios and the second of which (the classifier) classifies the real and the generated driving scenarios in relation to the set of training data as “real”, i.e. belonging to the training set, or “artificial” (generated), i.e. not belonging to the training set. The adversarial aspect is that the generator is incentivised (via a suitably-defined loss function) to try to “beat” the classifier by generating driving scenarios that the classifier classifies, incorrectly, as “real”, whereas the classifier is incentivised to try to beat the generator by classifying the driving scenarios accurately as real or artificial.”; [0116])
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the reinforcement learning of Ramamoorthy to safely navigate autonomous vehicles through unexpected scenarios (Ramamoorthy, [0116]). Hari and Ramamoorthy are analogous arts because they both concern autonomous vehicles with simulated traffic scenarios.
Dependent claim 19 is claim 12 in the form of a method and is rejected for the same reasons as claim 12 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Dependent claim 20 is claim 13 in the form of a method and is rejected for the same reasons as claim 13 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Claims 22, 23 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Hari and Dana in further view of Gupta et al. (Towards Safer Self-Driving Through Great PAIN (Physically Adversarial Intelligent Networks); hereinafter Gupta
Claim 22 is rejected over Hari, Dana and Gupta with the incorporation of claim 1.
Regarding claim 22, Hari does not teach wherein the sets of edge case scenarios drawn from clusters of edge case scenarios are selected by an automated teacher which draws edge case scenarios according to a weighting wi = li / sumNi=0 li where i is a cluster of edge case scenarios unseen by the AV model during training, and l is the average loss over said cluster.
However, Gupta teaches wherein the sets of edge case scenarios drawn from clusters of edge case scenarios are selected by an automated teacher which draws edge case scenarios according to a weighting wi = li / sumNi=0 li where i is a cluster of edge case scenarios unseen by the AV model during training, and l is the average loss over said cluster. (“The protagonist’s objective is to drive safely from a start to a goal location, while the adversary seeks to maximize the damage to the protagonist. This helps generate real-world, noisy data where the protagonist learns to anticipate and avoid impending direct collisions, while the adversary learns to cause increasingly-unpredictable collisions. The resultant data are better representative of real-world scenarios than that provided by pre-programmed or randomized simulation alone.”; page 2, paragraph 3; and “PER [26] is an algorithm for sampling a batch of experiences from a memory buffer to train a network. It improves the policy learned by DQN algorithms by increasing the replay probability of experiences that have a high impact on the learning process. These experiences may be rare but informative. The prediction error of the Q-learning algorithm is used to assign a priority value pi for each experience stored in a memory buffer, which generates a probability (23)”; page 11-12; Note: See page 12 of Gupta to see formula (23) used for normalization and the adversary acts like an automated teacher.)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention to have performed this in combination of Hari training an autonomous vehicle model on unsafe scenarios with the prioritized experience replay of Gupta to effectively deal with rare instances (Gupta, page 11, 5.2 Prioritized Experience Replay (PER)). Hari and Gupta are analogous art because they both concern simulating scenarios for autonomous vehicles.
Dependent claim 23 is claim 22 in the form of a system and is rejected for the same reasons as claim 22 stated above. For the rejection of the limitations specifically pertaining to the system of claim 8, see the rejection of claim 8 above.
Dependent claim 24 is claim 22 in the form of a method and is rejected for the same reasons as claim 22 stated above. For the rejection of the limitations specifically pertaining to the method of claim 15, see the rejection of claim 15 above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID H TRAN whose telephone number is (703)756-1525. The examiner can normally be reached M-F 9:30 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached on (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID H TRAN/ Examiner, Art Unit 2147
/HASSAN MRABI/Primary Examiner, Art Unit 2147