DETAILED ACTION
This is the first Office action drafted on the merits of the subject application. Claims 1-20 are pending. Claims 1-16, and 18-20 are rejected as cited below. Claim 17 is objected to as shown in the Allowable Subject Matter section. This application is a continuation of application no. 17/528,549.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed. The following title is suggested: “GENERATING SIMULATED TRAJECTORIES FOR AN AUTONOMOUS VEHICLE USING MODIFIED SENSOR DATA.”
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-4, 9-16, 18 and 20 are rejected under 35 U.S.C. 102(a)2 as being anticipated by Anthony (US Pub. 2020/0180647 A1; hereafter Anthony).
Anthony was cited in the IDS filed 12/20/2024.
Regarding claim 1, Anthony teaches:
A computer-implemented method for generating testing data for an autonomous vehicle, the method comprising:
obtaining scenario data descriptive of a traffic scenario (At least par [0047] “The sensors of an autonomous vehicle capture sensor data 160 representing a scene describing the traffic surrounding the autonomous vehicle.”), wherein the scenario data comprises a subject vehicle (vehicle 102), initial trajectories for one or more actors in an environment of the subject vehicle (At least par [0073] “The vehicle computing system 120 determines 704 motion parameters for the traffic entity, for example, speed and direction of movement of the traffic entity.”), and initial sensor data captured by the subject vehicle (At least par [0072] “The vehicle computing system 120 receives 700 sensor data from sensors of the autonomous vehicle.”);
generating a modified trajectory for at least a first actor of the one or more actors in the environment based on an initial trajectory for the first actor in the one or more actors (At least par [0098] “the GAN based simulation module 320 iteratively performs steps of perturbing 1010 the feature vector to modify one or more parameters for the feature vector…” Also, see par [0051] “the feature vector may specify that the traffic entity is a pedestrian, the location of the pedestrian, the orientation indicating the direction that the pedestrian is walking in, the speed with which the pedestrian is moving, and so on.”);
generating modified sensor data based on the modified trajectory by replacing at least some portions of the initial sensor data with simulated sensor data (at least par [0098] “providing 1002 the perturbed feature vector to the rendering engine to update the simulation data, applying 1004 the prediction engine 114 to the updated simulation data, and comparing 1006 the output generated by the prediction engine 114 to the desired output.”); and
providing the modified sensor data as input to an autonomous vehicle control system configured to process the simulated sensor data to generate an updated trajectory for the subject vehicle in the environment (At least par [0099] “The GAN based simulation module 320 may provide the generated output to a motion planner 340 for the autonomous vehicle for testing or development of the motion planner in a simulated environment…the motion planner 340 uses the simulated environment representation for generating a motion plan for a simulated autonomous vehicle or an actual autonomous vehicle being used in a test environment.”).
Regarding claim 2, Anthony teaches:
The computer-implemented method of claim 1, comprising:
generating a modified trajectory for the first actor of the one or more actors in the environment based on the initial trajectory for the first actor of the one or more actors in the environment (At least par [0098] “the GAN based simulation module 320 iteratively performs steps of perturbing 1010 the feature vector to modify one or more parameters for the feature vector…” Also, see par [0051] “the feature vector may specify that the traffic entity is a pedestrian, the location of the pedestrian, the orientation indicating the direction that the pedestrian is walking in, the speed with which the pedestrian is moving, and so on.”) and one or more perturbation values that are selected from a defined perturbation search space (At least par [0098] “the GAN based simulation module 320 perturbs the feature vector by accessing and sampling from a parameter space of the feature vector to obtain a stochastically generated gradient of the GAN based model performance.”).
Regarding claim 3, Anthony teaches:
The computer-implemented method of claim 2, comprising:
creating a set of physically feasible trajectories for the first actor based on the initial trajectory for the first actor (At least par [0096] “the GAN based simulation module 320 generates 1000 a feature vector. The feature vector represents characteristics for a set of traffic entities for the scenario. For example, the feature vector describes characteristics for the pedestrian wishing to cross the street, such as posture, orientation, walking speed, motions, expressions, and the like. The GAN based simulation module 320 provides 1002 the feature vector to the rendering engine to generate simulation data, such as image or video data, that includes the set of traffic entities with characteristics described by the feature vector.”); and
generating an initial perturbed trajectory for the first actor based on the one or more perturbation values (At least par [0098] “the GAN based simulation module 320 perturbs the feature vector by accessing and sampling from a parameter space of the feature vector to obtain a stochastically generated gradient of the GAN based model performance.”).
Regarding claim 4, Anthony teaches:
The computer-implemented method of claim 3, comprising:
projecting the initial perturbed trajectory onto the set of physically feasible trajectories to generate the modified trajectory (At least par [0098] “the GAN based simulation module 320 iteratively performs steps of perturbing 1010 the feature vector to modify one or more parameters for the feature vector, providing 1002 the perturbed feature vector to the rendering engine to update the simulation data, applying 1004 the prediction engine 114 to the updated simulation data, and comparing 1006 the output generated by the prediction engine 114 to the desired output.”).
Regarding claim 9, Anthony teaches:
The computer-implemented method of claim 1, comprising:
generating a perturbation search space (At least par [0098] “the GAN based simulation module 320 perturbs the feature vector by accessing and sampling from a parameter space of the feature vector …”); and
selecting one or more perturbation values from the perturbation search space by using a black-box optimization technique (At least [0028] “the system determines a derivative of the hidden context attribute value being predicted by machine learning based model using the generated simulation data. The derivative value represents a gradient of the feature vector. The system uses gradient descent techniques to determine the direction in which to perturb the feature vector so that the corresponding hidden context attributes change towards the desired values.”), wherein a modified trajectory for the first actor is generated based on the selection of the one or more perturbation values (At least par [0098] “the GAN based simulation module 320 perturbs the feature vector by accessing and sampling from a parameter space of the feature vector to obtain a stochastically generated gradient of the GAN based model performance.”).
Regarding claim 10, Anthony teaches The computer-implemented method of claim 9, comprising selecting the one or more perturbation values from the perturbation search space based on a reinforcement learning agent (At least par [0079] “The symbolic simulation is used to test the behavior of a motion planner including but not limited to an RRT (rapidly exploring random tree), a POMDP (partially observable Markov decision process …”. A Markov process is another form of a reinforcement learning agent.).
Regarding claim 11, Anthony teaches The computer-implemented method of claim 9, wherein the perturbation search space comprises initial state values and a change in curvature and acceleration values over a plurality of timesteps (At least par [0082] (“The symbolic simulation module 310 generates 810 a symbolic representation of each entity in the environment surrounding of the autonomous vehicle including non-stationary physical objects (e.g., pedestrians, cyclists, motorists, and so on)…The information describing the motion of a traffic entity may be specified using one or more vectors indicating a direction of movement, velocity, acceleration, and so on.” And par [0085] “By associating a traffic entity across a sequence of consecutive video frames, the system 110 tracks the traffic entity over a time interval of the video.”).
Regarding claim 12, Anthony teaches The computer-implemented method of claim 1, comprising selecting a closest reachable actor as the first actor (At least par [0074] “the vehicle computing system 120 may determine a safe distance from the traffic entity that the autonomous vehicle should maintain based on the motion parameters of the traffic entity.” Maintaining a safe distance from a traffic entity implies that said entity is the closest to the vehicle.).
Regarding claim 13, Anthony teaches The computer-implemented method of claim 1, comprising generating the modified trajectory to avoid collision with: (i) one or more existing trajectories of one or more other actors in the environment; or (ii) an initial trajectory for the subject vehicle in the environment (At least par [0052] “The motion planner 340 perform planning for the motion of the autonomous vehicle, for example, to ensure that the autonomous vehicle drives to its destination through a safe path, avoiding collisions with stationary or non-stationary objects.”).
Regarding claim 14, Anthony teaches The computer-implemented method of claim 1, wherein the autonomous vehicle control system is a simulated AV control system (At least par [0087] “The computing system 300 provides 830 the annotated simulated environment representation to the motion planner 340 as input. The motion planner 340 uses the simulated environment representation for generating a motion plan for a simulated autonomous vehicle or an actual autonomous vehicle being used in a test environment.”).
Claim 15 describes a system which performs the steps detailed in the method of claim 1, thus is rejected on the same basis. Additionally, Anthony teaches one or more processors (processor 202), and one or more computer-readable medium storing instructions (machine-readable medium 1122).
Regarding claim 16, Anthony teaches:
The computing system of claim 15, wherein:
the initial sensor data comprises light detection and ranging (LiDAR) data (At least par [0037] “Data is collected from cameras or other sensors 200 including solid state Lidar, rotating Lidar…”); and
the simulated sensor data comprises simulated LiDAR data (At least par [0096] “The GAN based simulation module 320 provides 1002 the feature vector to the rendering engine to generate simulation data, such as image or video data, that includes the set of traffic entities with characteristics described by the feature vector.”).
Regarding claim 18, Anthony teaches The computing system of claim 15, wherein the initial sensor data comprises real-world sensor data previously collected by one or more physical sensors in the environment (At least par [0063] “The “real world” or “live data” video or other sensor frames from a car-mounted sensor are provided 500 as input to the machine learning based model.”).
Claim 20 describes one or more non-transitory computer-readable medium storing instructions, which when executed by a processor, perform the steps detailed in claim 1, thus is rejected on the same basis. Additionally, Anthony teaches one or more machine-learned models (At least par [0023] “machine learning based model …”), and a non-transitory computer-readable medium (At least par [0102] “machine-readable medium …”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Anthony in view of Zhang et al. (US Pub. 2020/0257291 A1; hereafter Zhang).
Zhang was cited in the IDS filed 12/20/2024.
Regarding claim 5, Anthony teaches the computer-implemented method of claim 1.
Anthony does not teach:
wherein the modified trajectory is parameterized as a series of kinematic bicycle model states.
However, Zhang, within the same field of endeavor, teaches:
wherein the modified trajectory is parameterized as a series of kinematic bicycle model states (At least par [0054] “The state estimator may be based on a so-called “bicycle model” of vehicle kinematics, or any other suitable model for estimating such parameters.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Anthony with Zhang. This modification would have been obvious as both Anthony and Zhang contain subject matter within the same field of endeavor (autonomous vehicle control) and Anthony ¶ [0004] notes that “Failure of autonomous vehicles fail to accurately predict motion of non-stationary traffic objects results in unnatural movement of the autonomous vehicle, for example, as a result of the autonomous vehicle suddenly stopping due to a pedestrian moving in the road or the autonomous vehicle continuing to wait for a person to cross a street even if the person never intends to cross the street.” Introducing a kinematic bicycle model (i.e. state estimator), as taught by Zhang, to the system as described by Anthony, would help more accurately predict motion of non-stationary traffic objects (e.g. pedestrian, vehicle, etc.), as evidenced in Zhang ¶ [0054] “A state estimator 86 thereafter estimates vehicle state variables. In an exemplary embodiment, the state variables include longitudinal position, lateral position, yaw rate, and lateral velocity. The state estimator may be based on a so-called “bicycle model” of vehicle kinematics…” This may help Anthony reduce jerk of the autonomous vehicle and increase passenger comfort.
Claims 6-8 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Anthony in view of Mattyus et al. (US Pub. 2019/0147320 A1; hereafter Mattyus).
Mattyus was cited in the IDS filed 12/20/2024.
Regarding claim 6, Anthony teaches The computer-implemented method of claim 1.
Anthony does not teach:
optimizing an adversarial loss function based on the updated trajectory for the subject vehicle to generate an adversarial loss value.
However, Mattyus, within the same field of endeavor, teaches:
optimizing an adversarial loss function based on the updated trajectory for the subject vehicle to generate an adversarial loss value (At least par [0027] “modifying, using a loss function of the adversarial network that depends on the ground truth label and the prediction, one or more parameters of the generator network…” Modifying a parameter using a loss function is analogous to generating an adversarial loss value as a parameter may be a value. Also see [0140] “… vehicle computing system 106 can process, using an adversarial network model having a loss function that has been implemented based on a siamese discriminator network model, input data to determine output data.” Output data may be considered a value obtained from an adversarial loss function.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Anthony with Mattyus. This modification would have been obvious as both Anthony and Mattyus contain subject matter within the same field of endeavor (autonomous system training) and Anthony ¶ [0004] notes “Failure of autonomous vehicles fail to accurately predict motion of non-stationary traffic objects results in unnatural movement of the autonomous vehicle…“ Introducing the prediction system 230, as taught by Mattyus, to the system as described by Anthony, would help increase the accuracy of motion prediction for non-stationary traffic objects. Thus, decrease the number of instances an autonomous vehicle experiences unnatural movement (e.g. jerk, quick acceleration). This would also lead to increased passenger comfort.
Regarding claim 7, the combination of Anthony and Mattyus teaches The computer-implemented method of claim 6. Mattyus further teaches:
wherein the adversarial loss function comprises at least one of:
an imitation-learning cost term that encourages the updated trajectory to deviate from an original trajectory of the subject vehicle in the traffic scenario;
a cumulative collision cost term that encourages one or more perturbation values to cause the subject vehicle to collide with the one or more actors (At least par [0099] “In some non-limiting embodiments or aspects, the cost associated with the cost function increases and/or decreases based on autonomous vehicle 104 deviating from a motion plan (e.g., a selected motion plan, an optimized motion plan, a preferred motion plan, etc.). For example, the cost associated with the cost function increases and/or decreases based on autonomous vehicle 104 deviating from the motion plan to avoid a collision with an object.”); or
a comfort cost term that encourages the updated trajectory to have lane violations, high acceleration, or jerk.
Regarding claim 8, the combination of Anthony and Mattyus teaches The computer-implemented method of claim 6. Anthony further teaches:
generating a perturbation search space (At least par [0098] “the GAN based simulation module 320 perturbs the feature vector by accessing and sampling from a parameter space of the feature vector …”);
selecting one or more perturbation values from the perturbation search space based on data indicative of historical observations associated with previously selected perturbation values (At least par [0065] “The prediction is generated automatically by passing the sensor data through the model, where the information is transformed by the internal mechanisms of the model according to the parameters that were set in the training process. Because these summary statistics characterize the distribution of human responses that predict the state of mind of a road user pictured in the stimulus, the predicted statistics are therefore a prediction of the aggregate judgment of human observers of the state of mind of the pictured road user and thus an indirect prediction of the actual state of mind of the road user.”); and
adding the one or more perturbation values and the adversarial loss value to the data indicative of historical observations (At least par [0088] “transfer the generated hidden context parameters or human annotated hidden context parameters from real world situations into the simulation environment.”).
Regarding claim 19, Anthony teaches The computing system of claim 15, wherein the autonomous vehicle control system comprises one or more machine-learned models (At least par [0026] “The system uses a machine learning based model”) and the operations further comprising:
updating one or more values of one or more parameters of the one or more machine-learned models based on the adversarial loss function (At least par [0054] “The machine learning based model is trained by a process of progressively adjusting the parameters of the machine learning based model in response to the characteristics of the images and summary statistics given to it in the training phase to minimize the error in its predictions of the summary statistics for the training images in step 804. In one embodiment of the model training system 112, the machine learning based model can be a deep neural network. In this embodiment the parameters are the weights attached to the connections between the artificial neurons comprising the network.”). While Mattyus further teaches:
optimizing an adversarial loss function based on the updated trajectory for the subject vehicle to generate an adversarial loss value (At least par [0027] “modifying, using a loss function of the adversarial network that depends on the ground truth label and the prediction, one or more parameters of the generator network…” Modifying a parameter using a loss function is analogous to generating an adversarial loss value as a parameter may be a value. Also see [0140] “vehicle computing system 106 can process, using an adversarial network model having a loss function that has been implemented based on a siamese discriminator network model, input data to determine output data.” Output data may be considered a value obtained from an adversarial loss function.).
Allowable Subject Matter
Claim 17 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure:
Silva et al. (US Pub. 2021/0370921 A1)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jonathan E Reinert whose telephone number is (571)272-1260. The examiner can normally be reached Mon - Thurs 7AM - 5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James J Lee can be reached at (571) 270-5965. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.E.R./Examiner, Art Unit 3668
/JAMES J LEE/Supervisory Patent Examiner, Art Unit 3668
Prosecution Insights