DETAIL OFFICE ACTIONS
The United States Patent & Trademark Office appreciates the response filed for the current application that is submitted on 4/1/2026. The United States Patent & Trademark Office reviewed the following documents submitted and has made the following comments below.
Applicant Arguments:
In regards to Argument 1, Applicant/s state/s the prior art does not teach “wherein the one or more ground truth trajectories are generated by a teacher model using the first training input,” and therefore, the rejection of 102(a)(2) should be removed.
In regards to Argument 2, Applicant/s state/s the Examiner improperly interprets the limitation of “a vehicle” and the “the vehicle” for the claim 14, and therefore, the rejection of 35 U.S.C. 103 should be removed.
Examiner’s Responses:
In response to Argument 1, Applicant’s arguments, see Remarks, filed 4/1/2026, with respect to the rejection(s) of claim(s) 1-8, 10 and 12-13 under 35 U.S.C. 102(a)(2) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Refaat (U.S. Patent Pub. No. 2021/0133582, hereafter referred to as Refaat) in view of Elluswamy et al (U.S. Patent Pub. No. 2020/0250473, hereafter referred to as Elluswamy).
In response to Argument 2, Applicant’s arguments, see Remarks, filed 4/1/2026, with respect to the rejection(s) of claim(s) 9, 11, 14-20 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Refaat (U.S. Patent Pub. No. 2021/0133582, hereafter referred to as Refaat) in view of Elluswamy et al (U.S. Patent Pub. No. 2020/0250473, hereafter referred to as Elluswamy) in view of Yin et al (U.S. Patent Pub. No. 2022/0180202, hereafter referred to as Yin).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under pre-AIA 35 U.S.C. 103(a) are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-8, 10 and 12-13 are rejected under 35 U.S.C. § 103 as being unpatentable over Refaat (U.S. Patent Pub. No. 2021/0133582, hereafter referred to as Refaat) in view of Elluswamy et al (U.S. Patent Pub. No. 2020/0250473, hereafter referred to as Elluswamy).
Regarding Claim 1, Refaat discloses image processing system that control and vehicles by projecting trajectory of the vehicle. Specifically, Refaat teaches a method comprising obtaining first training data (paragraph 70, Refaat teaches training examples that include a training input that includes data characterization. The Examiner interprets the training input that is required for the training examples, which also include data characterization as an initial means such as first training data by which the method claimed will be carried out.), wherein the training data comprises: a first training input representative of a driving environment of a vehicle (paragraph 79, Refaat teaches a training network input that includes data characterizing a scene in an environment in a vicinity of the vehicle. The Examiner interprets the training network input as the first training input and the data characterized as a scene in an environment in a vicinity of the vehicle as a driving environment of a vehicle.), the driving environment comprising one or more objects (paragraphs 31-33, Refaat teaches that the environment in a vicinity of the vehicle includes an agent, which is described as anything present in the vicinity of the vehicle including a pedestrian, bicyclists, or other vehicles. The Examiner interprets the vicinity of the vehicle as the driving environment and any agent in the vicinity as one or more objects in the driving environment.), training, using the training data (paragraph 4, Refaat teaches using values of the parameters of the first sub neural network, which is known as the training network input, to generate a training intermediate representation. The Examiner interprets this as values from the first sub neural network being used to generate the training intermediate representation as the teacher model being trained using training data.), a student model to predict one or more trajectories (paragraph 4, Refaat teaches a second sub neural network to generate respective training confidence scores for each of the one or more candidate future trajectories. The Examiner interprets the second sub neural network as the student model and the confidence scores assigned to trajectories as a means to predict one or more trajectories.), wherein training the student model (paragraphs 4, 10, 16, 58, 60, Refaat teaches a difference between the training predicted future trajectory and the ground truth future trajectory that is used to compute gradients of first and second losses respective to the first and second sub neural networks to update the values of the parameters of the first and second sub neural networks. The Examiner interprets the difference between the training predicted future trajectory and the ground truth future trajectory as the difference between one or more trajectories predicted by the student model and one or more ground truth trajectories.) comprises reducing a difference between the one or more trajectories predicted by the student model and the one or more ground truth trajectories (paragraph 4, paragraph 60, Refaat teaches a difference between the training predicted future trajectory and the ground truth future trajectory that is used compute gradients of first and second losses respective to the first and second sub neural networks to update the values of the parameters of the first and second sub neural networks. The Examiner interprets the difference between the training predicted future trajectory and the ground truth future trajectory as the difference between one or more trajectories predicted by the student model and one or more ground truth trajectories.).
Refaat does not explicitly disclose one or more ground truth trajectories associated with a forecasted motion of the vehicle within the driving environment, wherein the one or more ground truth trajectories are generated by a teacher model using the first training input.
However, one or more ground truth trajectories associated with a forecasted motion of the vehicle within the driving environment (paragraph 58, Elluswamy teaches vehicle sensor with predicted 3D trajectories of the lane lines and captures image data for the input into a trained machine learning model), wherein the one or more ground truth trajectories are generated by a teacher model using the first training input (paragraph 57-paragraph 60, Elluswamy teaches the ground truth being created from a model for determining trajectories.). Elluswamy is analogous art because it is related to vehicle control using image processing and neural network analysis.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the scoring model of Refaat with the neural network model generation of Elluswamy to create a system that generates models for determining trajectories of vehicles because Elluswamy teaches that it is often difficult to collect and accurately label data that a machine learning model needs improvement on, and there exists a need to improve the process for generating training data with accurate labeled features (paragraph 1, Elluswamy), and one of ordinary skill would have recognized that incorporating this feature into the neural network scoring model of Refaat would lead to improved prediction accuracy and training efficiency of the scoring neural network (paragraph 17, Refaat).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
In regards to Claim 2, Refaat in view of Elluswamy teaches wherein the first training input representative of the driving environment of the vehicle comprises: a history of motion of (i) the vehicle and (ii) the one or more objects (paragraph 32 and 42, Refaat teaches the trajectory of an agent being made possible by data defining, which is known by the environment at the time point and characteristics of the motion of the agent at the time point, including velocity of the agent; as well as an input that includes data characterizing a scene in an environment in the vicinity of the vehicle, including data representing one or more candidate future trajectories; paragraph 15, paragraph 57-paragraph 60, Elluswamy teaches previous and subsequent time series elements in the groups of image data.).
In regards to Claim 3, Refaat in view of Elluswamy teaches In regards to Claim 3, Refaat teaches wherein the first training input representative of the driving environment of the vehicle further comprises at least one of: a status of one or more traffic lights of the driving environment of the vehicle, a roadgraph information associated with the driving environment of the vehicle, or a representation of a target route of the vehicle through the driving environment of the vehicle (paragraph 4, Refaat teaches the training network input being used to generate a training intermediate representation of the environment in a vicinity of the vehicle; paragraph 15, Elluswamy.).
In regards to Claim 4, Refaat in view of Elluswamy teaches wherein the one or more trajectories predicted by the student model comprise a first set of probable trajectories of the vehicle (paragraph 81, Refaat teaches the generation of respective training confidence scores for each of the one or candidate future trajectories using the second sub neural network; paragraph 57-paragraph 60, Elluswamy.), wherein each probable trajectory of the first set of probable trajectories of the vehicle comprises one or more locations of the vehicle (paragraphs 81 and 82, Refaat teaches future trajectories of location defining a path for travel of the vehicle, which would be the probable set of trajectories of the vehicle. Also each taught confidence score corresponding to a predicted likelihood that the agent will follow the corresponding candidate future trajectory. The Examiner interprets the correspondence of a predicted likelihood of the agent, which includes different objects at different moments in the environment of the vicinity of the vehicle as probable trajectories corresponding to one or more locations of the vehicles at one or more future times; paragraph 56-60, Elluswamy) at corresponding one or more future times (paragraph 81, Refaat teaches each confidence score corresponding to a predicted likelihood that the agent will follow the corresponding candidate future trajectory. The Examiner interprets the correspondence of a predicted likelihood of the agent, which includes different objects at different moments in the environment of the vicinity of the vehicle as probable trajectories corresponding to one or more locations of the vehicles at one or more future times; paragraph 56-60, Elluswamy.).
In regards to Claim 5, Refaat in view of Elluswamy teaches wherein each probable trajectory of the first set of probable trajectories of the vehicle further comprises one or more velocities of the vehicle at the corresponding one or more future times (paragraph 32, Refaat teaches the trajectories being predicted using characteristics of motion of an agent at specific time points such as location, velocities, acceleration, and headings or orientation of agents, which can be used to obtain the locations and velocities of the autonomous vehicle and agents in the driving environment. The Examiner interprets the characteristics of motion such as velocity at specific time points as velocity of the vehicle at the corresponding one or more future times; paragraph 51-paragraph 55, Elluswamy).
In regards to Claim 6, Refaat in view of Elluswamy teaches wherein the one or more trajectories predicted by the student model comprise a second set of probable trajectories of at least one object of the one or more objects of the driving environment of the vehicle (paragraphs 81 and 82, Refaat teaches the training predicted future trajectory, which contains data on the future path for an agent in the environment of the vehicle, that is predicted in combination of the second sub neural network and trajectory generation neural network. The Examiner interprets agents as one or more objects in the driving environment of the vehicle, the second sub neural network as the student model, and the training predicted future trajectory predicted by the second sub neural network as the second set of probable trajectories predicted by the student model; paragraph 47-paragraph 52, Elluswamy teaches determining several potential trajectories for a lane line are detected.).
In regards to Claim 7, Refaat in view of Elluswamy teaches wherein each of the one or more predicted trajectories comprises a plurality of temporal segments (paragraph 31, Refaat teaches predicted trajectories with multiple time points and spatial position occupied by the agent in the environment at the time point and characteristics of the motion of the agent at a time point, an example of temporal segment, which is the driving of image sequences into meaningful chunks based on time or content changes such as motion. The Examiner interprets this as predicted trajectories being comprised of a plurality of temporal segments predicted.), and wherein the plurality of temporal segments of a respective predicted trajectory are generated in parallel (paragraph 31, Refaat teaches predicted trajectories with multiple time points and spatial position occupied by the agent in the environment at the time point and characteristics of the motion of the agent at a time point, an example of temporal segment, which is the driving of image sequences into meaningful chunks based on time or content changes such as motion. The Examiner interprets this as predicted trajectories being comprised of a plurality of temporal segments predicted in parallel; paragraph 47-paragraph 52, Elluswamy teaches determining several potential trajectories for a lane line are detected, it assume by one of ordinary skilled in the art that the at the exact moment multiple trajectories are calculated.).
In regards to Claim 8, Refaat in view of Elluswamy teaches wherein each of the one or more ground truth trajectories comprises a plurality of temporal segments (paragraph 31, Refaat teaches predicted trajectories with multiple time points and spatial position occupied by the agent in the environment at the time point and characteristics of the motion of the agent at a time point, an example of temporal segment, which is the driving of image sequences into meaningful chunks based on time or content changes such as motion. The Examiner interprets this as predicted trajectories being comprised of a plurality of temporal segments), and wherein the plurality of temporal segments of a respective ground truth trajectory are generated iteratively (paragraph 31, Refaat teaches predicted trajectories with multiple time points and spatial position occupied by the agent in the environment at the time point and characteristics of the motion of the agent at a time point, an example of temporal segment, which is the driving of image sequences into meaningful chunks based on time or content changes such as motion. The Examiner interprets this as predicted trajectories being comprised of a plurality of temporal segments predicted iteratively; paragraph 47-paragraph 52, Elluswamy teaches determining several potential trajectories for a lane line are detected.), a later temporal segment of the respective ground truth trajectory being predicated on at least one earlier temporal segment of the respective ground truth trajectory (Fig. 5, paragraphs 33, 34, and 80, Refaat teaches series of data values in the time channel that correspond to spatial positions that define time point that the agent occupies to obtain ground truth outputs based on a ground truth future trajectory of the agent, showing that the actual path was predicated. The Examiner interprets this as a later temporal segment of the respective ground truth trajectory being predicated on an earlier temporal segment of the respective ground truth trajectory.).
In regards to Claim 10, Refaat in view of Elluswamy teaches the teacher model is trained by: obtaining a second training data (paragraph 4, Refaat teaches obtaining ground truth output as a second training data; paragraph 12-13, paragraph 33, Elluswamy teaches using different sensor data for label training) wherein the second training data comprises: a second training input associated with one or more driving missions (paragraphs 4, 15, 27, and 45, Refaat teaches the ground truth output serving as an objective standard used in training the vehicle’s planning system to adjust its own trajectory and avoid collisions by stopping, or turning right or left. The Examiner interprets avoiding collisions as a driving mission; paragraph 12-13, paragraph 33, paragraph 39, Elluswamy teaches using different sensor data for label training.), and one or more trajectories of the vehicle recorded during the one or more driving missions (paragraph 57-paragraph 60, Elluswamy teaches the ground truth being created from a model for determining trajectories.); and training, using the second training data, the teacher model to generate one or more training trajectories of the vehicle (paragraph 5, Refaat teaches backpropagation of the second sub neural network, where the input is the ground truth output. into the first sub neural network to update the parameter values of the first sub neural network used to generate training trajectories of the vehicle. The Examiner interprets the use of the ground truth output, known as the second training input, as the input into the second sub neural network that is backpropagated or fed into the first sub neural network, known as the teacher model, as using second training data to train the teacher model to generate one or more training trajectories., wherein training the teacher model comprises reducing a difference between the one or more training trajectories of the vehicle and the one or more recorded trajectories of the vehicle (paragraph 4, Refaat teaches a difference between the training predicted future trajectory and the ground truth future trajectory that is used compute gradients of first and second losses respective to the first and second sub neural networks to update the values of the parameters of the first and second sub neural networks. The Examiner interprets the difference between the training predicted future trajectory and the ground truth future trajectory as the difference between one or more trajectories predicted by the first model and one or more ground truth trajectories.).
In regards to Claim 12, Refaat in view of Elluswamy teaches further comprising: causing the student model to be deployed onboard an autonomous vehicle (paragraph 3, Refaat teaches a plurality of sub neural networks such as the first and second sub neural networks being implemented onboard an autonomous vehicle. The Examiner interprets the second sub neural network as the student model; paragraph 51, Elluswamy).
In regards to Claim 13, Refaat in view of Elluswamy teaches further comprising:
obtaining inference data representative of a new environment of the autonomous vehicle (paragraph 44, Elluswamy teaches surrounding environment);
applying the student model to the inference data to predict one or more trajectories of the autonomous vehicle in the new environment (paragraph 47-paragraph 50, paragraph 57-61, Elluswamy); and
causing a driving path of the autonomous vehicle to be modified in view of the one or more predicted trajectories of the autonomous vehicle (paragraph 27, Refaat teaches that the onboard system can generate planning decisions like the future trajectory of the vehicle to assist in operating the vehicle safely, such as adjusting the future trajectory of the vehicle, including obtaining environment data in response to determining the trajectory of another vehicle that is likely to cross the trajectory of the vehicle. The Examiner interprets generating planning decisions using obtained environment data to modify the trajectory of the vehicle as obtaining inference data representative of a new environment of the autonomous vehicle, applying the student model to the data for the prediction of trajectories in the new environment and modifying the driving path in view of predicted trajectories.).
Claim(s) 9, 11, 14-20 are rejected under 35 U.S.C. § 103 as being unpatentable over Refaat (U.S. Patent Pub. No. 2021/0133582, hereafter referred to as Refaat) in view of Elluswamy et al (U.S. Patent Pub. No. 2020/0250473, hereafter referred to as Elluswamy) in view of Li et al (U.S. Patent Pub. No. 2025/0171051, hereafter referred to as Li).
Regarding Claim 9, Refaat in view of Elluswamy discloses vision system that uses neural network to detect and determine trajectories of a vehicle.
Refaat in view of Elluswamy does not explicitly disclose wherein each of the teacher model and the student model comprise one or more of: a self-attention block of artificial neurons, a cross-attention block of artificial neurons, or a multilayer perceptron block of artificial neurons.
However, Li teaches wherein each of the teacher model and the student model comprise one or more of: a self-attention block of artificial neurons (paragraph 184, Li teaches layers of neurons and cross attention.), a cross-attention block of artificial neurons (paragraph 184, Li teaches layers of neurons and cross attention.), or a multilayer perceptron block of artificial (paragraph 184, Li teaches layers of neurons and cross attention.).
Li is analogous art because there is a neural network for control and analysis image data of vehicle to determine trajectory direction.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of model training of Refaat in view of Elluswamy with allowing refined models to train other models as taught by Li so that training does not overly rely on image data alone. Li teaches that the disclosed techniques, when implemented to control vehicles, can result in those vehicles being driven in a manner that is safer and more similar to how human drivers drive vehicles than what can typically be achieved using conventional machine learning models (paragraph 9, Li). One of ordinary skill would have recognized that incorporating this feature into the training of the neural network of Refaat in view of Elluswamy would allow the trained machine learning models to be used to guide decisions and/or perform tasks related to the data and/or other similar data (paragraph 3, Li).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 11, Refaat in view of Elluswamy discloses vision system that uses neural network to detect and determine trajectories of a vehicle.
Refaat in view of Elluswamy does not explicitly disclose wherein the second training input comprises the first training input.
However, Li teaches wherein the second training input comprises the first training input (paragraph 202-paragraph 209, Li teaches re-training machine learning for generating trajectories.).
Li is analogous art because there is a neural network for control and analysis image data of vehicle to determine trajectory direction.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of model training of Refaat in view of Elluswamy with allowing refined models to train other models as taught by Li so that training does not overly rely on image data alone. Li teaches that the disclosed techniques, when implemented to control vehicles, can result in those vehicles being driven in a manner that is safer and more similar to how human drivers drive vehicles than what can typically be achieved using conventional machine learning models (paragraph 9, Li). One of ordinary skill would have recognized that incorporating this feature into the training of the neural network of Refaat in view of Elluswamy would allow the trained machine learning models to be used to guide decisions and/or perform tasks related to the data and/or other similar data (paragraph 3, Li).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 14, Refaat discloses image processing system that control and vehicles by projecting trajectory of the vehicle. Specifically, Refaat teaches a sensing system of a vehicle, the sensing system (paragraph 70, Refaat teaches training examples that include a training input that includes data characterization. The Examiner interprets the training input that is required for the training examples, which also include data characterization as an initial means such as first training data by which the method claimed will be carried out.) configured to:
acquire sensing data for a driving environment of the vehicle (paragraph 79, Refaat teaches a training network input that includes data characterizing a scene in an environment in a vicinity of the vehicle. The Examiner interprets the training network input as the first training input and the data characterized as a scene in an environment in a vicinity of the vehicle as a driving environment of a vehicle.); and a data processing system of the vehicle, the data processing system configured to: generate, using the acquired sensing data, an inference data characterizing one or more objects in the environment of the vehicle (paragraphs 31-33, Refaat teaches that the environment in a vicinity of the vehicle includes an agent, which is described as anything present in the vicinity of the vehicle including a pedestrian, bicyclists, or other vehicles. The Examiner interprets the vicinity of the vehicle as the driving environment and any agent in the vicinity as one or more objects in the driving environment.).
Refaat does not explicitly disclose apply a first model to the inference data to predict one or more trajectories of the vehicle in the environment, wherein the first model is trained using a second model, wherein the second model comprises an autoregressive model; and cause a driving path of the vehicle to be modified in view of the one or more predicted trajectories.
However, apply a first model to the inference data to predict one or more trajectories of the vehicle in the environment (paragraph 58, Elluswamy teaches vehicle sensor with predicted 3D trajectories of the lane lines and captures image data for the input into a trained machine learning model). Elluswamy is analogous art because it is related to vehicle control using image processing and neural network analysis.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the scoring model of Refaat with the neural network model generation of Elluswamy to create a system that generates models for determining trajectories of vehicles because Elluswamy teaches that it is often difficult to collect and accurately label data that a machine learning model needs improvement on, and there exists a need to improve the process for generating training data with accurate labeled features (paragraph 1, Elluswamy), and one of ordinary skill would have recognized that incorporating this feature into the neural network scoring model of Refaat would lead to improved prediction accuracy and training efficiency of the scoring neural network (paragraph 17, Refaat).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention
Refaat in view of Elluswamy wherein the first model is trained using a second model, wherein the second model comprises an autoregressive model; and cause a driving path of the vehicle to be modified in view of the one or more predicted trajectories.
However, Li teaches wherein the first model is trained using a second model (paragraph 188-paragraph 191, Li teaches continuing to train the BEV planner model for generating trajectories.), wherein the second model comprises an autoregressive model (paragraph 188-paragraph 191, Li teaches continuing to train the BEV planner model for generating trajectories, and further teaches that the model trainer 116 can continue training for a fixed number of iterations, until the loss computed at stop 1006 stops improving.); and cause a driving path of the vehicle to be modified in view of the one or more predicted trajectories (paragraph 188-paragraph 191, Li teaches updating the generate trajectories in a loop for improvement). Li is analogous art because there is a neural network for control and analysis image data of vehicle to determine trajectory direction.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of model training of Refaat in view of Elluswamy with allowing refined models to train other models as taught by Li so that training does not overly rely on image data alone. Li teaches that the disclosed techniques, when implemented to control vehicles, can result in those vehicles being driven in a manner that is safer and more similar to how human drivers drive vehicles than what can typically be achieved using conventional machine learning models (paragraph 9, Li). One of ordinary skill would have recognized that incorporating this feature into the training of the neural network of Refaat in view of Elluswamy would allow the trained machine learning models to be used to guide decisions and/or perform tasks related to the data and/or other similar data (paragraph 3, Li).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 15, Refaat in view of Elluswamy in view of Li discloses wherein the inference data comprises one or more (The Examiner finds the claim states "one or more" under Broadest Reasonable Interpretation, the Examiner only need to find one limitation.) of: a history of motion of (i) the vehicle and (ii) the one or more objects, a status of one or more traffic lights of the driving environment of the vehicle, a roadgraph information associated with the driving environment of the vehicle, or a representation of a target route of the vehicle through the driving environment of the vehicle (paragraph 31, Refaat teaches environment data that characterizes a scene of an environment in a vicinity of the vehicle at a current time point, particularly as data that describes any agents that are present in the vicinity of the vehicle. The Examiner interprets the environment data, which is previously not seen before visual inputs from the environment in the vicinity of the vehicle as inference data that is comprised of a representation of a target route of the vehicle through the driving environment of the vehicle.).
Regarding Claim 16, Refaat in view of Elluswamy in view Li discloses wherein each of the one or more (The Examiner finds the claim states "one or more" under Broadest Reasonable Interpretation, the Examiner only need to find one limitation.) trajectories predicted by the first model comprises (i) one or more locations of the vehicle at corresponding one or more future times; and (ii) one or more velocities of the vehicle at the corresponding one or more future times (paragraph 32, Refaat teaches the trajectories being predicted using data defining, which incorporates the characteristics of motion of an agent at specific time points such as location, velocities, acceleration, and headings or orientation of agents, which is expressed in angular data. These components can be used to obtain the locations and velocities of the autonomous vehicle and agents in the driving environment. The Examiner interprets the characteristics of motion of an agent at specific time points in the driving environment as part of the prediction by the first model to comprise locations and velocities of the vehicle at corresponding future times.).
.
Regarding Claim 17, Refaat in view of Elluswamy in view Li discloses wherein the first model is trained by: obtaining training data (paragraph 4, Refaat teaches the first sub neural network being trained by the obtained values of the parameters of the training network input. The Examiner interprets the first sub neural network as the first model and the parameters of the training network input as training data.), wherein the training data comprises: a training input representative of a training environment of a reference vehicle, the training environment comprising a plurality of objects (paragraph 4, Refaat teaches the training intermediate representation of the environment in a vicinity of the vehicle, including agents, from the training network input. The Examiner interprets the training intermediate representation as a training input representative of the training environment of a reference vehicle and the environment of the vicinity of the vehicle including agents as the training environment comprising a plurality of objects.), and one or more ground truth trajectories associated with a forecasted motion of the reference vehicle within the training environment (paragraph 80, Refaat teaches a ground truth output that defines a ground truth future trajectory of the agent within the vicinity of the vehicle used in the development of the method. The Examiner interprets the ground truth output that defines the ground truth future trajectories as a ground truth trajectory that is associated with a forecasted motion of the vehicle since ground truth trajectories are real-world data that serve as reference outputs to define the actual path within the training environment.), wherein the one or more ground truth trajectories are generated by the autoregressive model using the training input (paragraph 188-paragraph 191, Li teaches continuing to train the BEV planner model for generating trajectories, and further teaches that the model trainer 116 can continue training for a fixed number of iterations, until the loss computed at stop 1006 stops improving.); and applying the first model to the training input to predict one or more training trajectories of the reference vehicle (figure 2, Refaat teaches the first sub neural network, Sub Neural Network A being feed into the trajectory generation neural network for the training predicted future trajectory. The Examiner interprets the first sub neural network as the first model and the training predicted future trajectory as the training trajectory.); and modifying parameters of the first model to reduce a difference between the one or more training trajectories predicted by the first model and the one or more ground truth trajectories generated by the autoregressive model (paragraph 4, Refaat teaches a difference between the training predicted future trajectory and the ground truth future trajectory that is used to compute gradients of first and second losses respective to the first and second sub neural networks to update the values of the parameters of the first and second sub neural networks. The Examiner interprets the difference between the training predicted future trajectory and the ground truth future trajectory as the difference between one or more trajectories predicted by the first model and one or more ground truth trajectories.).
Regarding Claim 18, Refaat in view of Elluswamy in view Li discloses wherein each of the one or more predicted trajectories of the vehicle comprises a plurality of temporal segments (paragraph 31, Refaat teaches predicted trajectories with multiple time points and spatial position occupied by the agent in the environment at the time point and characteristics of the motion of the agent at a time point, an example of temporal segment, which is the driving of image sequences into meaningful chunks based on time or content changes such as motion. The Examiner interprets this as predicted trajectories being comprised of a plurality of temporal segments predicted.) and wherein the plurality of temporal segments of a respective predicted trajectory are predicted in parallel (paragraph 31, Refaat teaches predicted trajectories with multiple time points and spatial position occupied by the agent in the environment at the time point and characteristics of the motion of the agent at a time point, an example of temporal segments, which is the driving of image sequences into meaningful chunks based on time or content changes such as motion. The Examiner interprets this as predicted trajectories being comprised of a plurality of temporal segments predicted in parallel.).
Regarding Claim 19, Refaat in view of Elluswamy in view Li discloses wherein the first model comprises an encoder neural network and a decoder neural network (paragraph 26, Yin teaches a teacher model, known as the first model, being trained with initial training text, known as the initial input, which is combined with another input to add label information, and example of encoding, and decoded to produce sample data outputs; all of which is performed using neural network models. The Examiner interprets the combination of the initial input and an additional input as an encoder neural network and the decoding to produce sample data outputs as a decoder neural network.), wherein the encoder neural network comprises: one or more self-attention blocks of artificial neurons (paragraph 184, Li teaches layers of neurons and cross attention.); and wherein the decoder neural network comprises: one or more cross-attention blocks of artificial neurons (paragraph 184, paragraph 200-pagraph 207, Li).
Regarding Claim 20, Refaat teaches a non-transitory computer-readable memory storing instructions that, when executed by a processing device, cause the processing device to: obtain an inference data characterizing one or more objects in an environment of a vehicle (paragraph 31, Refaat teaches the on-board system of the vehicle being able to use the obtained raw sensor data that is continually generated by the perception subsystem to generate environment data. The Examiner interprets the on-board system as the data processing system and the environment data generated by the onboard system that is continually generated by the perception subsystem as an inference data characterizing one or more objects in the environment of the vehicle.).
Refaat does not explicitly disclose apply a first model to the inference data to predict one or more trajectories of the vehicle in the environment (paragraph 58, Elluswamy teaches vehicle sensor with predicted 3D trajectories of the lane lines and captures image data for the input into a trained machine learning model).
However, Elluswamy teaches apply a first model to the inference data to predict one or more trajectories of the vehicle in the environment (paragraph 58, Elluswamy teaches vehicle sensor with predicted 3D trajectories of the lane lines and captures image data for the input into a trained machine learning model). Elluswamy is analogous art because it is related to vehicle control using image processing and neural network analysis.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the scoring model of Refaat with the neural network model generation of Elluswamy to create a system that generates models for determining trajectories of vehicles because Elluswamy teaches that it is often difficult to collect and accurately label data that a machine learning model needs improvement on, and there exists a need to improve the process for generating training data with accurate labeled features (paragraph 1, Elluswamy), and one of ordinary skill would have recognized that incorporating this feature into the neural network scoring model of Refaat would lead to improved prediction accuracy and training efficiency of the scoring neural network (paragraph 17, Refaat).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention
Refaat in view of Elluswamy does not teach wherein the first model is trained using a second model, wherein the second model comprises an autoregressive model and cause a driving path of the vehicle to be modified in view of the one or more predicted trajectories.
However, Li teaches wherein the first model is trained using a second model (paragraph 188-paragraph 191, Li teaches continuing to train the BEV planner model for generating trajectories.), wherein the second model comprises an autoregressive model (paragraph 188-paragraph 191, Li teaches continuing to train the BEV planner model for generating trajectories, and further teaches that the model trainer 116 can continue training for a fixed number of iterations, until the loss computed at stop 1006 stops improving.) and cause a driving path of the vehicle to be modified in view of the one or more predicted trajectories (paragraph 188-paragraph 191, Li teaches updating the generate trajectories in a loop for improvement). Li is analogous art because there is a neural network for control and analysis image data of vehicle to determine trajectory direction.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of model training of Refaat in view of Elluswamy with allowing refined models to train other models as taught by Li so that training does not overly rely on image data alone. Li teaches that the disclosed techniques, when implemented to control vehicles, can result in those vehicles being driven in a manner that is safer and more similar to how human drivers drive vehicles than what can typically be achieved using conventional machine learning models (paragraph 9, Li). One of ordinary skill would have recognized that incorporating this feature into the training of the neural network of Refaat in view of Elluswamy would allow the trained machine learning models to be used to guide decisions and/or perform tasks related to the data and/or other similar data (paragraph 3, Li).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Pertinent Art
The prior art made of record and not relied upon is considered pertinent to the applicant’s disclosure.
Abdelrouf et al U.S. Patent Publication No. 202410355206.
Cai et al U.S. Patent No. 11,409,304.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ONEAL R MISTRY whose telephone number is (313)446-4912. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights