DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 05/08/2026 has been entered.
Status of Application
Claims 1-4, 6-21 are pending. Claims 1, 19 and 20 are the independent claims. Claims 1, 2, 16, 19, and 20 have been amended. Claim 5 was previously cancelled. This office action is in response to the Amendments received on 05/08/2026.
Response to Arguments
With respect to Applicant’s Remarks filed on 05/08/2026 “Applicant Arguments/Remarks Made in an Amendment” have been fully considered.
With respect to claims 1, 19 and 20, Examiner agrees that Gochev in view of Amini, the previous art relied upon for the rejection of claim 1, fails to teach/suggest the newly added limitation of “determining, based on AV control commands associated with one or more of the plurality of AV control decisions, a ranking of a plurality of candidate AV control strategies for how the AV should navigate the given stream, wherein the plurality of candidate AV control strategies are representative of distinct navigational behaviors of the AV from a pre-defined list of navigational behaviors; selecting, based on the ranking, a given AV control strategy for how the AV should navigate the given stream;”. However, the other art of Record, Omari (previously relied upon for the rejection of claims 3-4, and 6-7) suggests the aforesaid feature recited in the claims. Therefore, this is the office stance that one with ordinary skill in the art would easily modify the method of determining an autonomous vehicle’s optimized action from plurality of actions, based on the predicted trajectories of the objects/agents in the vehicle’s surrounding and map data (including the information about the locations and traffic lanes), taught by Gochev, to include selecting the optimal action based on the rankings assigned to each action (i.e. strategy in the claim), taught by Omari, to improve the reliability of autonomous vehicle driving by capturing multiple possible control strategies and selecting the best strategy based on the associated ranking. See office action below for detailed rejections and mappings.
Office Note: Due to applicant’s amendments, further claim rejections appear on the record as stated in the below Office Action.
It is the Office’ stance that all of applicant arguments have been considered.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 8-21 are rejected under 35 U.S.C. 103 as being unpatentable over Gochev et al., US 20190161080 A1, hereinafter “Gochev”, in view of Omari et al., US 20210200221, hereinafter “Omari”, further in view of Halder US20240053763, hereinafter “Halder”.
Regarding claim 1, Gochev discloses:
a method for training one or more machine learning ("ML") models for use by an autonomous vehicle ("AV") (Abstract, “Systems and methods for controlling the motion of an autonomous”, Para [0006]-[0007], Para [0032], “the blocking model can include a machine-learned model”, Para [0034], “The vehicle action model can be a machine-learned model configured to determine a plurality of vehicle actions for the autonomous vehicle.”),
the method comprising: obtaining a plurality of actors for a past episode of locomotion of a vehicle, each of the plurality of actors corresponding to an object in an environment of the vehicle during the past episode (Para [0005], “The method includes obtaining,…, data associated with an object within a surrounding environment of an autonomous vehicle.”, Para [0023], “The state data can be indicative of one or more states (e.g., current or past state(s)) of one or more objects that are within the surrounding environment of the autonomous vehicle.”), Para [0028], [0057]);
obtaining a plurality of streams in the environment of the vehicle during the past episode (Para [0021], “obtain map data”, “The map data can provide information regarding: the identity and location of different roadways, road segments, buildings, sidewalks, or other items; the location and directions of traffic lanes (e.g., the boundaries, location, direction, etc. of a parking lane, a turning lane, a bicycle lane, or other lanes within a particular travel way)”),
each of the plurality of streams representing a candidate navigation path, for the vehicle or the object corresponding to a given one of the actors, in the environment of the vehicle (Para [0022], “the autonomy computing system can include a perception system, a prediction system, and a motion planning system.”, Para [0027], “predicted motion trajectory”, Para [0058], “the prediction system 126 can determine a predicted trajectory of an object. The predicted trajectory can be indicative of a predicted path that the object is predicted to travel over time and the timing associated therewith.”);
processing, using one or more ML model layers of one or more of the ML models, the plurality of actors and the plurality of streams to generate predicted output for each of the plurality of actors (at least see [0005], “determining, by the computing system, a motion plan for the autonomous vehicle based at least in part on the vehicle action sequence. The method includes causing, by the computing system, the autonomous vehicle to initiate motion control in accordance with at least a portion of the motion plan.”, [0019], “The autonomous vehicle can determine a vehicle action sequence based at least in part on the blocking information and other data (e.g., vehicle motion parameters, map data, object state data, etc.).”, [0021], “The map data can provide information regarding: the identity and location of different roadways, road segments, buildings, sidewalks, or other items; the location and directions of traffic lanes (e.g., the boundaries, location, direction, etc. of a parking lane, a turning lane, a bicycle lane, or other lanes within a particular travel way)”, [0028], “one or more characteristic(s) of the object relative to a planned vehicle trajectory and/or route associated with the autonomous vehicle (e.g., whether the object is moving parallel, towards, or away from the vehicle's current/future motion trajectory/travel route or a predicted point of intersection with the vehicle), etc.”, __Examiner’s Note: according to Gochev’s disclosure the vehicle’s action is determined according to the predicted trajectory of an object in the vehicle’s surrounding and the map data which including information about traffic lanes which reads on plurality of stream as recited in the claim.__, Para [0029], “the vehicle computing system can determine a respective vehicle action for the autonomous vehicle at each of the respective time steps based at least in part on whether the object is blocking or not blocking the autonomous vehicle at the respective time step.”, Para [0030], Para [0032], “the blocking model can include a machine-learned model (e.g., a machine-learned blocking model).”);
processing, using one or more additional ML model layers of one or more of the ML models, the predicted output for each of the plurality of actors to generate further predicted output for each of the plurality of streams and with respect to each of the plurality of actors (Para [0019], “The autonomous vehicle can predict a trajectory by which the object is to travel over a certain time period”, “The autonomous vehicle can utilize the vehicle action sequence to plan its motion and autonomously navigate through its environment.”, Para [0022], [0024], “prediction system”, Para [0029], “the predicted motion trajectory of the object”, “Then, the vehicle computing system can determine a respective vehicle action for the autonomous vehicle at each of the respective time steps based at least in part on whether the object is blocking or not blocking the autonomous vehicle at the respective time step. ”, Para [0033]- [0034], “The vehicle action model can be a machine-learned model”);
Gochev, further teaches:
wherein the further predicted output for each of the plurality of streams and with respect to each of the plurality of actors is indicative of a plurality of AV control decisions for how the AV should navigate a given stream, of the plurality of streams, based on each of the plurality of actors ([0072], “A vehicle action sequence can be indicative of plurality of vehicle actions for the vehicle 104 at a plurality of respective time steps associated with the predicted motion trajectory of an object.”, “The vehicle action model 138 can be configured to determine a plurality of vehicle actions for the vehicle 104.”, [0073]).
However, Gochev doesn’t teach or suggest determining, based on AV control commands associated with one or more of the plurality of AV control decisions, a ranking of a plurality of candidate AV control strategies for how the AV should navigate the given stream, wherein the plurality of candidate AV control strategies are representative of distinct navigational behaviors of the AV from a pre-defined list of navigational behaviors; selecting, based on the ranking, a given AV control strategy for how the AV should navigate the given stream;
Nevertheless Omari teaches:
determining, based on AV control commands associated with one or more of the plurality of AV control decisions, a ranking of a plurality of candidate AV control strategies for how the AV should navigate the given stream ([0047], “planning module 720 may generate, based on a given predicted contextual presentation, several different plans (e.g., goals or navigation instructions) for the vehicle. For each plan, the planning module 720 may compute a score that represents the desirability of that plan.”, [0048], __Note: AV control strategies in the claim, has been interpreted in light of paragraphs [0021] and [0045] of the instant specification__),
wherein the plurality of candidate AV control strategies are representative of distinct navigational behaviors of the AV from a pre-defined list of navigational behaviors ([0048], “For example, if the plan would likely result in the vehicle colliding with an agent at a predicted location for that agent, as determined based on the predicted contextual representation, the score for the plan may be penalized accordingly. Another plan that would cause the vehicle to violate traffic rules or take a lengthy detour to avoid possible collisions may also have a score that is penalized, but the penalty may be less severe than the penalty applied for the previous plan that would result in a collision. A third plan that causes the vehicle to simply stop or change lanes to avoid colliding with the agent in the predicted future may receive the highest score. Based on the assigned scores for the plans, the planning module 720 may select the best plan to carry out.”, __Examiner’s Note: the disclosed example shows distinct navigational behavior__);
selecting, based on the ranking, a given AV control strategy for how the AV should navigate the given stream ([0047], “Based on the assigned scores for the plans, the planning module 720 may select the best plan to carry out.”).
Gochev doesn’t explicitly teach generating, based on one or more reference labels for the past episode of locomotion and the given AV control strategy for how the AV should navigate the given stream, one or more losses;
However, the recited limitation is the technique of generating loss function based on comparing the reference labels and the output from machine learning, which is well-known in the art. Omari teaches generating loss value based on the difference between the predicted position of the agents and actual recorded position of the agents (See Omari, [0034], [0046]), however, since loss function in Omari does not refer to the given (predicted AV control strategy/action), Examiner relies on Halder, which discloses determining one or more actions for an autonomous vehicle based on machine-learning and artificial intelligence technique (See at least Abstract) and teaches generating, based on one or more reference labels for the past episode of locomotion and the given AV control strategy for how the AV should navigate the given stream, one or more losses ([0007], [0129], [0131]-[0132], [0040]-[0044])
Furthermore, Gochev doesn’t explicitly teach updating, based on the one or more losses, one or more of the additional ML model layers of one or more of the ML models, wherein one or more of the additional ML model layers of one or more of the ML models are subsequently utilized in controlling the AV.
However, Omari teaches:
updating, based on the one or more losses, one or more of the additional ML model layers of one or more of the ML models (Omari, [0034], “The loss value may be used (e.g., via back-propagation techniques) to update the configuration parameters of prioritization model 505 so that the loss value would be less if the prediction were to be made again.”, [0046], “The machine-learning model may be trained iteratively using a large set of training samples until a convergence or termination condition is met.”, __Examiner’s Note: the machine-learning model being trained iteratively reads on one or more additional ML layers of one or more of ML models__),
wherein one or more of the additional ML model layers of one or more of the ML models are subsequently utilized in controlling the AV (Omari [0034], [0046]-[0047], __Note: planning module utilize the predicted output from machine learning which is updated according to the losses (according to at least paragraph [0035]. The disclosed process reads on the claimed limitation__)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method for determining an action and motion plan (reads on AV control strategy) for an autonomous vehicle based on the predicted future locations of objects within the vehicle’s surrounding, as taught by Gochev, to include determining a given AV control strategy, based on the ranking of the plurality of control strategies, in order to navigate a path/stream, as taught by Omari, and further including the step of generating the loss function, as taught by Halder, for improving the accuracy of the predicted control strategy, with a reasonable expectation of success, in order to improve safety and reliability of autonomous vehicle (AV) driving by capturing multiple possible AV control strategies and selecting the best strategy based on the associated ranking. The system enables the autonomous vehicle to select navigation strategies that balance safety and efficiency under uncertainty.
Regarding claim 8, Gochev discloses wherein the further predicted output further comprises a predicted vehicle control strategy or predicted vehicle control commands (Para [0005], “controlling autonomous vehicle motion.”, Para [0019]-[0020], [0022], “determine a motion plan for controlling the motion of the autonomous vehicle.”, Para [0038]-[0040], [0061], [0085]-[0087], [0094]).
Regarding claim 9, Gochev discloses wherein the one or more reference labels comprise an associated reference label that corresponds to a ground truth vehicle control strategy or ground truth vehicle control commands that are determined during the past episode of locomotion of the vehicle or that is defined for the vehicle subsequent to the past episode of locomotion of the vehicle (Para [0036], “These labels can be utilized as ground-truth data to determine the accuracy and/or development of the vehicle action model as it is trained.”, Para [0078]).
Regarding claim 10, Gochev discloses wherein generating one or more of the losses comprises comparing the predicted vehicle control strategy or the predicted vehicle control commands to the ground truth vehicle control strategy or the ground truth vehicle control commands to generate one or more of the losses; (Para [0036], “the machine-learned vehicle action can be trained using supervised training techniques based on training data.”, “These labels can be utilized as ground-truth data to determine the accuracy and/or development of the vehicle action model as it is trained.”, Para [0078] and Fig. 2B, “The model trainer 254 can evaluate a training output 256 of the vehicle action model 138 to determine the accuracy and/or confidence level of the model as it is trained over time. The model trainer 254 can continue to train the vehicle action model 138 until a sufficient level of accuracy and/or confidence is achieved.”, __determining the accuracy and/or confidence level of the model as it is trained reads on generating losses__) and wherein updating the one or more additional ML model layers comprises backpropagating one or more of the losses across the one or more additional ML model layers. (Para [0115], “One example training technique is backwards propagation of errors.”)
Regarding claim 11, Gochev discloses wherein each stream, of the plurality of streams, corresponds to a sequence of poses that represent the candidate navigation path, in the environment of the vehicle, for the vehicle or the object corresponding to a given one of the actors. (Para [0021]-[0030])
Regarding claim 12, Gochev discloses wherein each stream, of the plurality of streams, is at least one of: a target stream corresponding to the candidate navigation path the vehicle will follow, a joining stream that merges into the target stream, a crossing stream that is transverse to the target stream, an adjacent stream that is parallel to the target stream, or an additional stream that is one-hop from the joining stream, the crossing stream, or the adjacent stream. (Para [0021]- [0022], “The map data can provide information regarding: the identity and location of different roadways, road segments,”, Para [0054], “determine a vehicle route for the vehicle 104 based at least in part on the map data 120.”)
Regarding claim 13, Gochev discloses wherein the object corresponding to each of the one or more actors is at least one of: an additional vehicle that is in addition to the vehicle, a bicyclist, or a pedestrian. (Para [0019], “presence of an object (e.g. pedestrian, vehicle, bicycle, or other object) that is within proximity of the autonomous vehicle.”)
Regarding claim 14, Gochev discloses wherein the object is dynamic in the environment of the vehicle along a particular stream of the plurality of streams. (Para [0020], “The object(s) can be static (e.g., not in motion) or dynamic (e.g., actors in motion).”).
Regrading claim 15, Gochev discloses wherein subsequently utilizing one or more of the additional ML model layers of one or more of the ML models in controlling the AV (e.g., Para [0032]-[0036]) comprises: processing, using the one or more ML model layers and the one or more additional ML model layers, sensor data generated by one or more sensors of the AV (Para [0023], “the perception system can process the sensor data from the sensor(s) to detect the one or more objects that are proximate to the autonomous vehicle as well as state data associated therewith.”, Para [0068], [0079]) to predict an AV control strategy or predict AV control commands; and causing the AV to be controlled based on the predicted AV control strategy or the predicted AV control commands. (e.g., Para [0034], [0056], “generate an appropriate motion plan through such surrounding environment. The autonomy computing system 114 can control the one or more vehicle control systems 116 to operate the vehicle 104 according to the motion plan.”)
Regarding claim 16, Gochev does not explicitly teach ranking a plurality of AV control strategies based on the processing, wherein the predicted AV control strategy is a highest ranked AV control strategy.
However, Omari teaches ranking a plurality of AV control strategies based on the processing, wherein the predicted AV control strategy is a highest ranked AV control strategy ([0047], “For each plan, the planning module 720 may compute a score that represents the desirability of that plan.”, “A third plan that causes the vehicle to simply stop or change lanes to avoid colliding with the agent in the predicted future may receive the highest score. Based on the assigned scores for the plans, the planning module 720 may select the best plan to carry out.”.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method for determining an action and motion plan (reads on AV control strategy) for an autonomous vehicle based on the predicted future locations of objects within the vehicle’s surrounding, as taught by Gochev, to include determining a given AV control strategy, based on the ranking of the plurality of control strategies, in order to navigate a path/stream, as taught by Omari, with a reasonable expectation of success, in order to improve the accuracy and reliability of the autonomous driving by outputting the best control strategy according to the different assigned control strategies ranking.
Regarding claim 17, Gochev discloses the method of claim 1, wherein the one or more ML model layers comprise a first portion of a given one of the one or more ML models, and wherein the one or more additional ML model layers comprise a second portion of the given one of the one or more ML models. (Para [0029], “the vehicle computing system can determine a respective vehicle action”, Para [0034], “The vehicle action model can be a machine-learned model”, Para [0036], [0078])
Regarding claim 18, Gochev discloses wherein the one or more ML model layers comprise a first one of the one or more ML models, and wherein the one or more additional ML model layers comprise at least a second one of the one or more ML models (e.g., Para [0032], [0035], [0066]).
Regarding claim 19, Gochev discloses:
a method for training one or more machine learning ("ML") models for use by an autonomous vehicle ("AV") (e.g., Abstract, “Systems and methods for controlling the motion of an autonomous”, Para Para [0006]-[0007], [0032], “the blocking model can include a machine-learned model”, Para [0034], “The vehicle action model can be a machine-learned model configured to determine a plurality of vehicle actions for the autonomous vehicle.”, Para [0041], [0089], Claim 7),
the method comprising: obtaining a plurality of training instances from a past episode of locomotion of a vehicle (e.g., Para [0057], “ obtain state data 130 that is indicative of one or more states (e.g., current and/or past state(s)) of one or more objects that are within a surrounding environment of the vehicle 104.”),
each of the plurality of training instances comprising: training instance input, the training instance input comprising: a given AV control strategy for how the AV should navigate a given stream, of a plurality of streams, based on each of a plurality of actors, the given AV control strategy being selected based on processing predicted output generated using one or more ML model layers of one or more of the ML models (at least see [0005], “determining, by the computing system, a motion plan for the autonomous vehicle based at least in part on the vehicle action sequence. The method includes causing, by the computing system, the autonomous vehicle to initiate motion control in accordance with at least a portion of the motion plan.”, [0019], “The autonomous vehicle can determine a vehicle action sequence based at least in part on the blocking information and other data (e.g., vehicle motion parameters, map data, object state data, etc.).”, [0021], “The map data can provide information regarding: the identity and location of different roadways, road segments, buildings, sidewalks, or other items; the location and directions of traffic lanes (e.g., the boundaries, location, direction, etc. of a parking lane, a turning lane, a bicycle lane, or other lanes within a particular travel way)”, [0028], “one or more characteristic(s) of the object relative to a planned vehicle trajectory and/or route associated with the autonomous vehicle (e.g., whether the object is moving parallel, towards, or away from the vehicle's current/future motion trajectory/travel route or a predicted point of intersection with the vehicle), etc.”, __Examiner’s Note: according to Gochev’s disclosure the vehicle’s action is determined according to the predicted trajectory of an object in the vehicle’s surrounding and the map data which including information about traffic lanes which reads on plurality of stream as recited in the claim.__, also see Para [0089], “The vehicle computing system 102 can input data indicative of the predicted motion trajectory 406 of the object 404 into the blocking model 136. The vehicle computing system can also input data indicative of the motion trajectory of the vehicle 104 into the blocking model 136. ” ), the predicted output being generated based on the plurality of actors and the plurality of streams (Para [0005], “The method includes determining, by the computing system, a motion plan for the autonomous vehicle based at least in part on the vehicle action sequence.”, Para [0019], “The autonomous vehicle can determine a vehicle action sequence based at least in part on the blocking information and other data (e.g., vehicle motion parameters, map data, object state data, etc.).”, Para [0033], [0057], “For example, the vehicle computing system 102 can process the sensor data 118, the map data 120, etc. to obtain state data 130.”) each of the plurality of actors corresponding to an object in an environment of the vehicle during the past episode (Para [0005], “The method includes obtaining,…, data associated with an object within a surrounding environment of an autonomous vehicle.”, Para [0023], “The state data can be indicative of one or more states (e.g., current or past state(s)) of one or more objects that are within the surrounding environment of the autonomous vehicle.” ), and each of the plurality of streams representing a candidate navigation path in the environment of the vehicle (Para [0027], “predicted motion trajectory”);
and training instance output, the training instance output comprising: one or more associated reference labels for the past episode of locomotion, each of the one or more associated reference labels corresponding to an action performed by the vehicle during the past episode of locomotion; and training one or more additional ML layers of one or more of the ML models based on the plurality of training instances, wherein one or more of the additional ML model layers of one or more of the ML models are subsequently utilized in controlling the AV. (Para [0077]-[0078]).
Further, although Gochev disclose the predicted output being indicative of a plurality of AV control decisions for how to navigate the given stream ([0007], “The autonomous vehicle includes a vehicle action model that is configured to determine a plurality of vehicle actions for the autonomous vehicle.” Para [0019], “The autonomous vehicle can determine a vehicle action sequence based at least in part on the blocking information and other data (e.g., vehicle motion parameters, map data, object state data, etc.).”, [0072]-[0073]),
However, Gochev doesn’t explicitly teach the given AV control strategy being selected based on a ranking of a plurality of candidate AV control strategies for how the AV should navigate the given stream, the plurality of candidate AV control strategies representative of distinct navigational behaviors of the AV from a pre-defined list of navigational behaviors, and the ranking being determined based on AV control commands associated with one or more of the plurality of AV control decisions;
Nevertheless, Omari also teaches the claimed limitation already mapped with Gochev in paragraph 35 of this office action, i.e., limitation the predicted output being indicative of a plurality of AV control decisions for how to navigate the given stream (See Omari [0020], [0047], “planning module 720 may generate, based on a given predicted contextual presentation, several different plans (e.g., goals or navigation instructions) for the vehicle. For each plan, the planning module 720 may compute a score that represents the desirability of that plan.”, [0048]), and Omari further teaches:
the given AV control strategy being selected based on a ranking of a plurality of candidate AV control strategies for how the AV should navigate the given stream (See Omari, [0047], “Based on the assigned scores for the plans, the planning module 720 may select the best plan to carry out.”),
the plurality of candidate AV control strategies representative of distinct navigational behaviors of the AV from a pre-defined list of navigational behaviors (See Omari [0048], “For example, if the plan would likely result in the vehicle colliding with an agent at a predicted location for that agent, as determined based on the predicted contextual representation, the score for the plan may be penalized accordingly. Another plan that would cause the vehicle to violate traffic rules or take a lengthy detour to avoid possible collisions may also have a score that is penalized, but the penalty may be less severe than the penalty applied for the previous plan that would result in a collision. A third plan that causes the vehicle to simply stop or change lanes to avoid colliding with the agent in the predicted future may receive the highest score. Based on the assigned scores for the plans, the planning module 720 may select the best plan to carry out.”, __Examiner’s Note: the disclosed example shows distinct navigational behavior); and the ranking being determined based on AV control commands associated with one or more of the plurality of AV control decisions ([0047], “planning module 720 may generate, based on a given predicted contextual presentation, several different plans (e.g., goals or navigation instructions) for the vehicle. For each plan, the planning module 720 may compute a score that represents the desirability of that plan.”, [0048], __Examiner’s Note: AV control strategies in the claim, has been interpreted in light of paragraphs [0021] and [0045] of the instant specification__);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method for determining an action and motion plan (reads on AV control strategy) for an autonomous vehicle based on the predicted future locations of objects within the vehicle’s surrounding, as taught by Gochev, to include determining a given AV control strategy, based on the ranking of the plurality of control strategies, in order to navigate a path/stream, as taught by Omari, with a reasonable expectation of success, in order to improve safety and reliability of autonomous vehicle (AV) driving by capturing multiple possible AV control strategies and selecting the best strategy based on the associated ranking. The system enables the autonomous vehicle to select navigation strategies that balance safety and efficiency under uncertainty.
Regarding claim 20, Gochev discloses:
a system for training one or more machine learning ("ML") models for use by an autonomous vehicle ("AV"), the system comprising: at least one processor; and at least one memory storing instructions (Para [0006])
that, when executed, cause the at least one processor to: obtain a plurality of actors for a past episode of locomotion of a vehicle, each of the plurality of actors corresponding to an object in an environment of the vehicle during the past episode (Para [0116], “sets of data from previous events (e.g., driving log data associated with previously observed objects).”, Para [0023], “he state data can be indicative of one or more states (e.g., current or past state(s)) of one or more objects that are within the surrounding environment of the autonomous vehicle.”);
obtain a plurality of streams in the environment of the vehicle during the past episode, each of the plurality of streams representing a candidate navigation path, for the vehicle or the object corresponding to a given one of the actors, in the environment of the vehicle (Para [0027]-[0028]);
process, using one or more ML model layers of one or more of the ML models, the plurality of actors and the plurality of streams to generate predicted output for each of the plurality of actors (Para [0029], “the vehicle computing system can determine a respective vehicle action for the autonomous vehicle at each of the respective time steps based at least in part on whether the object is blocking or not blocking the autonomous vehicle at the respective time step.”, Para [0030]);
process, using one or more additional ML model layers of one or more of the ML models, the predicted output for each of the plurality of actors to generate further predicted output for each of the plurality of streams and with respect to each of the plurality of actors (Para [0019], “The autonomous vehicle can predict a trajectory by which the object is to travel over a certain time period”, “The autonomous vehicle can utilize the vehicle action sequence to plan its motion and autonomously navigate through its environment.”, Para [0022], [0024], “prediction system”, Para [0029]-[0034], “the predicted motion trajectory of the object”);
Further, although Gochev disclose wherein the further predicted output for each of the plurality of streams and with respect to each of the plurality of actors is indicative of a plurality of AV control decisions for how the AV should navigate a given stream, of the plurality of streams, based on each of the plurality of actors ([0007], “The autonomous vehicle includes a vehicle action model that is configured to determine a plurality of vehicle actions for the autonomous vehicle.” Para [0019], “The autonomous vehicle can determine a vehicle action sequence based at least in part on the blocking information and other data (e.g., vehicle motion parameters, map data, object state data, etc.).”, [0072]-[0073]),
However, Gochev doesn’t teach or suggest determine, based on AV control commands associated with one or more of the plurality of AV control decisions, a ranking of a plurality of candidate AV control strategies for how the AV should navigate the given stream, wherein the plurality of candidate AV control strategies are representative of distinct navigational behaviors of the AV from a pre-defined list of navigational behaviors; select, based on the ranking, a given AV control strategy for how the AV should navigate the given stream;
Nevertheless, Omari also teaches the claimed limitation already mapped with Gochev in paragraph 41 of this office action, i.e., wherein the further predicted output for each of the plurality of streams and with respect to each of the plurality of actors is indicative of a plurality of AV control decisions for how the AV should navigate a given stream, of the plurality of streams, based on each of the plurality of actors ([0020], [0047], “planning module 720 may generate, based on a given predicted contextual presentation, several different plans (e.g., goals or navigation instructions) for the vehicle. For each plan, the planning module 720 may compute a score that represents the desirability of that plan.”, [0048]), and Omari further teaches:
determine, based on AV control commands associated with one or more of the plurality of AV control decisions, a ranking of a plurality of candidate AV control strategies for how the AV should navigate the given stream ([0047], “planning module 720 may generate, based on a given predicted contextual presentation, several different plans (e.g., goals or navigation instructions) for the vehicle. For each plan, the planning module 720 may compute a score that represents the desirability of that plan.”, [0048], __Note: AV control strategies in the claim, has been interpreted in light of paragraphs [0021] and [0045] of the instant specification__),
wherein the plurality of candidate AV control strategies are representative of distinct navigational behaviors of the AV from a pre-defined list of navigational behaviors ([0048], “For example, if the plan would likely result in the vehicle colliding with an agent at a predicted location for that agent, as determined based on the predicted contextual representation, the score for the plan may be penalized accordingly. Another plan that would cause the vehicle to violate traffic rules or take a lengthy detour to avoid possible collisions may also have a score that is penalized, but the penalty may be less severe than the penalty applied for the previous plan that would result in a collision. A third plan that causes the vehicle to simply stop or change lanes to avoid colliding with the agent in the predicted future may receive the highest score. Based on the assigned scores for the plans, the planning module 720 may select the best plan to carry out.”, __Examiner’s Note: the disclosed example shows distinct navigational behavior);
selecting, based on the ranking, a given AV control strategy for how the AV should navigate the given stream ([0047], “Based on the assigned scores for the plans, the planning module 720 may select the best plan to carry out.”).
Further, Gochev doesn’t explicitly teach generate, based on one or more reference labels for the past episode of locomotion and the given AV control strategy for how the AV should navigate the given stream, one or more losses;
However, the recited limitation is the technique of generating loss function based on comparing the reference labels and the output from machine learning, which is well-known in the art. Omari teaches generating loss value based on the difference between the predicted position of the agents and actual recorded position of the agents (See Omari, [0034], [0046]), however, since loss function in Omari does not refer to the given (predicted AV control strategy/action), Examiner relies on Halder which discloses determining one or more actions for an autonomous vehicle based on machine-learning and artificial intelligence technique (See at least Abstract) and teaches generate, based on one or more reference labels for the past episode of locomotion and the given AV control strategy for how the AV should navigate the given stream, one or more losses ([0007], [0129], [0131]-[0132], [0040]-[0044])
Furthermore, Gochev doesn’t explicitly teach update, based on the one or more losses, one or more of the additional ML model layers of one or more of the ML models, wherein one or more of the additional ML model layers of one or more of the ML models are subsequently utilized in controlling the AV.
However, Omari teaches:
update, based on the one or more losses, one or more of the additional ML model layers of one or more of the ML models (Omari, [0034], “The loss value may be used (e.g., via back-propagation techniques) to update the configuration parameters of prioritization model 505 so that the loss value would be less if the prediction were to be made again.”, [0046], “The machine-learning model may be trained iteratively using a large set of training samples until a convergence or termination condition is met.”, __Examiner’s Note: the machine-learning model being trained iteratively reads on one or more additional ML layers of one or more of ML models__),
wherein one or more of the additional ML model layers of one or more of the ML models are subsequently utilized in controlling the AV (Omari [0034], [0046]-[0047], __Note: planning module utilize the predicted output from machine learning which is updated according to the losses (according to at least paragraph [0035]. The disclosed process reads on the claimed limitation__)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method for determining an action and motion plan (reads on AV control strategy) for an autonomous vehicle based on the predicted future locations of objects within the vehicle’s surrounding, as taught by Gochev, to include determining a given AV control strategy, based on the ranking of the plurality of control strategies, in order to navigate a path/stream, as taught by Omari, and further including the step of generating the loss function, as taught by Halder, for improving the accuracy of the predicted control strategy, with a reasonable expectation of success, in order to improve safety and reliability of autonomous vehicle (AV) driving by capturing multiple possible AV control strategies and selecting the best strategy based on the associated ranking. The system enables the autonomous vehicle to select navigation strategies that balance safety and efficiency under uncertainty.
Regarding claim 21, Gochev in view of Omari and Halder teaches the method of claim 1, however, Gochev doesn’t teach wherein the plurality of AV control decisions comprise two or more of: a yield decision indicative of whether the AV should yield to one or more of the plurality of actors while navigating the given stream, a merge decision indicative of whether the AV should merge into the given stream or a given additional stream, or an intersection entry decision indicative of whether the AV should enter an intersection along the given stream.
However, Omari teaches wherein the plurality of AV control decisions comprise two or more of: a yield decision indicative of whether the AV should yield to one or more of the plurality of actors while navigating the given stream, a merge decision indicative of whether the AV should merge into the given stream or a given additional stream, or an intersection entry decision indicative of whether the AV should enter an intersection along the given stream (at least [0020], “A planning module of the driving-control system may utilize a contextual representation [] the driving-control system may determine candidate travel paths 106A-106C and associated driving operations for the vehicle to avoid possible collisions with agents 104A-104D.”, [0039], [0047], “planning module 720 may determine a particular navigation path and associated driving operations for the vehicle to avoid possible collisions with one or more agents. [] A third plan that causes the vehicle to simply stop or change lanes to avoid colliding with the agent” [0048])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method as taught by Gochev in view of Omari and Hadler, to include outputting an optimal vehicle controlling plan from plurality of AV control decisions comprising, for example, a yield or merge decision as taught by Omari, with a reasonable expectation of success, with the motivation of allowing the model to capture multiple possible outputs and providing broader information for safe decision-making. Predicting movement of actors on the road, predicting different possible scenarios of AV controls in navigating through a stream from a plurality of stream with respect to the predicted output for the plurality of actors in the environment of the vehicle, and making decisions of how to control the AV accordingly, enhances the reliability and safety of the AV control system.
Claims 2-4 and 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Gochev, in view of Omari and Halder, further in view of Amini et al., US 20200088525 A1, hereinafter “Amini”.
Regarding claim 2, although Gochev discloses wherein the one or more additional ML layers correspond to a plurality of disparate deciders (Para [0025], “The motion plan can include vehicle actions with respect to the objects within the surrounding environment of the autonomous vehicle as well as the predicted movements.”, Para [0027], “control the motion of the autonomous vehicle by determining a plurality of vehicle actions for a given object during each motion planning cycle.”, Para [0037], “the vehicle action sequence can include the discrete vehicle action decided for each respective time step of the predicted object trajectory.”), however, Gochev doesn’t explicitly disclose wherein the further predicted output for each of the plurality of streams and with respect to each of the plurality of actors comprises a respective probability distribution over the plurality of AV control decisions, and wherein the respective probability distribution over the plurality of AV control decisions comprises an associated probability for a predicted decision made by each decider, of the plurality of disparate deciders, for each of the plurality of streams and with respect to each of the plurality of actors.
Further, the other reference Omari also fails to teach the recited limitations (instead Omari suggests predicting the driving-control system navigation based on determining a predicted probability distribution indicating the likelihood of a particular agent (reads on actor in the claim) entering into an area that the current vehicle may travel.
However, Amini teaches:
wherein the further predicted output for each of the plurality of streams and with respect to each of the plurality of actors comprises a respective probability distribution over the plurality of AV control decisions (Abstract, [0005]-[0007], “a probability distribution for one or more steering trajectories that are available to a vehicle.”, __Note: one or more steering trajectories reads on plurality of AV control decisions__, [0031]-[0032], [0035], “The output from a final fully connected layer 340 b, probabilistic control output 350, can include the parameters of a probability distribution for one or more steering trajectories that are available to a vehicle.”, [0037], [0039], [0041], [0054], [0063]-[0070]),
and wherein the respective probability distribution over a plurality of AV control decisions comprises an associated probability for a predicted decision made by each decider, of the plurality of disparate deciders, for each of the plurality of streams and with respect to each of the plurality of actors ( at least Para [0020], [0034], [0039], “the probabilistic control output 350 includes parameters for a probability distribution (e.g., the GMM parameters) for the trajectories 430a, 430b, and 430c.”, __ Examiner’s Note: the underlined part reads on probability distribution over a plurality of AV control decisions (which are different trajectories that the AV control predicts for travelling of the AV). The prediction is based on the probability distribution of different predicted trajectories (decisions) and base on the road agents movement as disclosed according to at least paragraphs [0020], and [0041]__, Para [0041], “Three possible trajectories available to external road agent 530 are marked with heavy lines as trajectory 520a (left turn), trajectory 520b (proceed straight), and trajectory 520c (right turn), where the trajectories are named from the perspective of external road agent 530. In the embodiment discussed above in connection with FIG. 3, the probabilistic control output 350 includes parameters for a probability distribution (e.g., the GMM parameters) for the trajectories 520a, 520b, and 520c. In some embodiments, computation module 230 can predict that the external road agent 530 will travel along a particular one of the available steering trajectories based, at least in part, on the parameters of the probability distribution, as discussed above. In some embodiments, control module 250 can control one or more vehicle systems 140 of vehicle 100 such as steering system 143 in response to the predicted trajectory of the external road agent 530.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method as taught by Gochev in view of Omari, to include the step of determining/outputting a selection of an stream/trajectory from among plurality of streams to navigate the vehicle based on the probability distribution of different trajectories and based on each of the plurality of actors, as taught by Amini, with a reasonable expectation of success, in order to allow the model to capture multiple possible outputs and providing broader information for safe decision-making. This probabilistic output enables the autonomous vehicle to select navigation strategies that balance, safety and efficiency under uncertainty.
Regarding claim 3, although Gochev discloses wherein the one or more reference labels comprise an associated reference label, for each of the plurality of disparate deciders, that corresponds to a ground truth decision that is determined during the past episode of locomotion of the vehicle or that is defined for the vehicle subsequent to the past episode of locomotion of the vehicle (Para [0036], “the machine-learned vehicle action can be trained using supervised training techniques based on training data.”, “These labels can be utilized as ground-truth data to determine the accuracy and/or development of the vehicle action model as it is trained.”, Para [0068]-[0071]).
However, Gochev doesn’t explicitly discloses that the one or more reference labels corresponds to a ground truth probability for a decision that is determined…
Nevertheless, Omari teaches the one or more reference labels comprise an associated reference label, for each of the plurality of disparate deciders, that corresponds to a ground truth probability for a decision that is determined during the past episode of locomotion of the vehicle or that is defined for the vehicle subsequent to the past episode of locomotion of the vehicle (Para [0025]-[0027], [0032], “ground-truth probability distribution.”, “generate data corresponding to an inferred ground truth. In the context of training of prioritization model 505, the ground truth serves as the target or desired output for the associated training sample and may be referred to as the ground truth label for that training sample.”, “Based on the prerecorded sensor data indicating the locations of all the agents at time t.sub.1, the ground-truth for determining the accuracy of the prediction may be generated.”, [0033], Figs. 3-5).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method as taught by Gochev in view of Amini, to include the ground truth probability distribution as taught by Omari, with a reasonable expectation of success, in order to represent the real world uncertainty in the machine learning model which is used to approximate the true, real-world probability distribution of outcomes.
Regarding claim 4, Gochev discloses wherein generating one or more of the losses comprises comparing the associated predicted decision made by each of the plurality of disparate deciders to the ground truth decision, for each of the plurality of deciders, to generate one or more of the losses; and wherein updating the one or more additional ML model layers comprises backpropagating one or more of the losses across the one or more additional ML model layers (Para [0036], “the machine-learned vehicle action can be trained using supervised training techniques based on training data.”, “These labels can be utilized as ground-truth data to determine the accuracy and/or development of the vehicle action model as it is trained.”, Para [0078] and Fig. 2B, “The model trainer 254 can evaluate a training output 256 of the vehicle action model 138 to determine the accuracy and/or confidence level of the model as it is trained over time. The model trainer 254 can continue to train the vehicle action model 138 until a sufficient level of accuracy and/or confidence is achieved.”, __determining the accuracy and/or confidence level of the model as it is trained reads on generating losses__).
However, Gochev doesn’t explicitly discloses generating one or more of the losses comprises comparing one or more of the associated probability for the predicted decision made by each of the plurality of disparate deciders to one or more of the ground truth probability.
Nevertheless, Omari teaches generating one or more of the losses comprises comparing one or more of the associated predicted probability for the predicted decision made by each of the plurality of disparate deciders to one or more ground truth probability for the decision, for each of the plurality of deciders, to generate one or more of the losses ( at least [0022], [0025], “ground-truth for determining the accuracy of the prediction may be generated.”, [0026]-[0027], [0032], “compare the predicted probability distribution generated by prioritization model 505 with a ground-truth probability distribution”, [0034], “the predicted contextual representation may then be compared to the known second contextual representation (i.e., the ground-truth at time t.sub.1). The comparison may be quantified by a loss value or computed using a loss function.”, [0046]);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method as taught by Gochev in view of Amini, to include generating the losses by comparing the predicted probability distribution to the ground truth probability distribution as taught by Omari, with a reasonable expectation of success, in order to evaluate/measure the accuracy of the model in capturing uncertainty and making better probabilistic prediction.
Regarding claim 6, Gochev discloses wherein the one or more reference labels comprise an associated reference label, for each of the plurality of disparate deciders, that corresponds to a ground truth that is determined during on the past episode of locomotion of the vehicle or that is defined for the vehicle subsequent to the past episode of locomotion of the vehicle (See rejection of claim 3).
Gochev in view of Amini doesn’t disclose one or more reference labels comprise an associated reference label, for each of the plurality of disparate deciders, that corresponds to a ground truth probability distribution.
However, Omari teaches disclose one or more reference labels comprise an associated reference label, for each of the plurality of disparate deciders, that corresponds to a ground truth probability distribution (at least Para [0020], [0032], “In the context of training of prioritization model 505, the ground truth serves as the target or desired output for the associated training sample and may be referred to as the ground truth label for that training sample.”, “compare the predicted probability distribution generated by prioritization model 505 with a ground-truth probability distribution.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method as taught by Gochev in view of Amini, to include the ground truth probability distribution as taught by Omari, with a reasonable expectation of success, in order to represent the real world uncertainty in the machine learning model which is used to approximate the true, real-world probability distribution of outcomes.
Regarding claim 7, Gochev discloses generating one or more of the losses for each of the plurality of deciders; (Para [0036], “the machine-learned vehicle action can be trained using supervised training techniques based on training data.”, “These labels can be utilized as ground-truth data to determine the accuracy and/or development of the vehicle action model as it is trained.”, Para [0078] and Fig. 2B, “The model trainer 254 can evaluate a training output 256 of the vehicle action model 138 to determine the accuracy and/or confidence level of the model as it is trained over time. The model trainer 254 can continue to train the vehicle action model 138 until a sufficient level of accuracy and/or confidence is achieved.”), and wherein updating the one or more additional ML model layers comprises backpropagating one or more of the losses across the one or more additional ML model layers. (Para [0115])
Gochev doesn’t explicitly disclose generating one or more of the losses comprises comparing the associated predicted probability distribution to the ground truth probability distribution, for each of the plurality of deciders, to generate one or more of the losses;
However, Omari teaches generating one or more of the losses comprises comparing the associated predicted probability distribution to the ground truth probability distribution, for each of the plurality of deciders, to generate one or more of the losses ([0025], “ground-truth for determining the accuracy of the prediction may be generated.”, [0026]-[0027], [0032], “compare the predicted probability distribution generated by prioritization model 505 with a ground-truth probability distribution”, [0034], “the predicted contextual representation may then be compared to the known second contextual representation (i.e., the ground-truth at time t.sub.1). The comparison may be quantified by a loss value or computed using a loss function.”, [0046]);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the machine learning model used in the system and method as taught by Gochev in view of Amini, to include generating the losses by comparing the predicted probability distribution to the ground truth probability distribution as taught by Omari, with a reasonable expectation of success, in order to evaluate/measure the accuracy of the model in capturing uncertainty and making better probabilistic prediction.
Documents Considered but not Relied Upon
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Mehta et al., US 20200207375 A1 (See at least, for example, Abstract, [0003], Fig. 10)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAJAR HASSANIARDEKANI whose telephone number is (571)272-1448. The examiner can normally be reached Monday thru Friday 8 am-5 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Piateski can be reached at 5712707429. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.H./Examiner, Art Unit 3669
/Erin M Piateski/Supervisory Patent Examiner, Art Unit 3669
Prosecution Insights