DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This is a Final Action on the Merits. Claims 1-21 are currently pending and are addressed below.
Election/Restrictions
Applicant’s election without traverse of Group I (Claims 1-7 and 21) in the reply filed on January 30th, 2026 in response to the Office Action dated December 2nd, 2025 is acknowledged. Claims 8-20 are currently withdrawn and Claims 1-7 and 21 are currently examined below.
Response to Amendments
The Amendment filed on January 30th, 2026 has been considered and entered. Accordingly, claims 1, 3-4, 8, 10, 15, and 17 have been amended. Claim 21 has been newly added.
Response to Arguments
The Applicant’s Arguments with respect to claims 1-7 and 21 have been considered but are moot in view of the newly formulated grounds of rejections necessitated by the applicant’s amendments, however at least on pertinent argument remains.
The Applicant states (Amend. Pages 7-9 dated August 18th, 2025) that the combination of Zeng (US 20210149404 A1) (“Zeng”) and Brown (US 20200117958 A1) (“Brown”) “represents a non-obvious integration of disparate technical approaches”. The examiner respectfully disagrees. Zeng teaches game theoretic path planning for an agent based on identified dynamic agents, while Brown teaches a determination of a joint state of the dynamic agents in order to determine a path for the ego agent. Zeng and Brown are both in the same field of endeavor as the instant application, path planning for navigation of an ego agent, such that one of ordinary skill in the art would combine the teachings of Zeng with the teachings of Brown.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Zeng (US 20210149404 A1) (“Zeng”) in view of Brown (US 20200117958 A1) (“Brown”) in view of Olson (US 20180268281 A1) (“Olson”).
With respect to claim 1, Zeng teaches a computer-implemented method for game theoric path planning for social navigation, the method comprising:
identifying a set of dynamic agents in an agent environment based on sensor data from one or more agent sensors of an ego agent (See at least Zeng FIG. 5 and Paragraph 124 “The structured machine-learned model (e.g., structured machine-learned model 200 in FIG. 2) can detect, at 504, one or more objects in the area around the autonomous vehicle (e.g., autonomous vehicle 102 in FIG. 1) based at least in part on the sensor data. The objects can include actors such as vehicles, pedestrians, bicyclists, and any other moving objects. The detected objects can be included in a first intermediate representation that includes data describing one or more features of the object including the location of the object, the size of the object, the current velocity, heading, etc.”);
determining preference distributions for each dynamic agent of the set of dynamic agents, wherein a preference distribution for a dynamic agent of the set of dynamic agents is a probability distribution of a set of candidate trajectories to a goal state that do not account for agent interference of other dynamic agents of the set of dynamic agents (See at least Zeng FIG. 5 and Paragraphs 125-128 “The structured machine-learned model (e.g., structured machine-learned model 200 in FIG. 2) can determine, at 506, a plurality of candidate object trajectories for each object in the one or more objects. Each trajectory can be represented as a series of coordinates for a series of time steps (e.g., 1 second, 2 seconds, 3 seconds, and so on). For each respective object in the one or more objects, the structured machine-learned model (e.g., structured machine-learned model 200 in FIG. 2) can generate likelihood data for each candidate object trajectory in the plurality of candidate object trajectories. The structured machine-learned model (e.g., structured machine-learned model 200 in FIG. 2) can, for each object in the one or more objects, determine feature data for the object. The structured machine-learned model (e.g., structured machine-learned model 200 in FIG. 2) can determine trajectory feature data for each candidate trajectory. The plurality of candidate object trajectories can be used as input to one section of the structured machine-learned model. Using the plurality of candidate object trajectories as input, the structured machine-learned model (e.g., structured machine-learned model 200 in FIG. 2) can generate as output, at 508 likelihood data for each candidate object trajectory in the plurality of candidate object trajectories. This represents an inference step in the structured machine-learned model (e.g., using candidate object trajectories as input to the model and receiving likelihood data as output of the model). The likelihood data can include a likelihood distribution for the plurality of candidate object trajectories. In some examples, the likelihood data can involve discrete likelihood values for each candidate object trajectory. The object feature data can represent data determined based on region of interest pooling. Trajectory feature data associated with an object or actor can include the coordinates of the object at a sequence of timestamps. This information can represent the path of the trajectory over time … The likelihood data for all the one or more objects can be used as input to the structured machine-learned model. Using the likelihood data as input, the structured machine-learned model (e.g., structured machine-learned model 200 in FIG. 2) can determine, at 510, the updated likelihood data for the plurality of candidate object trajectories for each respective object in the one or more objects based on the likelihood data associated with candidate object trajectories for other objects in the one or more objects. The structured machine-learned model (e.g., structured machine-learned model 200 in FIG. 2) can generate updated likelihood data at least in part based on a message-passing stage in which the likelihood data for a respective plurality of candidate object trajectories associated with the respective object are compared to likely trajectories of one or more other candidates”).
Zeng, however, fails to explicitly disclose determining a joint state for the dynamic agents of the set of dynamic agents by applying a recursive model iteratively to each individual dynamic agent to calculate a trajectory likelihood based on an expected interference risk of a candidate trajectory and the preference distributions, wherein the joint state for the set of dynamic agents minimizes deviations from the goal state for each dynamic agent and minimizes the expected interference risk, and wherein the recursive model converges based on a joint probability threshold of agent interference; and causing the ego agent to execute a path plan based on the joint state for the dynamic agents.
Brown teaches determining a joint state for the dynamic agents of the set of dynamic agents by applying a recursive model iteratively to each individual dynamic agent to calculate a trajectory likelihood based on an expected interference risk of a candidate trajectory and the preference distributions, wherein the joint state for the set of dynamic agents minimizes deviations from the goal state for each dynamic agent and minimizes the expected interference risk (See at least Brown FIG. 10 and Paragraphs 101-106 “In an optional configuration, at block 1004, a reasoning level and a set of forward prediction models are assigned to each agent in a scene. The forward prediction models may be referred to as atomic models. The reasoning level of an agent may be an integer greater than or equal to zero. Each forward prediction model of an agent's assigned set of forward prediction models corresponds to a specific reasoning level greater than or equal to zero and less than or equal to the agent's assigned reasoning level. A given agent may have no more than one assigned atomic prediction model for each level … In an optional configuration, at block 1006, the prediction model partitions the scene into different neighborhoods. Each neighborhood may be assigned a different fidelity. The fidelity may be based on a proximity to the ego agent. For example, a high fidelity neighborhood may be centered on the ego agent. The fidelity of the agents may be based on a fidelity of a corresponding neighborhood. At block 1008, the prediction model recursively predicts future actions of the agents by traversing the scene. For example, for each agent that has an assigned level 0 forward prediction model, the recursive reasoning scheme generates a level 0 motion hypothesis using the assigned level 0 forward prediction model. Then, for each agent that has an assigned level 1 forward prediction model, the prediction model generates a level 1 motion hypothesis using the assigned level 1 forward prediction model … In one configuration, the future actions are recursively predicted based on an initial trajectory comprising historical observations of each agent. That is, an input to the prediction model may be a representation of the history of a scene. In another configuration, the future actions are recursively predicted based on an applicable policy for each agent. The policy may be based on a corresponding neighborhood of the agent and a scene structure. The scene structure may refer to the roadway geometry …”); and
causing the ego agent to execute a path plan based on the joint state for the dynamic agents (See at least Brown FIG. 10 and Paragraph 107 “Finally, at block 1010, the prediction model controls an action of an ego agent based on the predicted future actions of the agents. For example, the prediction model may alter a route, adjust a speed, or control another action. The prediction model may be a component of the ego agent.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Zeng to include determining a joint state for the dynamic agents of the set of dynamic agents by applying a recursive model iteratively to each individual dynamic agent to calculate a trajectory likelihood based on an expected interference risk of a candidate trajectory and the preference distributions, wherein the joint state for the set of dynamic agents minimizes deviations from the goal state for each dynamic agent and minimizes the expected interference risk; and causing the ego agent to execute a path plan based on the joint state for the dynamic agents, as taught by Brown as disclosed above, in order to ensure safe movement of the ego vehicle in various dynamic environments (Brown Paragraph 30 “Aspects of the present disclosure are directed to improving behavior prediction by combining a multi-fidelity framework into a recursive reasoning scheme”).
Zeng in view of Brown fail to explicitly disclose that the recursive model converges based on a joint probability threshold of agent interference.
Olson teaches that the recursive model converges based on a joint probability threshold of agent interference (See at least Olson FIG. 3 and Paragraph 11 “Simulating movement of the one or more monitored objects, includes representing trajectory of an object using a differentiable function. In one embodiment, the trajectory of an object is presented by recursively applying a transition function over a series of time steps, where the transition function is defined such that objects are repelled by other agents and attracted towards a goal in accordance with a social force model. In addition, perturbed seed states are determined by iteratively computing gradient for each time step in the series of time steps with respect to the perturbed seed states.” | Paragraphs 82-85 “Referring now to FIG. 3, a flowchart of an embodiment of the MPDM apparatus is illustrated. The controlled object 100 is always perceiving the states (e.g., locations, speed, etc.) of all the monitored objects in the environment and determining a trajectory based on the possible outcomes due to the initial configurations. To determine the trajectory, the controlled object 100 evaluates each of the policies 136. First control chooses a policy to evaluate 300 from all of the potential policies 136. Next, at step 304, control receives state data for each of the monitored objects 208, 212. The state data is obtained by the perception module 112. Seed states for each of the monitored objects 208, 212 are generated at 308 using the seed state generator 116. The seed states correspond to initial configurations that the simulations use as input. The simulator 120 then simulates using the chosen policy and the seed states at 312. At step 316, the outcome is quantified as the cost multiplied by the probability using the outcome quantifier 128. As discussed above, the cost function is calculated as a combination of blame for disturbing objects in the environment as well as progress towards the target 216. The probability is determined from the probability distributions 132. At step 320, control determines whether a condition has been met. The condition may be a variety of predetermined conditions, such as an amount of time, a number of perturbations or iterations, a policy-specific condition depending on which policy is currently being implemented by the controlled object 100, or any other condition that has been predetermined. In other words, step 320 controls the number of times the seed states are perturbed and simulated to determine which policy results in the most benign high-cost events. If the condition has not been met, control continues to step 324 where the seed states are perturbed. That is, the elements of the seed states (i.e., position, speed, etc.) are perturbed iteratively towards increasingly influential outcomes using a method such as backpropagation. This perturbation directs the perturbed seed state configurations towards outcomes that result in high-cost events. Then, at 328, the simulator 120 simulates using the chosen policy and the perturbed seed states. The outcome quantifier 128 then quantifies a perturbed outcome as the product of the perturbed cost and the perturbed probability at 332. The perturbed cost and perturbed probability are determined based on the perturbed seed states. The seed states are perturbed iteratively towards increasingly influential outcomes until the condition has been met. At that point, control determines whether each policy 136 has been simulated 336. If not, control returns to the beginning to select a different policy at 200. Once control determines that each policy has been simulated at 336, then scores are determined for each policy at 340. The policy with the best score is selected at 344. The best score can be described as the score indicating the fewest number of the most benign high-cost events. This ensures the best possible scenario for the controlled object 100. That is, the controlled object 100 selects the policy whose population of discovered likely high-cost outcomes is more benign or desirable. For example, a more desirable policy includes a lower chance that the selected policy will result in a collision or a near miss, etc. Once the policy is selected, control issues a command associated with the policy to the controlled object 100 at 348. As mentioned previously, the commands may be a command to accelerate, decelerate, etc. Control conducts the MPDM shown in the flowchart in real-time to determine, at all points in time, which policy 136 is best for the controlled object 100 to follow.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Zeng in view of Brown to include that the recursive model converges based on a joint probability threshold of agent interference, as taught by Olson as disclosed above, in order to ensure an accurate trajectory likelihood for the dynamic agents (Olson Paragraph 3 “The present disclosure relates to a method for multi-policy decision making of an object moving through an environment.”).
With respect to claim 2, Zeng in view of Brown in view of Olson teaches that the set of dynamic agents includes the ego agent and a number of biological entities (See at least Zeng FIG. 5 and Paragraph 124 “The structured machine-learned model (e.g., structured machine-learned model 200 in FIG. 2) can detect, at 504, one or more objects in the area around the autonomous vehicle (e.g., autonomous vehicle 102 in FIG. 1) based at least in part on the sensor data. The objects can include actors such as vehicles, pedestrians, bicyclists, and any other moving objects. The detected objects can be included in a first intermediate representation that includes data describing one or more features of the object including the location of the object, the size of the object, the current velocity, heading, etc.”).
With respect to claim 21, Zeng in view of Brown in view of Olson teach that the recursive model applies Bayesian belief updating until the joint probability threshold is satisfied (See at least Olson Paragraph 52 “Further details for an example embodiment of the MPDM system are set forth. In this embodiment, non-holonomic motion models are used for each observed agent i as well as for the robot. The robot maintains a probabilistic estimate of each observed agents' state—i.e. its position, velocity, angular velocity and inferred policy. An agent's policy πi=(Vdes,gsub), expresses an intent to move towards sub-goal gsub at a desired speed vdes. The collective state xt∈X consists of the states of the robot and all observed agents at time t. Throughout the disclosure, x0 is referred to as the collective state of all agents and the robot's state at the current time. The probabilistic estimate P(x0) is based on past observations of the pedestrians' positions. Several methods can be used for P(x0) based on past trajectories of agents. In the example embodiment, a Kalman Filter is used to infer position and velocity; whereas, a Naïve Bayes Classifier is used to infer an agent's policy. The robot's policy π is elected from amongst a set of closed-loop policies π.”).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Zeng (US 20210149404 A1) (“Zeng”) in view of Brown (US 20200117958 A1) (“Brown”) in view of Olson (US 20180268281 A1) (“Olson”) further in view of Li (US 20190235505 A1) (“Li”).
With respect to claim 3, Zeng in view of Brown in view of Olson teach a determination of the expected interference risk and the preference distribution (See at least Brown FIG. 10 and Paragraphs 101-106).
Zeng in view of Brown in view of Olson, however, fail to explicitly disclose that the trajectory likelihood is an inverse exponential function of the expected interference risk and the preference distributions.
Li teaches that the trajectory likelihood is an inverse exponential (See at least Li FIG. 6 and Paragraphs 51-54 “FIG. 6 is a flow diagram illustrating a method to control an ADV according to one embodiment. Processing 600 may be performed by processing logic which may include software, hardware, or a combination thereof. For example, process 600 may be performed by sampling module 307 of FIGS. 3A-3B. Referring to FIG. 6, at block 601, processing logic receives a first set of reference points based on a map and a route information, the first set of reference points representing a reference line in which the ADV is to follow. At block 602, processing logic selects a second set of reference points along the reference line corresponding to the first set of reference points, including iteratively performing, at block 603, selecting a current reference point from the first set of reference points. At block 604, determining a sampling distance along the first set of reference points based on the currently selected reference point using a nonlinear algorithm. At block 605, selecting a next reference point along the first set of reference points based on the determined sampling distance such that a density of the second set of reference points close to the ADV is higher than a density of the selected reference points farther away from the ADV. At block 606, processing logic plans a trajectory for the ADV using the second set of reference points to control the ADV … In one embodiment, the nonlinear algorithm is an inverse of an exponential function”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Zeng in view of Brown in view of Olson to include that the trajectory likelihood is an inverse exponential, as taught by Li as disclosed above, such that trajectory likelihood is an inverse exponential function of the expected interference risk and the preference distributions, in order to ensure an accurate trajectory likelihood is determined (Li Paragraph 1 “Embodiments of the present disclosure relate generally to operating autonomous vehicles. More particularly, embodiments of the disclosure relate to dynamically adjusted reference line sampling point density for autonomous driving vehicles (ADVs).”).
Claims 4 and 7 are rejected under 35 U.S.C. 103 as being unpatentable over Zeng (US 20210149404 A1) (“Zeng”) in view of Brown (US 20200117958 A1) (“Brown”) in view of Olson (US 20180268281 A1) (“Olson”) further in view of Wyffels (US 20180203446 A1) (“Wyffels”) further in view of Subramanian (US 20200279136 A1) (“Subramanian”).
With respect to claim 4, Zeng in view of Brown in view of Olson fail to explicitly disclose that the recursive model is applied iteratively until a convergence criterion is satisfied, and wherein the convergence criterion is a joint probability threshold of agent interference that ensures stable Nash equilibrium solutions.
Wyffels, however, teaches that the recursive model is applied iteratively until a convergence criterion is satisfied, and wherein the convergence criterion is a joint probability threshold of agent interference (See at least Wyffels FIG. 7 and Paragraphs 40-42 “Process 700 begins at step 702 where computing device 115 determines Bayesian probabilities conditioned on previously determined objects. Expressed as random variables, latent model parameters Z (object parameters) and observed variables X (enclosing rectangles) are sets of N independent, identically distributed variables, wherein Z={z1 . . . zn} and X={x1 . . . ,xn}. Bayesian probabilities can the expressed as the posterior distribution p(Z|X), which is the probability that 3D data points 402, 404, 406, 408, 410 are associated with traffic objects 206, 208, 210, 502, 504, conditioned on model evidence p(X) based on traffic objects 206, 208, 210, and boundaries 308, 310, 312, 308, 310, 312 as discussed above in relation to FIGS. 3, 4 and 5 … At step 706 computing device 115 solves equation 1 by Variational Bayesian Inference by recursively fitting equation 1 to the tractable subset q(Z) based on partial derivatives to find optimal solutions q*j(Z1) for the joint Bayesian probability distributions p(X, Z). The tractable subset q(Z) can be determined to complete model fitting in a small, finite number of steps. At step 708 computing device 115 can determine that model fitting is done. Model fitting is done when random variable Z changes less than a predetermined amount from the previous step, for example. At step 710 computing device 115 can output random variable Z which includes probability distributions of object parameters associated with, for example, traffic objects 206, 208, 210, 502, 504. The process 700 ends following the block 704.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Zeng in view of Brown in view of Olson to include that the recursive model is applied iteratively until a convergence criterion is satisfied, and wherein the convergence criterion is a joint probability threshold of agent interference, as taught by Wyffels as disclosed above, in order to ensure accurate trajectories are determined (Wyffels Paragraph 43 “The proposed approach advantageously leverages this prior information to seed/initialize the clustering routine in a rigorous Bayesian probabilistic framework”).
Zeng in view of Brown in view of Olson in view of Wyffles fails to explicitly disclose ensuring stable Nash equilibrium solutions.
Subramanian teaches ensuring stable Nash equilibrium solutions (See at least Subramanian Paragraphs 25-28 “In some embodiments, the type of each of the other agents is defined. In some embodiments, the type of each of the other agents is initially unknown. In some embodiments, the method further comprises performing k-means clustering to approximate a type for the initially unknown types. In some embodiments, the recursively updating is performed to converge to a fixed point within a bounded distance of a Nash equilibrium.” | Paragraph 70 “In Multi Agent Reinforcement Learning (MARL), there is a notion of stochastic games [3], where the state and action space are incorporated as Cartesian products of individual states and actions of different agents in the environment. A stochastic game can be considered to be a special type of normal form game [9], where a particular iteration of the game depends on previous game(s) played and the experiences of all the agents in the previous game(s). A stochastic game can be defined as a tuple <S, N, A, P, R> where S is a finite set of states (assumed to be the same for all agents), N is a finite set of n agents, A=A1× . . . ×An where Aj denotes the actions of agent j. P is the transition probability function P(s′|s, a) where a=(a1, . . . , an) and Rj(s,a)=rj is the reward function with rj denoting the reward received by agent j. Each agent is trying to learn a policy that maximizes its return upon consideration of opponent behaviours. Agents are typically self-interested and the combined system moves towards a Nash equilibrium [11]. Typically MARL is constrained by a problem of scale—scalability in MARL environments is often a bottleneck. Many research efforts are aimed at handling only up to a handful of agents and the solutions or approaches considered become intractable in large agent scenarios.” | Paragraph 114 “By iterating through equations (21), (22) and (23), the mean actions and respective policies of all agents may keep improving. As shown in the proof for Theorem 3.2 this approach converges to a fixed point within a small bounded distance of the Nash equilibrium.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Zeng in view of Brown in view of Olson in view of Wyffles to include ensuring stable Nash equilibrium solutions, as taught by Subramanian as disclosed above, in order to ensure accurate path planning of the dynamic agents (Subramanian Paragraph 2 “Embodiments of the present disclosure generally relate to the field of machine learning, and more specifically, embodiments relate to devices, systems and methods for multi-agent reinforcement machine learning.”).
With respect to claim to claim 7 Zeng in view of Brown in view of Olson fail to explicitly disclose that the recursive model is based on a Bayes' Rule for conditional probability, and wherein the joint state is a Nash equilibrium.
Wyffels teaches that the recursive model is based on Bayes’ Rule for conditional probability (See at least Wyffels FIG. 7 and Paragraphs 40-42 “Process 700 begins at step 702 where computing device 115 determines Bayesian probabilities conditioned on previously determined objects. Expressed as random variables, latent model parameters Z (object parameters) and observed variables X (enclosing rectangles) are sets of N independent, identically distributed variables, wherein Z={z1 . . . zn} and X={x1 . . . ,xn}. Bayesian probabilities can the expressed as the posterior distribution p(Z|X), which is the probability that 3D data points 402, 404, 406, 408, 410 are associated with traffic objects 206, 208, 210, 502, 504, conditioned on model evidence p(X) based on traffic objects 206, 208, 210, and boundaries 308, 310, 312, 308, 310, 312 as discussed above in relation to FIGS. 3, 4 and 5 … At step 706 computing device 115 solves equation 1 by Variational Bayesian Inference by recursively fitting equation 1 to the tractable subset q(Z) based on partial derivatives to find optimal solutions q*j(Z1) for the joint Bayesian probability distributions p(X, Z). The tractable subset q(Z) can be determined to complete model fitting in a small, finite number of steps. At step 708 computing device 115 can determine that model fitting is done. Model fitting is done when random variable Z changes less than a predetermined amount from the previous step, for example. At step 710 computing device 115 can output random variable Z which includes probability distributions of object parameters associated with, for example, traffic objects 206, 208, 210, 502, 504. The process 700 ends following the block 704.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Zeng in view of Brown in view of Olson to include that the recursive model is based on Bayes’ Rule for conditional probability, as taught by Wyffels as disclosed above, in order to ensure accurate trajectories are determined (Wyffels Paragraph 43 “The proposed approach advantageously leverages this prior information to seed/initialize the clustering routine in a rigorous Bayesian probabilistic framework”).
Zeng in view of Brown in view of Olson in view of Wyffles fails to explicitly disclose that the joint state is a Nash equilibrium.
Subramanian teaches that the joint state is a Nash equilibrium (See at least Subramanian Paragraphs 25-28 “In some embodiments, the type of each of the other agents is defined. In some embodiments, the type of each of the other agents is initially unknown. In some embodiments, the method further comprises performing k-means clustering to approximate a type for the initially unknown types. In some embodiments, the recursively updating is performed to converge to a fixed point within a bounded distance of a Nash equilibrium.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Zeng in view of Brown in view of Olson in view of Wyffles to include that the joint state is a Nash equilibrium, as taught by Subramanian as disclosed above, in order to ensure accurate path planning of the dynamic agents (Subramanian Paragraph 2 “Embodiments of the present disclosure generally relate to the field of machine learning, and more specifically, embodiments relate to devices, systems and methods for multi-agent reinforcement machine learning.”).
Claims 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Zeng (US 20210149404 A1) (“Zeng”) in view of Brown (US 20200117958 A1) (“Brown”) in view of Olson (US 20180268281 A1) (“Olson”) further in view of Engstrom (US 20240051582 A1) (“Engstrom”).
With respect to claim 5, Zeng in view of Brown in view of Olson teach that the trajectory likelihood is calculated for each candidate trajectory of the preference distribution for each dynamic agent (See at least Brown FIG. 10 and Paragraphs 101-106) and defines costs for unilateral movements for a corresponding agent of the set of dynamic agents (See at least Brown FIGS. 4A-5A and Paragraphs 69-72 “As shown in FIG. 4B, after observing the scene 400, the prediction model assigns a reasoning level and a set of atomic prediction models to each of the observed agents 404. In one configuration, agents 406 adjacent to the ego agent 402 are assigned a higher fidelity level in comparison to the other agents 404. The adjacent agents 406 are a subset of the observed agents 404. The adjacent agents 406 are in a first pattern around the ego agent 402. The ego agent 402 is in a second pattern. Additionally, an agent 408 that is in front of other agents 404, 406 in a same lane 410 as the ego agent 402 may be assigned the higher fidelity level because actions of the other agents 404, 406 may be dependent on actions of the front agent 408. The front agent 408 is a subset of observed agents 404. As shown in FIG. 4C, after assigning the fidelity levels, the prediction model generates an interaction graph to encode relationships between agents based on the structure of the environment. The interaction graph may also be generated before assigning fidelity levels. Directed edges (not shown in FIG. 4C) encode a direction of influence between two agents. A connection 412 between agents 404, 406, 408 identifies a constraint between two agents. In a directed graph, each edge includes a direction depicted with an arrow-tip (not shown in FIG. 4C). In the case of bi-directional relationships, the edge should have an arrow tip on both ends (not shown in FIG. 4C). That is, if the actions of a first agent 426 have an effect on a second agent 424 the edge should point from the first agent 426 to the second agent 424. The successive subsets of level 0, 1, . . . , K behavior predictions are selected by traversing the graph from highest to lowest priority … FIG. 5A illustrates an example of two distinct level 0 scenarios according to aspects of the present disclosure. As shown in FIG. 5A, scenarios 500A, 500B are generated for a first set of agents 510 (e.g., vehicles on a road). The first set of agents 510 includes four agents 502, 504, 506, 508. A fidelity level of a second agent 504 of the set of agents 510 is higher than a fidelity level of other agents 502, 506, 508 in the set of agents 510. An arrow 520 identifies a predicted movement of each agent 502, 504, 506, 508.”).
Zeng in view of Brown in view of Olson fail to explicitly disclose that the trajectory likelihoods for each candidate trajectory define a probability distribution of posterior beliefs of the corresponding agent.
Engstrom teaches that the trajectory likelihoods for each candidate trajectory define a probability distribution of posterior beliefs of the corresponding agent (See at least Engstrom FIG. 5 and Paragraph 64-65 “The system can perform the process 500 either online to compute a live response time or offline to assess AV performance. In particular, the system can receive a request to compute a predicted response time for the agent (step 510). At each time step during a sequence of time steps within a given traffic scenario trajectory, the system can obtain a distribution of previously predicted trajectories at previous time steps for another entity vehicle with a generative model (step 520). For example, in the case of the other entity vehicle running a stop sign at a four-way stop intersection in front of the agent, the generated distribution of trajectories might reflect the expected behavior that the other entity vehicle yield at the stop sign. The system can then use the expected trajectories to compute a measure of surprise by comparing the updated state of the entity with the distribution of expected movement (step 530). As explained in FIGS. 3A and 3B, the discrepancy between the updated state of the other entity with the distribution of expected movements generates a measure of surprise that is accumulated at each time step (step 540). In the case of the other entity vehicle running the stop sign, the surprise accumulates from the moment of the first expected slowdown when the vehicle fails to decelerate to when the vehicle broaches the intersection causing an impending collision. Upon determining that the accumulated measure of surprise crosses a threshold corresponding to the evasive maneuver onset (step 550), the system generates a predicted response time for the agent (step 560).” | Paragraphs 36-37 “FIG. 3A is a diagram of a framework that enables the modeling of agent response times in real-world scenarios given the need to define onset and end times for indefinite stimuli and this situation-dependency. The framework conceptualizes an agent's response to a traffic scenario as an inference process, specifically an inference process where the agent's behavior is guided by an initial hypothesis or prior belief that is consistent with the agent's expectations about the causes of a sensory input, such as what another entity vehicle is doing on the road. As the situation evolves, the sensory input is updated and continually processed into an alternative hypothesis or posterior belief. If the sensory input remains consistent with the agent's initial belief, the posterior remains the same as the prior. However, if the sensory input becomes inconsistent with the agent's initial belief, it generates a prediction error as the inconsistency grows, and the posterior changes accordingly to reflect a measure of surprise. For example, within the context of the collision scenario depicted in FIG. 2 , the agent's prior belief whether vehicle B will continue ahead or brake is represented as probability distribution P(B). Initially, the belief that vehicle B will stop at the intersection P(B1), the expected behavior distribution 300, dominates over the belief that vehicle B will not stop at the intersection P(B2), the surprising behavior distribution 310, in P(B). However, the agent 200 driver notices that something is awry when vehicle B 210 continues to approach the intersection at constant speed. The agent driver 200 notes that vehicle B 210 not slowing down is inconsistent with their expectation for how the traffic scenario interaction should go, which generates a prediction error between beliefs that is quantified as a measure of surprise. This surprise accumulates over time over the course of the traffic conflict scenario. More specifically, the surprise drives an update of the prior belief distribution to a posterior belief dominated by the alternative belief P(B2) 310 as vehicle B 210 continues ahead into the intersection.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of Zeng in view of Brown in view of Olson to include that the trajectory likelihoods for each candidate trajectory define a probability distribution of posterior beliefs of the corresponding agent, as taught by Engstrom as disclosed above, in order to ensure accurate determination of the candidate trajectories (Engstrom Paragraph 38 “In particular, the framework specifies that the agent's behavior is guided by prior beliefs that are continuously updated to posterior beliefs based on the accumulation of surprising evidence over time. This belief updating can be represented as a process of evidence accumulation where the strength of the belief grows over time as more evidence comes in towards a minimum onset decision threshold 320”).
With respect to claim 6, Zeng in view of Brown in view of Olson in view of Engstrom teach that the costs are minimized for the unilateral movement in the joint state (See at least Brown FIGS. 4A-5A and Paragraphs 69-72 “As shown in FIG. 4B, after observing the scene 400, the prediction model assigns a reasoning level and a set of atomic prediction models to each of the observed agents 404. In one configuration, agents 406 adjacent to the ego agent 402 are assigned a higher fidelity level in comparison to the other agents 404. The adjacent agents 406 are a subset of the observed agents 404. The adjacent agents 406 are in a first pattern around the ego agent 402. The ego agent 402 is in a second pattern. Additionally, an agent 408 that is in front of other agents 404, 406 in a same lane 410 as the ego agent 402 may be assigned the higher fidelity level because actions of the other agents 404, 406 may be dependent on actions of the front agent 408. The front agent 408 is a subset of observed agents 404. As shown in FIG. 4C, after assigning the fidelity levels, the prediction model generates an interaction graph to encode relationships between agents based on the structure of the environment. The interaction graph may also be generated before assigning fidelity levels. Directed edges (not shown in FIG. 4C) encode a direction of influence between two agents. A connection 412 between agents 404, 406, 408 identifies a constraint between two agents. In a directed graph, each edge includes a direction depicted with an arrow-tip (not shown in FIG. 4C). In the case of bi-directional relationships, the edge should have an arrow tip on both ends (not shown in FIG. 4C). That is, if the actions of a first agent 426 have an effect on a second agent 424 the edge should point from the first agent 426 to the second agent 424. The successive subsets of level 0, 1, . . . , K behavior predictions are selected by traversing the graph from highest to lowest priority … FIG. 5A illustrates an example of two distinct level 0 scenarios according to aspects of the present disclosure. As shown in FIG. 5A, scenarios 500A, 500B are generated for a first set of agents 510 (e.g., vehicles on a road). The first set of agents 510 includes four agents 502, 504, 506, 508. A fidelity level of a second agent 504 of the set of agents 510 is higher than a fidelity level of other agents 502, 506, 508 in the set of agents 510. An arrow 520 identifies a predicted movement of each agent 502, 504, 506, 508.”).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IBRAHIM ABDOALATIF ALSOMAIRY whose telephone number is (571)272-5653. The examiner can normally be reached M-F 7:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Faris Almatrahi can be reached at 313-446-4821. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IBRAHIM ABDOALATIF ALSOMAIRY/Examiner, Art Unit 3667 /KENNETH J MALKOWSKI/Primary Examiner, Art Unit 3667
Prosecution Insights