DETAILED ACTION
This Office Action is taken in response to Applicant’s Amendment and Remarks filed on 03/04/2026 regarding Application No. 18/483,479 originally filed on 10/09/2023. Claims 1-20 are currently pending and have been considered as follows:
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant argues “Applicant has amended independent claims 1, 9, and 17 to recite, in part, predicting modes of trajectories by grouping trajectories into fewer modes based on similarities between the trajectories, where a mode represents a class of trajectories, and using the predicted modes to train a trajectory prediction model… Applicant submits that the cited references fail to teach or suggest such features as recited in amended independent claims 1, 9, and 17.” [Remarks, pp. 15-16]. The examiner respectfully disagrees.
The rejection does not rely on Huang alone for the newly added grouping limitation. Huang is relied upon for game-theoretic multimodal trajectory prediction, including predicting multimodal trajectories for agents in an autonomous driving environment. Ma is relied upon for the similarity-based grouping and mode/class limitation. Ma teaches “a similarity-clustered based technique to obtain the multi-modal ground-truth future,” where “similar initial poses may be grouped,” and “their corresponding future poses may be viewed as the pseudo-possible future motions for each initial pose in the group,” with similar poses grouped recursively to obtain shared futures (¶32). Ma further defines a human motion trajectory as a sequence of poses over a time horizon and teaches a future trajectories distribution having a dominant mode and other modes (¶37). Ma also teaches selecting a proper fixed number of future poses to capture different modes, including using a k-DPP and a similarity matrix based on distances between future pose sequences over a time horizon (¶65, ¶67-¶68). Thus, Ma teaches or suggests grouping trajectory/pose sequences into fewer modes based on similarity, where a mode represents a class or group of possible future motions.
Applicant’s reliance on the application’s example of alternate trajectories being grouped into semantically distinct modes does not distinguish the claims from the applied art. Ma similarly teaches grouping similar initial poses and sharing corresponding future poses as common ground-truth data, which may lead to additional, different modes of motion (¶129). Ma further teaches diversity loss and sampling directed to capturing different modes, including minor or rarer modes, and evaluating differences between samples from accuracy and diversity samplers (¶37, ¶55-¶56). Accordingly, the combination of Huang and Ma teaches or suggests the amended limitations of independent claims 1, 9, and 17.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 9-10, 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Huang (NPL Title: GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving, Year: 2023) in view of Ma (US Pub. No. 20230141610).
As per Claim 1, Huang discloses of game-theoretic modeling and learning, comprising:
obtaining training data including first trajectories for a first plurality of moving bodies and first map information of a first environment for a past time horizon; (as per “We consider a driving scene with N agents, where the AV is denoted as A0 and its neighboring agents as A1, · · · , AN-1 at the current time t = 0. Given the historical states of all agents (including the AV) over an observation horizon Th, S = {si−Th:0}, as well as the map information M including traffic lights and road waypoints, the goal is to jointly predict the future trajectories of neighboring agents Y over the future horizon Tf… In order to capture the uncertainty, the results are multi-modal future trajectories for the AV and neighboring agents,… where yj1:Tf is a sequence of predicted states, pj the probability of the trajectory, and M the number of modalities” in P3, 3.1 Game-theoretic formulation)
applying the training data (as per “Given the historical states of all agents (including the AV) over an observation horizon Th, S = {si−Th:0}, as well as the map information M including traffic lights and road waypoints” in P3, 3.1 Game-theoretic formulation) to a game-theoretic mode-finding algorithm to generate a mode-finding model (as per “Formally, the i-th agent’s level-k policy is set to optimize the following objective… where L(·) is the loss (or cost) function.” in P3, 3.1 Game-theoretic formulation) for each moving body that predicts modes of the first trajectories, (as per “The future behavior of an agent is modeled as a Gaussian mixture model (GMM), where each mode m at each time step t is represented by a Gaussian distribution over (x, y), characterized by a mean µmt and covariance σmt.” in P5, 3.4. Learning Process, as per “We leverage level-k game theory to model agent interactions in an iterative manner. Instead of simply predicting a single set of trajectories, we predict a hierarchy of trajectories to model the cognitive interaction process. At each reasoning level, with the exception of level-0, the decoder takes as input the prediction results from the previous level, which effectively makes them a part of the scene, and estimates the responses of agents in the current level to other agents in the previous level” in P3, 3.1 Game-theoretic formulation)
predicting future trajectories for a second plurality of moving bodies based on applying observed data to the trajectory prediction model, (as per “the goal is to jointly predict the future trajectories of neighboring agents Y over the future horizon Tf… In order to capture the uncertainty, the results are multi-modal future trajectories for the AV and neighboring agents,… where yj1:Tf is a sequence of predicted states, pj the probability of the trajectory, and M the number of modalities” in P3, 3.1 Game-theoretic formulation) wherein the observed data includes second trajectories for a second plurality of moving bodies and second map information of a second environment; (as per “The input data comprises historical state information of agents… where ds represents the number of state attributes, and local vectorized map polylines… For each agent, we find Nm nearby map elements such as routes and crosswalks, each containing Np waypoints with dp attributes. The inputs are normalized according to the state of the ego agent…” in P4, 3.2. Scene Encoding, as per “At the final level of interaction decoding, we can obtain multi-modal trajectories for the AV and neighboring agents” in P5, 3.3 Future Decoding with Level-k Reasoning)
autonomously controlling a given moving body of the second plurality of moving bodies based on the predicted future trajectories to change movement of the given moving body. (as per “the results of the final decoding layer (the most-likely future evaluated by the trained scorer) are utilized as the plan for the AV and predictions for other agents” in P7, 4.2.2 Open-loop Planning, as per “To determine the optimal reasoning levels for planning, we analyze the impact of decoding layers on open-loop planning performance, and the results are presented in Table 2” in P6, 4.2.2 Open-loop Planning, as per “We evaluate the closed-loop planning performance of our model in a simulated environment that replays the logged trajectories of other agents while updating the ego agent’s state according to the planned trajectory” in P8, 4.2.3 Closed-loop Planning)
Huang fails to expressly disclose:
wherein predicting the modes of the first trajectories comprises grouping the first trajectories into a fewer number of the modes based on similarity of the first trajectories, wherein a given mode represents a class of trajectories;
training a trajectory prediction model on the predicted modes as a coverage loss term between the predicted modes;
Ma discloses of diversity prior based future prediction, comprising:
wherein predicting the modes of the first trajectories comprises grouping the first trajectories into a fewer number of the modes based on similarity of the first trajectories, (as per “, a similarity-clustered based technique to obtain the multi-modal ground-truth future may be used. Similar initial poses may be grouped, and their corresponding future poses may be viewed as the pseudo-possible future motions for each initial pose in the group. Such logic may be applied recursively. At certain steps, similar poses may be grouped again and the shared futures obtained” in ¶32, as per “Since there might be many similar poses to the given initial poses and most of the corresponding future poses may be very similar, a proper fixed number of future poses may be selected in X in order to capture the different modes” in ¶65-¶68, as per Eqn(s) in ¶37) wherein a given mode represents a class of trajectories; (as per “A human motion trajectory with time horizon T may be defined as Xt:t+T−1=[Xt, Xt+1, . . . , Xt+T−1], where Xt∈Rd is the human joints Cartesian coordinates at time step t. Given an observation C═Xt−T H +1:t, the future trajectories' distribution P(Xt+1:t+T f |C,ρ) may be obtained. Since such conditional probabilistic distribution may have one dominant mode…” in ¶37, as per “Similar initial poses may be grouped, and their corresponding future poses may be viewed as the pseudo-possible future motions for each initial pose in the group.” in ¶32, as per “the processor 102 or the system to cluster one or more similar initial poses (e.g., first time group similar pose dashed circle) and share their future poses as common ground-truth data” in ¶129)
training a trajectory prediction model on the predicted modes as a coverage loss term between the predicted modes; (as per “The larger ρ becomes, the more diverse samples may be generated and focused on the rarer cases. Conversely, the smaller ρ becomes, the prediction may be more focused on the most likely modes” in ¶37, as per “The diversity loss may be defined as:… The system may evaluate the differences between samples from the accuracy sampler and the diversity sampler. When the weight of diversity loss is large, this may have a negative influence on the accuracy sampler to approximate the data distribution. A goal or intention of the system may be to disentangle the accuracy objective and diversity objective,” in ¶55-¶56)
In this way, Ma operates to include an explicit diversity/coverage loss across predicted samples/modes (¶2-¶4, ¶28-¶32). Like Huang, Ma is concerned with autonomous systems (¶28).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Huang with Ma’s similarity-clustered multimodal future technique to provide a known way of grouping similar trajectory/motion examples into modes. Such modification would have predictably improved Huang’s multimodal trajectory prediction by providing representative trajectory-mode classes for training and by encouraging coverage of diverse plausible futures.
As per Claim 2, the combination of Huang and Ma teaches or suggests all limitations of Claim 1. Huang further discloses wherein the moving body of the second plurality of moving bodies is a vehicle comprising an autonomous driving system. (as per “Accurately forecasting the future behaviors of surrounding traffic participants and making safe and socially compatible decisions are critical capabilities for modern autonomous driving systems. However, predicting a traffic participant’s future behavior is a highly challenging task” in P1, 1. Introduction, as per Fig. 1)
PNG
media_image1.png
270
363
media_image1.png
Greyscale
As per Claim 9, Huang discloses of game-theoretic modeling and learning, comprising:
obtain training data including first trajectories for a first plurality of moving bodies and first map information of a first environment for a past time horizon; (as per “We consider a driving scene with N agents, where the AV is denoted as A0 and its neighboring agents as A1, · · · , AN-1 at the current time t = 0. Given the historical states of all agents (including the AV) over an observation horizon Th, S = {si−Th:0}, as well as the map information M including traffic lights and road waypoints, the goal is to jointly predict the future trajectories of neighboring agents Y over the future horizon Tf… In order to capture the uncertainty, the results are multi-modal future trajectories for the AV and neighboring agents,… where yj1:Tf is a sequence of predicted states, pj the probability of the trajectory, and M the number of modalities” in P3, 3.1 Game-theoretic formulation)
train a trajectory prediction model on modes of the first trajectories predicted by a mode-finding model trained by applying the training data to a game-theoretic mode-finding algorithm, (as per “The model employs a hierarchical decoding structure to capture agent interactions, iteratively refine predictions, and is trained based on the level-k game formalism” in P2, Introduction, as per “We denote the predicted multi-modal trajectories (essentially a Gaussian mixture model) of agent i at reasoning level k as π (k)” in P3, 3.1. Game-theoretic Formulation, as per “Then, at each iteration k, the level-k decoder takes as input the predicted trajectories from the level-(k−1) decoder, along with the background information, to predict the agent’s trajectories at the current level” in P2, Introduction)
predict trajectories for a second plurality of moving bodies based on applying observed data to the trajectory prediction model, (as per “the goal is to jointly predict the future trajectories of neighboring agents Y over the future horizon Tf… In order to capture the uncertainty, the results are multi-modal future trajectories for the AV and neighboring agents,… where yj1:Tf is a sequence of predicted states, pj the probability of the trajectory, and M the number of modalities” in P3, 3.1 Game-theoretic formulation) wherein the observed data includes second trajectories for a second plurality of moving bodies and second map information of a second environment; (as per “The input data comprises historical state information of agents… where ds represents the number of state attributes, and local vectorized map polylines… For each agent, we find Nm nearby map elements such as routes and crosswalks, each containing Np waypoints with dp attributes. The inputs are normalized according to the state of the ego agent…” in P4, 3.2. Scene Encoding, as per “At the final level of interaction decoding, we can obtain multi-modal trajectories for the AV and neighboring agents” in P5, 3.3 Future Decoding with Level-k Reasoning)
autonomously control a given moving body of the second plurality of moving bodies based on the predicted trajectories to change movement of the given moving body. (as per “the results of the final decoding layer (the most-likely future evaluated by the trained scorer) are utilized as the plan for the AV and predictions for other agents” in P7, 4.2.2 Open-loop Planning, as per “To determine the optimal reasoning levels for planning, we analyze the impact of decoding layers on open-loop planning performance, and the results are presented in Table 2” in P6, 4.2.2 Open-loop Planning, as per “We evaluate the closed-loop planning performance of our model in a simulated environment that replays the logged trajectories of other agents while updating the ego agent’s state according to the planned trajectory” in P8, 4.2.3 Closed-loop Planning)
Huang fails to expressly disclose:
a memory configured to store instructions; and
one or more processors communicably coupled to the memory;
wherein predicting the modes of the first trajectories comprises grouping the first trajectories into a fewer number of the modes based on similarity of the first trajectories, wherein a given mode represents a class of trajectories;
Ma discloses of diversity prior based future prediction, comprising:
a memory configured to store instructions; (as per ¶21)
one or more processors communicably coupled to the memory; (as per ¶20)
wherein predicting the modes of the first trajectories comprises grouping the first trajectories into a fewer number of the modes based on similarity of the first trajectories, (as per “, a similarity-clustered based technique to obtain the multi-modal ground-truth future may be used. Similar initial poses may be grouped, and their corresponding future poses may be viewed as the pseudo-possible future motions for each initial pose in the group. Such logic may be applied recursively. At certain steps, similar poses may be grouped again and the shared futures obtained” in ¶32, as per “Since there might be many similar poses to the given initial poses and most of the corresponding future poses may be very similar, a proper fixed number of future poses may be selected in X in order to capture the different modes” in ¶65-¶68, as per Eqn(s) in ¶37) wherein a given mode represents a class of trajectories;(as per “A human motion trajectory with time horizon T may be defined as Xt:t+T−1=[Xt, Xt+1, . . . , Xt+T−1], where Xt∈Rd is the human joints Cartesian coordinates at time step t. Given an observation C═Xt−T H +1:t, the future trajectories' distribution P(Xt+1:t+T f |C,ρ) may be obtained. Since such conditional probabilistic distribution may have one dominant mode…” in ¶37, as per “Similar initial poses may be grouped, and their corresponding future poses may be viewed as the pseudo-possible future motions for each initial pose in the group.” in ¶32, as per “the processor 102 or the system to cluster one or more similar initial poses (e.g., first time group similar pose dashed circle) and share their future poses as common ground-truth data” in ¶129)
In this way, Ma operates to include an explicit diversity/coverage loss across predicted samples/modes (¶2-¶4, ¶28-¶32). Like Huang, Ma is concerned with autonomous systems (¶28).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Huang with Ma’s similarity-clustered multimodal future technique to provide a known way of grouping similar trajectory/motion examples into modes. Such modification would have predictably improved Huang’s multimodal trajectory prediction by providing representative trajectory-mode classes for training and by encouraging coverage of diverse plausible futures.
Claim 10 is rejected using the same rationale, mutatis mutandis, applied to Claim 2 above, respectively.
As per Claim 17, Huang discloses of game-theoretic modeling and learning, comprising:
collecting observed trajectories for a plurality of moving bodies and map information of a first environment; (as per “We consider a driving scene with N agents, where the AV is denoted as A0 and its neighboring agents as A1, · · · , AN-1 at the current time t = 0. Given the historical states of all agents (including the AV) over an observation horizon Th, S = {si−Th:0}, as well as the map information M including traffic lights and road waypoints, the goal is to jointly predict the future trajectories of neighboring agents Y over the future horizon Tf… In order to capture the uncertainty, the results are multi-modal future trajectories for the AV and neighboring agents,… where yj1:Tf is a sequence of predicted states, pj the probability of the trajectory, and M the number of modalities” in P3, 3.1 Game-theoretic formulation)
predicting future trajectories for the plurality of moving bodies based on weighted modes output by a game-theoretic mode-finding model trained to detect modes as groups of trajectories and assign weights to each mode, (as per “We consider a driving scene with N agents, where the AV is denoted as A0 and its neighboring agents as A1, · · · , AN-1 at the current time t = 0. Given the historical states of all agents (including the AV) over an observation horizon Th, S = {si−Th:0}, as well as the map information M including traffic lights and road waypoints, the goal is to jointly predict the future trajectories of neighboring agents Y over the future horizon Tf… In order to capture the uncertainty, the results are multi-modal future trajectories for the AV and neighboring agents,… where yj1:Tf is a sequence of predicted states, pj the probability of the trajectory, and M the number of modalities” in P3, 3.1 Game-theoretic formulation, as per “to decode the GMM components of predicted futures… and the scores of these components” in P4, 3.3. Future Decoding with Level-k Reasoning, as per “Then, we apply weighted-average-pooling on the modality axis with the predicted scores from the level-(k − 1) layer… to obtain the agent future features… to model the interactions between agent future trajectories” in P4, 3.3. Future Decoding with Level-k Reasoning)
autonomously controlling a given moving body of the plurality of moving bodies based on the predicted future trajectories to change movement of the given moving body. (as per “the results of the final decoding layer (the most-likely future evaluated by the trained scorer) are utilized as the plan for the AV and predictions for other agents” in P7, 4.2.2 Open-loop Planning, as per “To determine the optimal reasoning levels for planning, we analyze the impact of decoding layers on open-loop planning performance, and the results are presented in Table 2” in P6, 4.2.2 Open-loop Planning, as per “We evaluate the closed-loop planning performance of our model in a simulated environment that replays the logged trajectories of other agents while updating the ego agent’s state according to the planned trajectory” in P8, 4.2.3 Closed-loop Planning)
Huang fails to expressly disclose:
non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system,
wherein trajectories are grouped into a fewer number of the modes based on similarity of the trajectories, wherein a given mode represents a class of trajectories;
Ma discloses of diversity prior based future prediction, comprising:
non-transitory computer-readable storage medium (as per ¶21) including instructions that, when executed by at least one processor of a computing system, (as per ¶20)
wherein trajectories are grouped into a fewer number of the modes based on similarity of the trajectories, (as per “, a similarity-clustered based technique to obtain the multi-modal ground-truth future may be used. Similar initial poses may be grouped, and their corresponding future poses may be viewed as the pseudo-possible future motions for each initial pose in the group. Such logic may be applied recursively. At certain steps, similar poses may be grouped again and the shared futures obtained” in ¶32, as per “Since there might be many similar poses to the given initial poses and most of the corresponding future poses may be very similar, a proper fixed number of future poses may be selected in X in order to capture the different modes” in ¶65-¶68, as per Eqn(s) in ¶37) wherein a given mode represents a class of trajectories; (as per “A human motion trajectory with time horizon T may be defined as Xt:t+T−1=[Xt, Xt+1, . . . , Xt+T−1], where Xt∈Rd is the human joints Cartesian coordinates at time step t. Given an observation C═Xt−T H +1:t, the future trajectories' distribution P(Xt+1:t+T f |C,ρ) may be obtained. Since such conditional probabilistic distribution may have one dominant mode…” in ¶37, as per “Similar initial poses may be grouped, and their corresponding future poses may be viewed as the pseudo-possible future motions for each initial pose in the group.” in ¶32, as per “the processor 102 or the system to cluster one or more similar initial poses (e.g., first time group similar pose dashed circle) and share their future poses as common ground-truth data” in ¶129)
In this way, Ma operates to include an explicit diversity/coverage loss across predicted samples/modes (¶2-¶4, ¶28-¶32). Like Huang, Ma is concerned with autonomous systems (¶28).
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang with diversity prior based future prediction as taught by Ma to enable another standard means of encouraging mode coverage, avoid mode collapse, and improve diversity while maintaining accuracy (¶28-¶32).
Claim 18 is rejected using the same rationale, mutatis mutandis, applied to Claim 2 above, respectively.
As per Claim 19, the combination of Huang and Ma teaches or suggests all limitations of Claim 17. Huang further discloses wherein the game-theoretic mode-finding model is trained by predicting modes from a plurality of training trajectories from a past time horizon. (as per “We consider a driving scene with N agents, where the AV is denoted as A0 and its neighboring agents as A1, · · · , AN-1 at the current time t = 0. Given the historical states of all agents (including the AV) over an observation horizon Th, S = {si−Th:0}, as well as the map information M including traffic lights and road waypoints, the goal is to jointly predict the future trajectories of neighboring agents Y over the future horizon Tf… In order to capture the uncertainty, the results are multi-modal future trajectories for the AV and neighboring agents,… where yj1:Tf is a sequence of predicted states, pj the probability of the trajectory, and M the number of modalities” in P3, 3.1 Game-theoretic formulation, as per “Given the historical states of all agents (including the AV) over an observation horizon Th, S = {si−Th:0}, as well as the map information M including traffic lights and road waypoints” in P3, 3.1 Game-theoretic formulation)
Huang fails to expressly disclose applying a coverage loss term between the predicted modes.
See Claim 17 for teachings of Ma. Ma further discloses applying a coverage loss term between the predicted modes. (as per “he larger ρ becomes, the more diverse samples may be generated and focused on the rarer cases. Conversely, the smaller ρ becomes, the prediction may be more focused on the most likely modes” in ¶37, as per “The diversity loss may be defined as:… The system may evaluate the differences between samples from the accuracy sampler and the diversity sampler. When the weight of diversity loss is large, this may have a negative influence on the accuracy sampler to approximate the data distribution. A goal or intention of the system may be to disentangle the accuracy objective and diversity objective,” in ¶55-¶56)
In this way, Ma operates to include an explicit diversity/coverage loss across predicted samples/modes (¶2-¶4, ¶28-¶32). Like Huang, Ma is concerned with autonomous systems (¶28).
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang with diversity prior based future prediction as taught by Ma to enable another standard means of encouraging mode coverage, avoid mode collapse, and improve diversity while maintaining accuracy (¶28-¶32).
As per Claim 20, the combination of Huang and Ma teaches or suggests all limitations of Claim 19. Huang further discloses wherein the game-theoretic mode-finding model is trained by predicting modes from map information of a second environment from the past time horizon. (as per “We consider a driving scene with N agents, where the AV is denoted as A0 and its neighboring agents as A1, · · · , AN-1 at the current time t = 0. Given the historical states of all agents (including the AV) over an observation horizon Th, S = {si−Th:0}, as well as the map information M including traffic lights and road waypoints, the goal is to jointly predict the future trajectories of neighboring agents Y over the future horizon Tf… In order to capture the uncertainty, the results are multi-modal future trajectories for the AV and neighboring agents,… where yj1:Tf is a sequence of predicted states, pj the probability of the trajectory, and M the number of modalities” in P3, 3.1 Game-theoretic formulation)
Claim(s) 3 & 11 are rejected under 35 U.S.C. 103 as being unpatentable over Huang (NPL Title: GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving, Year: 2023) in view of Ma (US Pub. No. 20230141610) in further view of Olson (US Pub. No. 20230289557).
As per Claim 3, the combination of Huang and Ma teaches or suggests all limitations of Claim 1. Huang further discloses:
providing the joint trajectory proposals as input to the game-theoretic mode-finding algorithm; (as per “We leverage level-k game theory to model agent interactions in an iterative manner. Instead of simply predicting a single set of trajectories, we predict a hierarchy of trajectories to model the cognitive interaction process. At each reasoning level, with the exception of level-0, the decoder takes as input the prediction results from the previous level, which effectively makes them a part of the scene, and estimates the responses of agents in the current level to other agents in the previous level” in P3, 3.1. Game-theoretic Formulation)
outputting a number of weighted modes for each moving body of the first plurality of moving bodies from the mode-finding model. (as per “the results are multi-modal future trajectories for the AV and neighboring agents, denoted by… where yj1:T is a sequence of predicted states, pj the probability of the trajectory, and M the number of modalities.)
PNG
media_image2.png
26
249
media_image2.png
Greyscale
Huang and Ma fail to expressly disclose:
generating joint trajectory proposals by perturbing the first trajectories, wherein each joint trajectory proposal comprises a perturbed first trajectory of each moving body of the first plurality of moving bodies;
Olson discloses constructing outcomes to guide multi-policy decision making, comprising:
generating joint trajectory proposals by perturbing the first trajectories, (as per “The joint state space of the system is… the collective state x(t)∈X includes the robot state plus all the agents visible to the robot at time t” in ¶31, as per “where xt∈X is the collective state consisting of the robot state plus all the agents at time t of the forward simulation. The transition function T( ) captures the trajectory that each agent is executing while at the same time considering the interactions with all other agents” in ¶54) wherein each joint trajectory proposal comprises a perturbed first trajectory of each moving body of the first plurality of moving bodies; (as per “In contrast, by computing accurate gradients, one can perturb all the agents simultaneously without divergence” in ¶73, as per “simulated using the perturbed inputs. Simulating movement of the one or more monitored objects and the controlled objects with different perturbed inputs is repeated until a predetermined condition is met, thereby generating a plurality of perturbed outcomes” in ¶9)
In this way, Olson operates to perturb seed states/configurations/state estimates that are derived from trajectories and simulate to obtain perturbed multi-agent trajectories (¶53-¶54, ¶9). Like Huang and Ma, Olson is concerned with autonomous systems.
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang and Ma with the multi-policy decision making as taught by Olson to enable another standard means of generating joint trajectory proposals comprising perturbed trajectories of each agent via perturbation and simulation (¶31, ¶9).
Claim 11 is rejected using the same rationale, mutatis mutandis, applied to Claim 3 above, respectively.
Claim(s) 4 & 12 are rejected under 35 U.S.C. 103 as being unpatentable over Huang (NPL Title: GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving, Year: 2023) in view of Ma (US Pub. No. 20230141610) in view of Olson (US Pub. No. 20230289557) in view of Luo (US Pub. No. 20230406361) in further view of Shi (NPL Title: Motion Transformer with Global Intention Localization and Local Movement Refinement, Year: 2023).
As per Claim 4, the combination of Huang, Ma, and Olson teaches or suggests all limitations of Claim 3.
Huang fails to expressly disclose:
for each perturbed first trajectory for a respective moving body of the first plurality of moving bodies, combining the respective perturbed first trajectory to a perturbed first trajectory of other moving bodies of the first plurality of moving bodies to generate the joint trajectory proposals;
scoring each joint trajectory proposal based on similarity to the first trajectories;
identifying a local maximum score for the joint trajectory proposals; and
outputting the number of weighted modes for each moving body based on the identified local maximum.
See Claim 3 for teachings of Ma. Ma further discloses:
PNG
media_image3.png
79
285
media_image3.png
Greyscale
scoring each joint trajectory proposal based on similarity to the first trajectories; (as per Eqn. 6, as per “a similarity-clustered based technique to obtain the multi-modal ground-truth future may be used. Similar initial poses may be grouped, and their corresponding future poses may be viewed as the pseudo-possible future motions for each initial pose in the group” in ¶32)
In this way, Ma operates to include an explicit diversity/coverage loss across predicted samples/modes (¶2-¶4, ¶28-¶32). Like Huang and Olson, Ma is concerned with autonomous systems (¶28).
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang & Olson with diversity prior based future prediction as taught by Ma to enable another standard means of encouraging mode coverage, avoid mode collapse, and improve diversity while maintaining accuracy (¶28-¶32).
Huang and Ma fail to expressly disclose:
for each perturbed first trajectory for a respective moving body of the first plurality of moving bodies, combining the respective perturbed first trajectory to a perturbed first trajectory of other moving bodies of the first plurality of moving bodies to generate the joint trajectory proposals;
identifying a local maximum score for the joint trajectory proposals; and
outputting the number of weighted modes for each moving body based on the identified local maximum.
See Claim 3 for teachings of Olson. Olson further discloses:
identifying a local maximum score for the joint trajectory proposals; (as per “The objective function P(x0)C(X) can have multiple local-minima depending on the number of agents and the complexity of the initial configuration. Finding the global maximum through exhaustive search is computationally infeasible due to the large state-space. The goal is to quickly find an influential configuration whose value is comparable to the global optimum even if it may not be the highest-valued configuration” in ¶59, as per “The algorithm samples an initial configuration from P(x0) as indicate at line 5 and optimizes it, perturbing the sampled configuration iteratively towards increasingly influential outcomes until convergence to a local optima whose objective function value is U*” in ¶61, as per “a policy score is determined for the respective policy, where the policy score correlates to the perturbed outcome having highest value amongst the plurality of perturbed outcomes for the respective policy” in ¶10, as per ¶61)
In this way, Olson operates to perturb seed states/configurations/state estimates that are derived from trajectories and simulate to obtain perturbed multi-agent trajectories (¶53-¶54, ¶9). Like Huang and Ma, Olson is concerned with autonomous systems.
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang and Ma with the multi-policy decision making as taught by Olson to enable another standard means of generating joint trajectory proposals comprising perturbed trajectories of each agent via perturbation and simulation (¶31, ¶9).
Huang, Ma, and Olson fail to expressly disclose:
for each perturbed first trajectory for a respective moving body of the first plurality of moving bodies, combining the respective perturbed first trajectory to a perturbed first trajectory of other moving bodies of the first plurality of moving bodies to generate the joint trajectory proposals;
outputting the number of weighted modes for each moving body based on the identified local maximum.
Luo discloses of structured multi-agent interactive trajectory forecasting, comprising:
for each perturbed first trajectory for a respective moving body of the first plurality of moving bodies, combining the respective perturbed first trajectory to a perturbed first trajectory of other moving bodies of the first plurality of moving bodies to generate the joint trajectory proposals; (as per “this case, the possible outcomes are represented by {Agent A, Agent B}×{going straight, turning left}. For example, the marginal probability of agent A turning left is 0.5, the marginal probability of agent A going straight is 0.5, the marginal probability of agent B turning left is 0.5, and the marginal probability of agent B going straight is 0.5… a system may not fully capture that two of four possible outcomes in {Agent A, Agent B}×{going straight, turning left} are not feasible (e.g., can result in a collision). That is, the marginal probability predictions do not reflect that, to avoid a collision, Agent A will likely not go straight if Agent B turns left, and Agent B will likely not go straight if Agent A turns left.” in ¶52-¶53, as per “the described system models the joint distribution of all agents of interest I in a scene as multiple joint probabilities of predicted trajectories for each agent (e.g., p(s1, s2, . . . , sI)) in order to generate more complete and accurate trajectory predictions” in ¶55, “the trajectory prediction system 114 uses the marginal trajectory prediction neural network to generate the marginal trajectory prediction 302 for each agent of the multiple agents. The marginal trajectory prediction 302 defines multiple possible future trajectories for each agent after the current time point and a respective likelihood score for each of the possible future trajectories” in ¶62))
In this way, Luo operates to build joint proposals from marginal predicted trajectories (¶62, ¶75). Like Huang, Ma, and Olson, Luo is concerned with autonomous systems (¶11).
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang, Ma, and Olson with the structured multi-agent interactive trajectory forecasting as taught by Luo to enable another standard means of modeling all agents of interest in a scene as multiple joint probabilities of predicted trajectories for each agent. Such modification generates more complete and accurate trajectory predictions (¶28-¶32).
Huang, Ma, Olson, and Luo fail to expressly disclose:
outputting the number of weighted modes for each moving body based on the identified local maximum.
Shi discloses of a motion transformer, comprising
outputting the number of weighted modes for each moving body (as per “we predict the probability p and parameters…. of each Gaussian component as follows… includes K Gaussian components N1:K( x x; y y; ) with probability distribution p1:K.” in P6, 3.2 Transformer Decoder with Motion Query Pair, as per Eqn. 9) based on the identified local maximum. (as per “To generate 6 future trajectories for evaluation, we use non-maximum suppression (NMS) to select top 6 predictions from 64 predicted trajectories by calculating the distances between their endpoints, and the distance threshold is set as 2.5m” in P7, 4.1 Experimental Setup, as per “we combine the marginal predictions of two interacting agents into joint prediction a in [7, 15, 45], where we take the top 6 joint predictions from 36 combinations of these two agents. The
PNG
media_image4.png
47
512
media_image4.png
Greyscale
confidence of each combination is the product of marginal probabilities” in P8, 4.2 Main Results)
In this way, Shi operates to output a number of modes for each agent by performing non-maximum suppression to select top predictions from predicted trajectories. Like Huang, Ma, Olson, and Luo, Shi is concerned with autonomous systems.
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang, Ma, Olson, and Luo with the motion transformer as taught by Shi to enable another standard means of modeling all agents of outputting a fixed count of modal trajectories based on a peak/local-maximum selection (P8, 4.2 Main Results).
Claim 12 is rejected using the same rationale, mutatis mutandis, applied to Claim 4 above, respectively.
Claim(s) 5, 7, 13, & 15 are rejected under 35 U.S.C. 103 as being unpatentable over Huang (NPL Title: GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving, Year: 2023) in view of Ma (US Pub. No. 20230141610) in further view of Rus (US Pub. No. 20210146964).
As per Claim 5, the combination of Huang and Ma teaches or suggests all limitations of Claim 1. Huang and Ma fail to expressly disclose:
applying the training data to a machine learning reward algorithm to generate a reward model for each moving body,
wherein generating the mode-finding model for each moving body is based on providing the reward models to the game-theoretic mode-finding algorithm.
Rus discloses of social behavior for autonomous vehicles, comprising:
applying the training data (as per “game's reward functions are dynamic and dependent on the vehicles' states and the environment. The reward functions is learned from human driving data” in ¶6) to a machine learning reward algorithm to generate a reward model for each moving body, (as per “also corresponds to the actions maximizing the likelihood under the maximum entropy model… used to learn our rewards by IRL. Under this model, the probability of actions u is proportional to the exponential of the utility encountered along the trajectory” in ¶39-¶41)
wherein generating the mode-finding model for each moving body is based on providing the reward models to the game-theoretic mode-finding algorithm. (as per “The AV can predict future motion of the vehicle 120 for candidate SVOs based on a utility-maximizing decision model (see, e.g., Section 3)” in ¶19, as per “Using the reward (payoff) structure learned from data and our utility-maximizing behavior model, it generates candidate trajectories based on possible SVO values. The most likely SVO is the one that best matches a candidate trajectory to the actual observed trajectory” in ¶26, as per “The Nash equilibrium yields a control law for the AV ui* as well as predicted actions u¬i* for all other m−1 agents N time steps into the future. Based on learned reward functions and the maximum entropy model, (5), u¬i* are also maximum likelihood predictions” in ¶44)
In this way, Rus operates to improve prediction/planning realism/social compliance by learning reward/utility from real driving data and using those learned rewards in the game-theoretic decision/prediction step (¶6, ¶25, ¶38-¶44)). Like Huang and Ma, Rus is concerned with autonomous systems.
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang and Ma with the social behavior for autonomous vehicles as taught by Rus to enable another standard means of parameterizing the game-theoretic optimization using data-learned reward structures.
As per Claim 7, the combination of Huang, Ma, and Rus teaches or suggests all limitations of Claim 5. Huang and Ma fail to expressly disclose wherein the machine learning reward algorithm comprises an inverse reinforcement learning (IRL) algorithm.
See Claim 5 for teachings of Rus. Rus further discloses wherein the machine learning reward algorithm comprises an inverse reinforcement learning (IRL) algorithm. (as per “the AV control approach enables social compliance by learning human reward functions through Inverse Reinforcement Learning (IRL)” in ¶24)
In this way, Rus operates to improve prediction/planning realism/social compliance by learning reward/utility from real driving data and using those learned rewards in the game-theoretic decision/prediction step (¶6, ¶25, ¶38-¶44)). Like Huang and Ma, Rus is concerned with autonomous systems.
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang and Ma with the social behavior for autonomous vehicles as taught by Rus to enable another standard means of parameterizing the game-theoretic optimization using data-learned reward structures.
Claim 13 is rejected using the same rationale, mutatis mutandis, applied to Claim 5 above, respectively.
Claim 15 is rejected using the same rationale, mutatis mutandis, applied to Claim 7 above, respectively.
Claim(s) 6 & 14 are rejected under 35 U.S.C. 103 as being unpatentable over Huang (NPL Title: GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving, Year: 2023) in view of Ma (US Pub. No. 20230141610) in view of Rus (US Pub. No. 20210146964) in view of Olson (US Pub. No. 20230289557) in further view of Gao (WO Pub. No. 2021080507).
As per Claim 6, the combination of Huang, Ma, and Rus teaches or suggests all limitations of Claim 1. Huang, Ma, and Rus fail to expressly disclose:
computing trajectory variations for each of the first trajectories;
assigning a reward to each trajectory variation and each of the first trajectories, wherein the rewards are assigned to encourage each of the first trajectories.
Olson discloses constructing outcomes to guide multi-policy decision making, comprising:
computing trajectory variations for each of the first trajectories; (as per “Seed states are perturbed (e.g, using backpropagation) and movement of the one or more monitored objects and the controlled object objects is simulated using the perturbed inputs. Simulating movement of the one or more monitored objects and the controlled objects with different perturbed inputs is repeated until a predetermined condition is met, thereby generating a plurality of perturbed outcomes” in ¶9, as per “That is, the simulator 120 forward simulates each of the possible variations of movements for each of the monitored objects while the controlled object 100 is executing different policies 136. With each simulation, a cost and a probability may be determined for each of the policies 136” in ¶63)
In this way, Olson operates to perturb seed states/configurations/state estimates that are derived from trajectories and simulate to obtain perturbed multi-agent trajectories (¶53-¶54, ¶9). Like Huang, Ma, and Rus, Olson is concerned with autonomous systems.
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang, Ma, and Rus with the multi-policy decision making as taught by Olson to enable another standard means of generating joint trajectory proposals comprising perturbed trajectories of each agent via perturbation and simulation (¶31, ¶9).
Huang, Ma, Rus, and Olson fail to expressly disclose:
assigning a reward to each trajectory variation and each of the first trajectories, wherein the rewards are assigned to encourage each of the first trajectories.
Gao discloses of autonomous vehicle control using context aware reward, comprising:
assigning a reward to each trajectory variation and each of the first trajectories, wherein the rewards are assigned to encourage each of the first trajectories. (as per “maximum entropy is used to maintain a probability distribution over all possible trajectories. However, any optimisation algorithm may be implemented - e.g. Bayesian optimisation, expectation maximisation, gradient descent (standard mini-batch or stochastic),” in P4¶15, as per “maximum entropy the probability of user preference for any given trajectory between specified start and goal states is proportional to the exponential of the reward along the trace” in P4¶16, as per “The reward may be highest for leaf nodes associated with control parameters resulting in the autonomous vehicle travelling along the path. The reward may be decreasingly lower for leaf nodes associated with control parameters resulting in increasingly larger deviations from the path” in P2¶10)
In this way, Gao operates to evaluate any given trajectory within all possible trajectories based on reward along the trace (P4P15-¶16). Like Huang, Ma, Rus, and Olson, Gao is concerned with autonomous systems.
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang, Ma, Rus, and Olson with the autonomous vehicle control using context-aware reward as taught by Gao to enable another standard means of scoring/rewarding each simulated trajectory variation and shaping those rewards to encourage certain trajectories (P3¶3).
Claim 14 is rejected using the same rationale, mutatis mutandis, applied to Claim 6 above, respectively.
Claim(s) 8 & 16 are rejected under 35 U.S.C. 103 as being unpatentable over Huang (NPL Title: GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving, Year: 2023) in view of Ma (US Pub. No. 20230141610) in further view of Peters (NPL Title: Learning mixed strategies in trajectory games, Year: 2022).
As per Claim 8, the combination of Huang and Ma teaches or suggests all limitations of Claim 1. Huang and Ma fail to expressly disclose wherein the game-theoretic mode-finding algorithm comprises a local optimization algorithm to enumerate modes.
Peters discloses learning mixed strategies in trajectory games, comprising wherein the game-theoretic mode-finding algorithm comprises a local optimization algorithm to enumerate modes. (as per “a motion planning problem may be encoded as a nonlinear program and solved efficiently to a locally-optimal solution” in P1, Introduction, as per “Nash equilibria can be intractable to compute and modern methods often settle for local equilibria, in which players’ trajectories are only locally optimal, as per “which allows players to optimize multiple candidate trajectories in unison and thereby construct more competitive “mixed” strategies” in P1, Abstract, as per “A Nash equilibrium for Game (5) can be found by simultaneous gradient descent over each player’s reference generator parameters, θ1 and θ2” in P4, IV. Approach)
In this way, Peters operates to handle multiple “modes” (i.e. players “optimize multiple candidate
trajectories in unison” as per Abstract). Like Huang and Ma, Peters is concerned with autonomous systems.
It would have been obvious for one of ordinary skill in the art before the effective filing date to have modified the system(s) of Huang and Ma with the learning mixed strategies in trajectory game as taught by Peters to enable computation of game solutions where “modern methods often settle for local equilibria” while also supporting multi-candidate trajectory outputs (Abstract & Introduction).
Claim 16 is rejected using the same rationale, mutatis mutandis, applied to Claim 8 above, respectively.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TYLER R ROBARGE whose telephone number is (703)756-5872. The examiner can normally be reached Monday - Friday, 8:00 am - 5:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ramon Mercado can be reached on (571) 270-5744. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.R.R./Examiner, Art Unit 3658
/Ramon A. Mercado/Supervisory Patent Examiner, Art Unit 3658
Prosecution Insights