DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1- 28 have been presented for examination based on the amendment filed on 03/25/2026.
This action is final rejection.
Information Disclosure Statement
The IDS filed on 12/27/2023 is reviewed and considered. See attached file.
Response to Arguments
Following Applicants amendments to the drawing , the objections of the
drawing is Withdrawn
Following Applicants amendments to the Specification, the objections of the
Specification is Withdrawn.
Following Applicants amendments to the Claims, the objections of the Claims is
Withdrawn.
Following Applicants amendments, the 112 (b) interpretation and rejection of the
claims is Withdrawn, but a newly 112(b) rejection found on claim 3, see 112(b) rejection below.
Applicants Argument: Applicant’s arguments directed the 103 rejection are based on newly amended subject matter.
Examiner’s Response: All arguments are addressed in the 103 rejection of the claims below.
Applicant’s argument: Shuncheng – Sriram – Siddiqui in combination or individually doesn’t teach wherein one or more of the plurality of simulated agents are designated as interactive agents," and that, for each partial trajectory at each particular time step, the method includes "determining, for each interactive agent, a respective
action to be performed by the interactive agent at the particular time step," "generating ... a joint action that includes (i) the respective actions for the interactive agents and (ii) the respective actions for any simulated agents that are not designated as interactive agents," and "providing the joint action to a transition function for the simulated environment.
Examiner’s Response: The examiner agrees, the combined model does not explicitly teach wherein one or more of the plurality of simulated agents are designated as interactive agents, generating ... a joint action that includes (i) the respective actions for the interactive agents and (ii) the respective actions for any simulated agents that are not designated as interactive agents," and "providing the joint action to a transition function for the simulated environment, but the newly added prior art by Isele teaches this limitations as it is addressed on 103 claim rejection.
While the examiner disagrees on examiners argument about the combined model of Shuncheng – Sriram – Siddiqui in combination or individually doesn’t teach for each partial trajectory at each particular time step, the method includes "determining, for each interactive agent, a respective action to be performed by the interactive agent at the particular time step.
Examiner Response: Shuncheng teaches as it shown on figure 3, "impact factor" based on "the future predicted trajectories of the surrounding vehicles, the respective action (Queuing, jumping the queue and crossing) of conventional vehicles and partial trajectory is interpreted as convectional vehicles trajectory as shown on Fig. 3 and on section 5.1, Shuncheng also teaches the presence of interaction between the autonomous and convectional vehicles. If the applicant do not agree on examiners interpretation, the claim should be amended to clearly point out what the partial trajectories and the respective action are. See claim 1 newly 103 rejection below.
Therefore, the 103 rejection is maintained.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3-4 and 7-10 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 3, recites “ a joint action” There is insufficient antecedence basis for this limitation in the claim since “ a joint action” is introduced in claim 1.
Claims 4 and 7-10 are also rejected since they depends on claim 3.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 21 and 27-18 are rejected under 35 U.S.C. 103 as being unpatentable over Shuncheng (Shuncheng Liu1 , Han Su1 ,2* , Yan Zhao3, Kai Zeng4 , Kai Zheng1 ,2*. 2021. Lane Change Scheduling for Autonomous Vehicle: A Predictionand-Search Framework. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '21 ), August 14-18, 2021.) in view of Sriram (Sri ram N N 1, Buyu Liu, Francesco Pittaluga, and Manmohan Chand raker. "SMART: Simultaneous Multiagent Recurrent Trajectory Prediction."arXiv:2007.13078v1, 26 Jul 2020), further in the view of Isele; David Francis (US 11242054 B2).
As of claim 1, Shuncheng teaches A method performed by one or more computers, the method comprises: (Section 5.1, We implemented all models and algorithms in Python on Linux, and ran the experiments on a machine with an Intel(R) CPU i7-
4770@3.4GHz and 32G RA)
obtaining initial state data specifying an initial state of a simulated environment that includes a plurality of simulated agents at an initial time step, (section 2.2, The autonomous vehicle can obtain the real-time location of surrounding vehicles through its sensors or loV (Internet of Vehicles), and make decisions on lane change maneuver at each time instant within a target time duration T of interest). As stated above Shuncheng disclose obtaining real time information's of vehicles( plurality) at each time instance) wherein the simulated environment is a simulation of a real-world environment; (section 5.1, most of the experiments were conducted in a simulated environment called SIM .... Furthermore, we tested the trajectory prediction model (GASLED) on a dataset constructed by merging two commonly-used real-world datasets).
generating a plurality of simulated trajectories that each start from the
initial state of the simulated environment, Shuncheng stated above most of this
experiment was made in a simulated environment and they perform a future prediction
using a deep model for z time step (section 2.2, In order to reduce the impact on
conventional vehicles, the autonomous vehicle needs to predict the future trajectories of
surrounding conventional vehicles. To this end, we develop an LSTM-based deep
model, called GAS-LED, to predict the trajectories of z time steps ahead by utilizing
historical trajectories in the past n time steps).
for each of a sequence of a plurality of time steps starting from the initial
time step and until a final time step, (section 2.2, we develop an LSTM-based deep
model, called GAS-LED, to predict the trajectories of z time steps ahead by utilizing
historical trajectories in the past n time steps). (i) state data characterizing a state of
the simulated environment at the time step, wherein the sequence of time steps
includes a plurality of pruning time steps,(section 4.2, At each time step, once
reaching the search depth (i.e., z), the algorithm will evaluate the solutions found during
search at various depths and return the best one (the one with the highest cumulative
weights)). (ii) a respective action performed by each of the plurality of simulated
agents at the time step (section4.2, the best one (the one with the highest cumulative
weights, example following the threshold y , and the non-candidate nodes are pruned
accordingly.) this shows they perform an action at each time step as it was listed on the
above limitation. wherein the sequence of time steps includes a plurality of pruning
time steps(section 4.2, example following the threshold y, and the non-candidate
nodes are pruned accordingly), and wherein the generating comprises, at each
particular time step in the sequence: obtaining data specifying a set of partial
trajectories as of the particular time step that each include respective state data
for the particular time step that characterizes a respective state of the
simulated(section 2.1, A trajectory T consists of a sequence of vehicle's trajectory
points, ordered by time step t, i.e., T = (p 1 , p 2, • • • , pt). We use TCi and TA to
denote the trajectory of conventional vehicle Ci and autonomous vehicle A respectively)
for each partial trajectory: determining, for each interactive agent, a respective action to be performed by the interactive agent at the particular time step;( Section 4.2, Formally, the impact factor Ft im is defined as the sum of the impact FCi ,t im for each surrounding conventional vehicles Ci ∈ C, i.e., Ft im =∑ FCi ,t im, … Figure 3 illustrates the three scenarios, where the red shaded area, called conflicting zone, is the location that both vehicles plan to arrive at the next time step. Prior transportation studies have shown that crossing usually has the greatest impact, while queuing has the least [1, 17]. So we simply assign 1, 2 and 3 as the impact factors for queuing, jumping the queue and crossing. Next we present how to predict the three impact situations….Section 5.1 As it requires interaction between the autonomous and conventional vehicles, most of the experiments were conducted in a simulated environment called SIM
PNG
media_image1.png
170
472
media_image1.png
Greyscale
)
as it is shown on Figure 3, the respective actions for each conventional and autonomous vehicle is shown and its impact factor(action of vehicle) is also computed at each time step.
Shuncheng doesn't teach explicitly teach wherein one or more of the plurality of simulated agents are designated as interactive agents, generating joint action that includes (i) the [a] respective actions the interactive agents and (ii) the respective actions for any simulated agents that are not designated as interactive agents; providing the joint action to a transition function for the simulated environment; obtaining, using I[a]] the transition function for the simulated environment and based on the joint action, next state data characterizing a next state of the environment at a next time step given that the joint action is performed by the plurality of simulated agents when the simulated environment is in the respective state characterized by the respective state data at the particular time step in the partial trajectory and updating the partial trajectory to include the joint action for the particular time step and the next state data characterizing the next state of the environment at the next time step.
While Sriram teaches wherein one or more of the plurality of simulated agents are designated as interactive agents(section3, “dynamic simulation” In order to limit the acceleration under safety limit for the any traffc situation and to incorporate interactions among different agents in the scene we use an IDM[37] behavior for the simulated vehicles)
obtaining, using I[a]] the transition function for the simulated environment
and based on the joint action, next state data characterizing a next state of the environment at a next time step given that the joint action is performed by the plurality of simulated agents when the simulated environment is in the respective state characterized by the respective state data at the particular time step in the partial trajectory ( Section 4.2, "Method", At any timestep t, the inputs to the network conditional generator are the following, a scene context map I (HxWx3), a single representation of all agents current state St (HxWx2), location mask Mt (HxWx1 ), a one-hot trajectory specific label for each agent projected at agent specific locations in a grid from C At (HxWx3) and a latent vector map ZAt (HxWx16) containing zi obtained from Qcp(zi I RXi , ci) during training phase or sampled from prior distribution Pv(Zi I RXi, ci) at test time. Formally the network input Et is given by: Et= [I, S1 ,M1 , C1 , zt ],)
updating the partial trajectory to include the joint action for the particular
time step and the next state data characterizing the next state of the environment
at the next time step( section 4.2, page 10, During the prediction phase (tabs, ... , T), the outputs are not directly fed back as inputs to the network rather the agent's state is updated to the next location in the scene based on the predictions. The relative predicted location RXA1- 1 i gets updated to absolute predicted location x·1 i to obtain an updated scene state map SA1 containing updated locations of all the agents in the scene)
Shuncheng and Sriram are both considered to be analogous to the claimed
invention because they focus on trajectory prediction of vehicles in order to improve
autonomous vehicles interaction on the road. Therefore, it would be obvious to one of
the ordinary skills in the art before the effective filling date to have applied Sriram teaching of predict the future trajectories for every agent at its corresponding grid map
location in the scene with time, it uses a function ( a conditional generator) to generate
updated input (network input and the agent's state is also updated to the next location in
the scene based on the predictions. So, applying this taught to Shuncheng in order to perform a partial trajectory prediction for agents based on time step.
The motivation would have been is to increase the accuracy and efficiency of
trajectory prediction and in addition, we demonstrate that our simulated dataset is
diverse and general, thus, is useful to train or test prediction models (Sriram, section 6,
"conclusion")
The modified model does not explicitly teach generating joint action that includes (i) the [a] respective actions the interactive agents and (ii) the respective actions for any simulated agents that are not designated as interactive agents; providing the joint action to a transition function for the simulated environment.
While Isele teaches generating joint action that includes (i) the [a] respective actions the interactive agents and (ii) the respective actions for any simulated agents that are not designated as interactive agents ([0020], As described herein, subscript/superscript notation variable time agent, action may be utilized (i.e., a superscript may be agent or action related and a subscript may be time related). In a stochastic game, at time t each agent i in state st takes an action at i according to their policy πi. All the agents then transition to the state st+1 and receive a reward rt i. Stochastic games can be described as a tuple (S, A, P, R) where S is the set of states, and A={A1, . . . , An} is the joint action space consisting of the set of each agent's actions, where n is the number of agents.
[0029], To reduce the number of traffic participant actions, a target interactive agent may be selected and then non-interactive predictions may be assumed for the other traffic agents).
providing the joint action to a transition function for the simulated environment ([0020], All the agents then transition to the state st+1 and receive a reward rt i. Stochastic games can be described as a tuple (S, A, P, R) where S is the set of states, and A={A1, . . . , An} is the joint action space consisting of the set of each agent's actions, where n is the number of agents. The reward functions R={R1, . . . , Rn} describe the reward for each agent S×A→R. The transition function P:S×A×S→[0,1] describes how the state evolves in response to all the agents' collective actions).
Isele is considered to be analogous to the claimed invention because it teaches autonomous vehicle decision making. Therefore it would be obvious for a person of ordinary skill in the art before the effective filling date to integrate Isele teaching of generating a joint action of different agents and providing the joint action into the transitional function, with the modified model.
The motivation would have been by identifying a traffic participant from the two or more traffic participants associated with the selected gap based on a coarse probability of a successful merge between the autonomous vehicle and a corresponding traffic participant (Isele, [0002]).
Claim 27 and 28 is similar in scope to that of claim 1, with additional limitations of which Isele teaches one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations and non-transitory computer-readable media storing instructions ([0064], In other aspects, the computing device 712 includes additional features or functionality. For example, the computing device 712 may include additional storage such as removable storage or non-removable storage, including magnetic storage, optical storage, among others. Such additional storage is illustrated in FIG. 7 by storage 720. In one aspect, computer readable instructions to implement one aspect provided herein are in storage 720).
Therefore claim 27 and 28 is rejected on the same rational as claim 1.
As of claim 5, the modified model teaches the limitation of claim 1, and Shuncheng also teaches wherein the initial state of the simulated environment corresponds to a reference initial state of (i) the real-word environment or (ii) the simulated environment at an initial time step in a reference trajectory ( section 5.1, Furthermore, we tested the trajectory prediction model (GASLED) on a dataset constructed by merging two commonly-used real-world datasets: NGSIM US-101 [9] and 1-80 [16]. The merged dataset consists of real trajectories of conventional vehicles traveling on a 1.14km-length freeway segment with six straight lane). wherein each of the plurality of simulated agents corresponds to (i) a different real-world agent in the reference initial state of the real-world environment or (ii) a different simulated agent in the reference initial state of the simulated environment( section 4.1, A vehicle's status at time t can be described using a quadruple (p1 .L, p 1+1 .L, p 1 .Dion, p 1+1 .Dion), which is used to describe its position at t and its immediate future position at t + 1 ). Shuncheng uses a real-world data set as it cited above and it also find the vehicles position with time, this shows the plurality of simulated agents corresponding to different real-world agent with time in a real-world environment.
As of claim 21, the modified model teaches all the limitation of claim 1 and Shuncheng also teaches wherein removing one or more partial trajectories from the set of partial trajectories based on the respective scores comprises: removing a threshold number of partial trajectories that have the worst respective scores.( section 4.2, At each time step, once reaching the search depth (i.e., z), the algorithm will evaluate the solutions found during search at various depths and return the best one (the one with the highest cumulative weights)
Claim 2 is rejected under 35 U.S. C. 103 as being unpatentable over Shuncheng (Shuncheng Liu1 , Han Su1 ,2* , Yan Zhao3 , Kai Zeng4 , Kai Zheng1 ,2* . 2021. Lane Change Scheduling for Autonomous Vehicle: A Prediction-and-Search Framework. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '21), August 14-18, 2021) in view of Sriram (Sriram N N1, Buyu Liu, Francesco Pittaluga, and Manmohan Chandraker. "SMART: Simultaneous Multiagent Recurrent Trajectory Prediction." arXiv:2007.13078v1, 26 Jul 2020) further in the view of Isele; David Francis (US 11242054 B2), further in the view of Richard (R. P. Ma, F-S. Tsung, and M-H. Ma. 1989. A dynamic load balancer for a parallel branch and bound algorithm. In Proceedings of the third conference on Hypercube concurrent computers and applications - Volume 2 (C3P)).
As of claim 2, the modified model teaches all the limitations of claim 1 and Shuncheng also teaches wherein the generating comprises, at each particular time step in the sequence: when the particular time step is a pruning time step of the plurality of pruning time steps ( section 4.2, Specifically, in i-th layer, we rank all the nodes based on their cumulative weights CW, i.e., summation of (V tA - F1 im) where i E [t, t + i]. Then, we can select the high-confidence nodes (fromTop1 toTopK) as the candidate nodes based on y, i.e. CWropK -CWropK+1 ~ y. Thereafter, the nodes in i+1-th layer are the child nodes of the candidate nodes in i-th layer, while the non-candidate nodes are pruned to reduce search space).
determining a respective score for each partial trajectory that represents a likelihood that the partial trajectory satisfies one or more criteria for the simulated trajectories (section 4.2, based on the current trajectory point of the autonomous vehicle, some invalid nodes can be discarded based on the following rules: example When the autonomous vehicle is in the rightmost lane, three direct child nodes (denoting the behavior of 'Change right') of the current node will be removed, .... Each edge maintains a cumulative weight. In this case, the number of candidate nodes in each layer is different ( 4-6-7-5-5), following the threshold y , and the non candidate nodes are pruned accordingly). As it was listed above, Shuncheng use the respective weight as the score in order to prune some invalid nodes based on some rules (criteria as it is listed on the claim limitation).
removing one or more partial trajectories from the set of partial trajectories based on the respective scores (section 4.2, Each edge maintains a cumulative weight. In this case, the number of candidate nodes in each layer is different ( 4-6-7-5-5), following the threshold y , and the non-candidate nodes are pruned accordingly, see Fig .4, "adaptive beam search example")
While the modified model teaches pruning of invalid nodes as taught in claim 1, but the modified model doesn't explicitly teach replacing the partial trajectory in the set with a copy of one of the partial trajectories that was not removed from the set.
However, Richard teaches for each of the one or more partial trajectories that were removed from the set, replacing the partial trajectory in the set with a copy of one of the partial trajectories that was not removed from the set.( Section 2.2, The BB( branch and bound algorithm) technique partitions the problem into subproblems (branching) and eliminates generated subproblems that are not better than the ones already known (bounding). And on section 3.3, The busy node estimates its remaining work in order to decide whether or not to transfer some work to an idling node). As it is stated, BB algorithm divide the nodes in to branches and it prune worse. It also transfers works to replace pruned node which shows coping of unpruned nodes as stated on the claim limitation.
Richard is considered to be analogous to the claimed invention because they focus on trajectory prediction of vehicles by performing pruning and replacing in parallel searching. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Richard's teaching of branching and bound algorithm into the modified model order to perform pruning of less likely trajectories and to replace pruned by a copy of unpruned by transferring of some of the works to idle nodes.
The motivation would have been is to minimize the communication overhead by finding a way to communicate pruned and unpruned by transferring work load so that it will balance work load between the nodes, as parallel beam search does the same thing as claimed to keep the number of nodes the same.( Richard, section 3.1 ).
Claims 3-4, 7, 9, and 12-20 are rejected under 35 U.S.C. 103 as being
unpatentable over Shuncheng (Shuncheng Liu1 , Han Su1 ,2* , Yan Zhao3 , Kai
Zeng4, Kai Zheng1 ,2*. 2021. Lane Change Scheduling for Autonomous Vehicle: A
Prediction-and-Search Framework. In Proceedings of the 27th ACM SIGKDD
Conference on Knowledge Discovery and Data Mining (KDD '21 ), August 14-18,
2021.) in view of Sriram (Sriram N N1, Buyu Liu, Francesco Pittaluga, and
Manmohan Chandraker. "SMART: Simultaneous Multiagent Recurrent Trajectory
Prediction." arXiv:2007 .13078v1, 26 Jul 2020),further in the view of Isele; David Francis (US 11242054 B2), further in the view of Richard (R. P. Ma, F-S. Tsung, and M-H. Ma. 1989. A dynamic load balancer for a parallel branch and bound algorithm. In Proceedings of the third conference on Hypercube concurrent computers and applications - Volume 2 (C3P)), further in the view of Siddiqui (US 20200353943).
As of claim 3, the modified model teaches all the limitation of claim 2, and Isele also teaches wherein generating a joint action that includes (i) the respective actions for the interactive agents and (ii) the respective actions for any simulated agents that are not designated as interactive agents (([0020], As described herein, subscript/superscript notation variable time agent, action may be utilized (i.e., a superscript may be agent or action related and a subscript may be time related). In a stochastic game, at time t each agent i in state st takes an action at i according to their policy πi. All the agents then transition to the state st+1 and receive a reward rt i. Stochastic games can be described as a tuple (S, A, P, R) where S is the set of states, and A={A1, . . . , An} is the joint action space consisting of the set of each agent's actions, where n is the number of agents.
[0029], To reduce the number of traffic participant actions, a target interactive agent may be selected and then non-interactive predictions may be assumed for the other traffic agents).
The modified model does not explicitly teach for each interactive agent: generating, from the respective state data at the particular time step, a policy input that characterizes the respective state of the simulated environment at the particular time step relative to the interactive agent; and processing the policy input using a policy neural network to generate a policy output that defines a next action to be performed by the interactive agent at the particular time step.
While Siddiqui teaches generating, from the respective state data at the particular time step, a policy input that characterizes the respective state of the simulated environment at the particular time step relative to the interactive agent;( paragraph 63, The machine learning network 130 includes policies that keep the dynamic objects (a.k.a., agents) from colliding and also from getting close to colliding. The system 100 may determine a policy over many different dynamic objects. The policy is a heuristic that suggests a particular set of actions in response to a current state of the agent (e.g., a state of a particular dynamic object) and the agent's environment (e.g., the road network topology and other dynamic objects)).
processing the policy input using a policy neural network to generate a policy
output that defines a next action to be performed by the interactive agent at the
particular time step (paragraph 63, In other words, the determined action for a
particular state of a dynamic object is what action the dynamic object should take based on the dynamic object's state and the dynamic object's surrounding environment. The policy maps the various states to particular actions to be taken by the dynamic object).
Siddiqui is considered to be analogous to the claimed invention because it focus on trajectory prediction of vehicles and providing a simulated environment. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Siddiqui teaching of using a machine learning networks including policies in order to create a policy output, on the modified teaching of interactive agent.
The motivation would have been is to allow autonomous vehicle to interact with one or more of the dynamic objects and to improves the overall score of the trajectories(Siddiqui, paragraph 64).
As of claim 4, the modified model teaches all the limitation claim 3 , and Shuncheng also teaches all of the simulated agents are designated as interactive agents (section 2.1, In this study, we consider an interactive environment where there are one autonomous vehicle A and a set of conventional vehicles C driving on a straight multi-lane road)
As of claim 7, the modified model teaches the limitation of claim 3, and Siddiqui also teaches: wherein the policy input comprises data representing a goal to be performed by the agent during the simulated trajectory( Paragraph 52, For each identified dynamic object, the system 100 determines the dynamic object's local goal (e.g., by determining whether the dynamic object is moving straight, turning right or left by a certain number of degrees). For example, the dynamic object's local goal may be the final position of the dynamic object at a particular time interval)
As of claim 9, the modified model teaches the limitation of claim 7, and Shuncheng also teaches the goal to be performed by the agent is a route to be traveled by the agent over the sequence of time steps in the simulated trajectory, and wherein the set of possible goals is a set of possible routes for the agent (section 4.1, Specifically, we categorize the impact situations of maneuver within radius R into three types: queuing, jumping the queue, and crossing. Figure 3 illustrates the three scenarios, where the red shaded area, called conflicting zone, is the location that both vehicles plan to arrive at the next time step.)
As of claim 12, the modified model teaches all the limitation of claim 3, and Shuncheng also teaches wherein determining a respective score for each partial trajectory that represents a likelihood that the partial trajectory satisfies one or more criteria for the simulated trajectories comprises: for each time step after a most recent pruning time step, determining a respective time step score for each interactive agent from the next state data that was added to the partial trajectory at the time step( section 4.2, At each time step, the maneuver tree will be reinitialized as a nine-complete tree with a fixed depth z, with the latest weights on each edge. First, based on the current trajectory point of the autonomous vehicle, some invalid nodes can be discarded based on the following rules: When the autonomous vehicle is in the rightmost lane, three direct child nodes Page 24 (denoting the behavior of 'Change right') of the current node will be removed, When the autonomous vehicle is in the leftmost lane, three direct child nodes (representing the behavior of 'Change left') of the current node will be removed). And aggregating the
respective time step scores for the interactive agents to determine the respective
score for the partial trajectory (section 4.2, After deleting the invalid points, we use
the adaptive beam search algorithm to search for the optimal path in the tree. Finally,
the tree will be updated to prepare for the next search process).
As of claim 13, the modified model teaches all the limitation of claim 12 and Shuncheng also teaches wherein aggregating the respective time step scores for the interactive agents to determine the respective score for each partial trajectory comprises: for each interactive agent, determining a first summary statistic of the time step scores for the interactive agent (section 4.2, Specifically, in i-th layer, We rank all the nodes based on their cumulative weights CW, i.e., summation of (V I A - F I im) where i E [t, t + i]. Then, we can select the highconfidence nodes (fromTop1 to TopK) as the candidate nodes based on y, i.e.,CWropK -CWTopK+1 ~ Y )
determining the respective score for the partial trajectory as a second summary
statistic of the first summary statistics for the interactive agents( section 4.2, Fig 4.
Adaptive beam search example. In fact, the width may depend on the weight distribution
of each layer, and thus should be predetermined more adaptively. For instance, in one layer, the weights are (0.01, 0.4, 0.5, 0.003), in another layer, the weights are (0.7, 0.6,
0.8, 0.9). If the algorithm fixes a small-sized width, it will ignore some valuable nodes
(0.7,0.6), leading to a local optimum solution. Alternatively, if the algorithm fixes a large sized width, the algorithm will consider excessive nodes (0.01 ,0.03)).
As of claim 14, the modified model teaches all the limitation of claim 13, and Shuncheng also teaches wherein the first summary statistic of the time step scores is the maximum or minimum of the time step scores and the second summary statistic of the summary statistics is the average of the first summary statistics (section 4.2, Each edge maintains a cumulative weight. In this case, the number of candidate nodes in each layer is different (4-6-7-5-5), following the threshold y , and the non-candidate nodes are pruned accordingly. The search result is a path of length z (z = 5) with the maximum cumulative weights of VA - Fim (yellow line), on Fig. 4).
As of claim 15, the modified model teaches all the limitation of claim 12, and Shuncheng also teaches wherein determining a respective time step score for each interactive agent from the next state data that was added to the partial trajectory at the time step comprises: generating from the next state data that was added to the partial trajectory at the time step a discriminator input for the interactive agent that characterizes the respective next state of the simulated environment relative to the interactive agent( section 3.2 , The input features X of the models are (xt-n+1, Xt-n+2, • • • , Xt) with the input window length n, where each x has 14 features: For the surrounding vehicles C1-6, we choose them based on the previous work [14], which have the most effect on a vehicle's motion. Each of them has two features: current lane number p I cq .Land relative longitudinal distance from the predicted vehicle d(p I co, p I cq ), where q E {1,2,3,4,5,6}).
While Shuncheng does not teach processing the discriminator input using a discriminator neural network and generating the time step score from the discriminator score.
However, Siddiqui teaches processing the discriminator input using a discriminator neural network that is configured to process the discriminator input to generate as output a discriminator score that represents a likelihood that the discriminator input was generated from state data characterizing an observed state of the real-world environment rather than a state of the simulated environment ( Paragraph 69, The machine learning network 130 acts as a generator of data. The system 100 using a discriminator may evaluate the generated data by comparing the generated data against the real data (i.e., the driving scenario data 122) to determine if the generated data looks similar to the real data. The system 100 may use the discriminator to compute a score indicating the extent to which the generated data looks realistic.)
generating the time step score from the discriminator score (Paragraph 69. The system 100, based on the computed scores, may perform a policy update to the machine learning network 130 such that the trajectories of dynamic objects are better enabled to achieve their goals. Over multiple GAN updates, newly generated trajectories created by the generator would look more and more like realistic trajectory data from the original driving scenario data 122. The system 100 may also update the discriminator to better separate the trajectories from the real trajectories.). According to the specification, the system then generates the time step score from the discriminator score, e.g., by directly using the discriminator scores as the time step score or by applying a specified function to the discriminator scores, e.g., computing the logarithm of the discriminator score, computing the negative of the discriminator score, normalizing the discriminator score, and so on( paragraph 114) and at it cited above Siddiqui use a GAN and computed scores which is a time step score.
Siddiqui is considered to be analogous to the claimed invention because it focus on trajectory prediction of vehicles and providing a simulated environment. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Siddiqui teaching of using a machine learning networks including policies in order to create a policy output, on the modified teaching of interactive agent.
The motivation would have been is to allow autonomous vehicle to interact with
one or more of the dynamic objects and to improves the overall score of the
trajectories(Siddiqui, paragraph 64).
As of claim 16, the modified model teaches the limitation of claim 3, also Siddiqui teaches wherein the policy neural network has been trained through behavior cloning on at least training data generated from observed trajectories of agents in the real-world environment (paragraph 42, This situation exemplifies a prime example of a real-world driving scenario which may be used to train the machine learning network 130. And on paragraph 61, One method of training the machine learning network 130 is using imitation learning (e.g., supervised learning), where the system takes the five input signals 710, 720, 730, 750, 760 as inputs and determines the three output control signals 770, 780, 790).
As of claim 17, the modified model teaches all the limitation of claim 16, and Siddiqui also teaches wherein the policy neural network has been trained through behavior cloning on training data generated from observed trajectories of agents in the real-world environment ( paragraph 29, Based on the trained machine learning network 130, the simulation module 108 generates a real-world simulated driving environment for training self-driving cars with multiple driving scenarios, and on paragraph 65, the system 100 may identify some or all of the dynamic objects of a particular type and find the trajectories for that type of dynamic object and center on the object and train the machine learning network 130 with those agents) and training data generated from simulated trajectories (Paragraph 73 , The system 100 may use the additional driving scenario data to train the machine learning network 130. For example, the system 100 may perturb the current trajectories of the dynamic objects in the driving scenario data 122. The system 100 may change a dynamic object to move faster or slower along its trajectory).
As of claim 18, the modified model teaches all the limitation of claim 3, and Siddiqui also teaches wherein the policy neural network has been trained through model-based generative adversarial imitation learning (MGAIL) on training data generated from observed trajectories of agents in the real-world environment and training data generated from simulated trajectories( paragraph 61, One method of training the machine learning network 130 is using imitation learning (e.g., supervised learning) ... Also, the machine learning network 130 may be trained through reinforcement learning methods. Reinforcement learning is where the system 100 implicitly learns the control signals by achieving goals from a high-level objective function. In the case of reinforcement learning, the system 100 creates a driving scenario and plays the scenario forward. The system 100 may combine different imitation, reinforcement, and Generative Adversarial Network (GAN) learning methods to train the machine learning network 130). As stated on claim 17 the training data are from real world and it also generated from the simulated trajectories).
As of claim 19, the modified model teaches all the limitation of claim 18, and Siddiqui also teaches wherein a discriminator neural network has been trained through MGAIL jointly with the policy neural network( paragraph 19, Additionally, the system 100 may use a Generative Adversarial Network (GAN) and perform a GAN update. A GAN is a process to a make a generative model by having two machine learning networks try to compete with one another. A discriminator tries to distinguish real data from unrealistic data created by a generator. The generator uses random noise or data perturbations to create imitations of the real data in an attempt to trick the discriminator into believing the data is realistic).
As of claim 20, the modified model teaches all the limitation of claim 19, and Siddiqui also teaches wherein the discriminator neural network has been trained to optimize a generative adversarial imitation learning objective on training data generated from observed trajectories of agents in the real-world environment and training data generated from simulated trajectories(Paragraph 69, The system 100 may evaluate a machine learning network 130 that was generated from the driving scenario data. The machine learning network 130 acts as a generator of data. The system 100 using a discriminator may evaluate the generated data by comparing the generated data against the real data (i.e., the driving scenario data 122))
Claims 6, and 22-26 are rejected under 35 U.S.C. 103 as being unpatentable over Shuncheng (Shuncheng Liu1 , Han Su1 ,2* , Yan Zhao3 , Kai Zeng4, Kai Zheng1 ,2*. 2021. Lane Change Scheduling for Autonomous Vehicle: A Prediction-and-Search Framework. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '21 ), August 14-18, 2021.) in view of Sriram (Sriram N N1, Buyu Liu, Francesco Pittaluga, and Manmohan Chandraker. "SMART: Simultaneous Multiagent Recurrent Trajectory Prediction." arXiv:2007 .13078v1, 26 Jul 2020),further in the view of Isele; David Francis (US 11242054 B2), further in the view of Siddiqui (US 20200353943).
As of claim 6, the modified model teaches the limitation of claim 5, and Isele also teaches wherein only a proper subset of the simulated agents are designated as interactive agents ([0029], To reduce the number of traffic participant actions, a target interactive agent may be selected and then non-interactive predictions may be assumed for the other traffic agents).
The modified model does not explicitly teach wherein, for each of one or more agents that are not an interactive agent, the joint action assigns to the agent the action performed by the corresponding agent in the reference environment at a time step in the reference trajectory that corresponds to the particular time step.
While Siddiqui teaches wherein, for each of one or more agents that are not an interactive agent, the joint action assigns to the agent the action performed by the corresponding agent in the reference environment at a time step in the reference trajectory that corresponds to the particular time step([0092], Additionally, the system 100 may detect a likely impact with another dynamic object based on the velocity and trajectory of the primary agent 810. When the system 100 determines that such a situation has occurred, the system 100 changes an otherwise non-reactive dynamic object to a reactive dynamic object thereby allowing the dynamic object to respond to the actions of the primary agent 810 . ... The dynamic object may perform an action based on its state and the environment.)
Siddiqui is considered to be analogous to the claimed invention because it focus on trajectory prediction of vehicles and providing a simulated environment. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Siddiqui teaching of assign a joint action for the corresponding agent on the modified model of non-interactive agents.
The motivation would have been is to allow autonomous vehicle to interact with
one or more of the dynamic objects and to improves the overall score of the
trajectories(Siddiqui, paragraph 64).
As of claim 22, the modified model teaches all the limitation of claim 1, but it does not explicitly teach wherein the state data characterizing the state of the environment at any given time step comprises (i) static scene features of the simulated environment, ii) dynamic scene features of the simulated environment at the given time step, iii) respective state features for each of the simulated agents at the given time step.
While Siddiqui teaches wherein the state data characterizing the state of the environment at any given time step comprises (i) static scene features of the simulated environment( paragraph 78, , Additionally, the system 100 may identify from the driving scenario data 122 that certain dynamic objects are quickly decelerating or decelerating and coming a complete stop. The system 100 may classify these driving scenarios as a possible accident. The system 100 then may train the machine learning network 130 as to how other dynamic objects respond to the road debris, vehicles swerving and/or vehicles quickly decelerating). ii) dynamic scene features of the simulated environment at the given time step (paragraph 38, the system 100 evaluates a dynamic object's velocity and direction by comparing a dynamic object in one video frame to the next video frame (i.e., from one time instant to the next time instant)). iii) respective state features for each of the simulated agents at the given time step( paragraph 38, The system 100 then may determine a dynamic object's velocity and direction from one time instant to another time instant and/or from an initial time instant over a series of time instants (e.g., over a 5 second time window).
Siddiqui is considered to be analogous to the claimed invention because it focus on trajectory prediction of vehicles and providing a simulated environment. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Siddiqui teaching of using a static dynamic scene features at a given time step, on the modified teaching of interactive agent.
The motivation would have been is to allow autonomous vehicle to interact with
one or more of the dynamic objects and to improves the overall score of the
trajectories(Siddiqui, paragraph 64).
As of claim 23, the modified model teaches all the limitation of claim 1, but the modified model does not teach wherein one of the simulated agents is controlled by control software for an autonomous vehicle while generating the simulated trajectories.
While Siddiqui teaches wherein one of the simulated agents is controlled by control software for an autonomous vehicle while generating the simulated trajectories( paragraph 29, The system 100 uses high definition (HD) base map data 124 along with driving scenarios to allow simulated control of vehicles by way of the AP I module 110. And on paragraph 81, he system 100 provides an API module 110 where external software, applications or other systems may interact with the system 100. Via the API module 110 the external system may experience driving in a simulated environment generated by the simulation module 108. The API module 110 of system 100 provides functions for sending and receiving data and instructions to and from the external system).
Siddiqui is considered to be analogous to the claimed invention because it focus on trajectory prediction of vehicles and providing a simulated environment. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Siddiqui teaching of a control software to control simulated agents on the modified model.
The motivation would have been is to allow autonomous vehicle to interact with
one or more of the dynamic objects and to improves the overall score of the
trajectories(Siddiqui, paragraph 64).
As of claim 24, the modified model teaches all the limitation of claim 23, and Siddiqui also teaches wherein one of the simulated agents are an ego agent, the state data for any given time step (Paragraph 53, For example, the system may determine for a particular time interval (t)+period of time (e.g., 1 second) a speed and steering angle or a velocity, or a serious of control points (i.e., a driving decision for the Ego vehicle at t+ period_ of_ time). comprises data that captured by sensors of the ego agent at the given time step, (paragraph 32, In addition to capturing aerial video, the UAVs, using on-board sensors, may capture atmospheric and other conditions about the environment in which the video was captured, and on paragraph 49, The system 100 evaluates driving scenario data 122 based on different inputs related to the Ego vehicle 612) and wherein the ego agent is controlled by the control software for the autonomous vehicle( paragraph 52, In training the machine learning network 130, with given inputs 610, 620, 630, 640 for the Ego vehicle 612, the system 100 may learn how to control the Ego vehicle 612 in such a way that that the Ego vehicle 612 achieves its high-level goal (e.g., the vehicle's intended destination).
As of claim 25, the modified model teaches all the limitation of claim 1, but it does not explicitly teach evaluating a performance of control software for an autonomous vehicle using the simulated trajectories.
While Siddiqui teaches evaluating a performance of control software for an autonomous vehicle using the simulated trajectories ( paragraph 99, to assess the performance of the primary agent 910 for a simulated driving session, the system 100 determines one or more values of a performance score for the autonomous vehicle 910. The system 100 may determine the performance by one more predefined calculations or heuristics. For example, the system 100 may determine the performance score by evaluating one or more: a safety value (e.g. the average distance between the primary agent and the primary agent's surrounding dynamic objects over the course of a driving scenario).
Siddiqui is considered to be analogous to the claimed invention because it focus on trajectory prediction of vehicles and providing a simulated environment. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Siddiqui teaching of evaluating a performance of control software for an autonomous vehicle using the simulated trajectories on the modified model.
The motivation would have been is to allow autonomous vehicle to interact with
one or more of the dynamic objects and to improves the overall score of the
trajectories(Siddiqui, paragraph 64).
As of claim 26, the modified model teaches all the limitation of claim 1, but it does not explicitly teach training one or more neural networks on training data generated from at least the plurality of simulated trajectories; deploying the one or more trained neural networks onboard an autonomous vehicle for use in controlling the autonomous vehicle as the vehicle navigates through the real-world environment
While Siddiqui teaches training one or more neural networks on training data generated from at least the plurality of simulated trajectories;(Fig 2, block 230 -240 and on paragraph 07, FIG. 2 illustrates a flowchart of an example process for training a machine learning model with driving scenarios). deploying the one or more trained neural networks onboard an autonomous vehicle for use in controlling the autonomous vehicle as the vehicle navigates through the real-world environment.(Paragraph 30, Now referring to FIG. 2, the figure illustrates a block diagram of an example process for training a machine learning network with driving scenario data and providing a simulated 3-dimensional driving
environment ... then the system 100 simulates interaction of an autonomous vehicle with the dynamic objects based on the trained machine learning network (block 250).
Siddiqui is considered to be analogous to the claimed invention because it focus on trajectory prediction of vehicles and providing a simulated environment. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Siddiqui teaching of training a neural network on training data and deploy the trained neural network on the autonomous vehicle on the modified model to control autonomous vehicles navigation on real world environment.
The motivation would have been is to allow autonomous vehicle to interact with
one or more of the dynamic objects and to improves the overall score of the
trajectories(Siddiqui, paragraph 64).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Shuncheng (Shuncheng Liu1 , Han Su1 ,2* , Yan Zhao3 , Kai Zeng4 , Kai Zheng1 ,2* . 2021. Lane Change Scheduling for Autonomous Vehicle: A Prediction-and-Search Framework. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '21), August 14-18, 2021) in view of Sriram (Sriram N N1, Buyu Liu, Francesco Pittaluga, and Manmohan Chandraker. "SMART: Simultaneous Multiagent Recurrent Trajectory Prediction." arXiv:2007.13078v1, 26 Jul 2020), further in the view of Isele; David Francis (US 11242054 B2), further in the view of Richard (R. P. Ma, F-S. Tsung, and M-H. Ma. 1989. A dynamic load balancer for a parallel branch and bound algorithm. In Proceedings of the third conference on Hypercube concurrent computers and applications - Volume 2 (C3P)),in the view of Siddiqui(US 20200353943). In view of Narayanan(US 20210276547), further in the view of Liu(US 11858536).
As of claim 8, the modified model teaches all the limitation of claim 7 but it does not explicitly teaches the limitations of claim 8, however Narayanan teaches for each interactive agent, generating from the data specifying the initial state a goal input for the agent that characterizes the initial state of the simulated environment relative to the interactive agent;(abstract, Trajectory scenarios are simulated based on the encoded training vectors, with each simulated trajectory scenario representing one or more agents with respective agent trajectories, to generate simulated training data).
Narayanan is considered to be analogous to the claimed invention because it focus on trajectory prediction of vehicles. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to apply Narayanan teaching of generating a goal input for the agent on to a modified model for each interactive agent.
The motivation would have been by using a machine learning it helps to predict the trajectory of multiple agents and this trajectories improve the diversity of the training data and capture scenarios that may occur rarely in recorded training data (Narayanan, [0017]).
While the modified model does not teach processing the goal input using a goal generating policy neural network generate a score.
However, Liu teaches processing the goal input using a goal generating policy neural network to generate a score distribution over a set of possible goals to be performed by the agent; and selecting the goal to be performed by the agent from the set of possible goals using the score distribution ( Col. 2 line 3 -15, The machine-learned model framework can use a total cost that is a function of the above-mentioned cost functions to determine a vehicle motion trajectory for execution by the autonomous vehicle. For example, a plurality of candidate vehicle trajectories can be scored according to the cost function(s). A trajectory for execution can be selected based on the scores (e.g., as having the best score, such as the lowest cost). In this manner, the autonomous vehicle can leverage example implementations of the joint prediction/planning operations of the presently disclosed machine-learned model framework to better predict how objects will interact with the autonomous vehicle and plan the vehicle's motion accordingly.)
Liu is considered to be analogous to the claimed invention, because it focus on trajectory prediction of vehicles. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Liu teaching of using the machine learned model framework of calculating score on the modified model selecting the goal to be performed by the agent.
The motivation would have been is to improve the ability and efficiency of the autonomous vehicle for completing complex maneuvers within an environment (Liu, Col.2 line 15-17).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Shuncheng (Shuncheng Liu1 , Han Su1 ,2* , Yan Zhao3 , Kai Zeng4 , Kai Zheng1 ,2* . 2021. Lane Change Scheduling for Autonomous Vehicle: A Prediction-and-Search Framework. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '21), August 14-18, 2021.) in view of Sriram (Sriram N N1, Buyu Liu, Francesco Pittaluga, and Manmohan Chandraker. "SMART: Simultaneous Multiagent Recurrent Trajectory Prediction." arXiv:2007.13078v1, 26 Jul 2020), further in the view of Isele; David Francis (US 11242054 B2), further in the view of Richard (R. P. Ma, F-S. Tsung, and M-H. Ma. 1989. A dynamic load balancer for a parallel branch and bound algorithm. In Proceedings of the third conference on Hypercube concurrent computers and applications - Volume 2 (C3P)), in the view of Siddiqui; (US 20200353943) further in the view of Wolff(US 11731653).
As of claim 10, the modified model teaches the limitation of claim 9 but it does not explicitly teach the limitations of claim 10, However Wolff teaches the route to be traveled is represented as a sequence of road graph lane segments starting at a lane segment corresponding to an initial state of the agent when the simulated environment is in the initial state(Col. 18 line 35 - 40, In general, the trajectory generator 1304 generates a set of candidate trajectories for the vehicle. Each candidate trajectory represents a path or route that can be traveled by the vehicle from an initial location (e.g., a start or current location) toward a destination location (e.g., an end or goal location).Col 19, line 15-25, For example, given information about the vehicle (e.g., current position, destination, velocity, etc.) and the vehicle's environment (e.g., map data and object data), the candidate trajectory generator 1304 generates a set of candidate trajectories 1400 that can be traveled by the vehicle toward its destination.)
Wolff is considered to be analogous to the claimed invention, because they focus on trajectory and motion prediction. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Wolff's teaching of generating a set of trajectories, which represents a path or route to the modified model to know the route is a sequence of road graph.
The motivation would have been is to improve safety and efficiency in vehicle
maneuvers. For example, by accounting for the effect of a trajectory on both the vehicle and other vehicles, a trajectory can be selected that maximizes efficiency and avoids collisions or other unsafe events for each actor (Wolff, Col.2 -3, line 66-69)
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Shuncheng (Shuncheng Liu1 , Han Su1 ,2* , Yan Zhao3 , Kai Zeng4 , Kai Zheng1 ,2* . 2021. Lane Change Scheduling for Autonomous Vehicle: A Prediction-and-Search Framework. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '21), August 14-18, 2021.) in view of Sriram (Sriram N N1, Buyu Liu, Francesco Pittaluga, and Manmohan Chandraker. "SMART: Simultaneous Multiagent Recurrent Trajectory Prediction." arXiv:2007.13078v1, 26 Jul 2020) further in the view of Isele; David Francis (US 11242054 B2), further in the view of Richard (R. P. Ma, F-S. Tsung, and M-H. Ma. 1989. A dynamic load balancer for a parallel branch and bound algorithm. In Proceedings of the third conference on Hypercube concurrent computers and applications - Volume 2 (C3P)), in the view of Siddiqui (US 20200353943). In view of Narayanan (US 20210276547), in the view of Liu (US 11858536), further in the view of Dolben (US 11731652).
As of claim 11, the modified model teach all the limitation of claim 8, but it does not explicitly teach the limitations of cliam11.
While Dolben teaches the goal generating policy neural network has been trained to match a distribution of goals occurring in training data generated from
observed trajectories of agents in the real-world environment( Col 6 line 14 -20,
For example, a machine learning model can be trained based on training data that
includes trajectories of vehicles encountered in real-world scenarios. Based on the
training data, the machine learning model can be trained to generate candidate
trajectories or select candidate trajectories that are similar to those trajectories of
vehicles encountered in real world scenarios.)
Dolben is considered to be analogous to the claimed invention
Because they focus on trajectory prediction. Therefore, it would be obvious to one of the ordinary skills in the art before the effective filling date to have applied Dolben teaching of training a machine learning model based on real-world scenario into the modified to generating policy neural network.
The motivation would have been is to improved computer simulation of realistic
agent behavior in various virtual environments, such as scenarios. In various embodiments, behavior of an agent (e.g., a simulated dynamic object, a simulated vehicle, etc.(Dolben, Col.5 line 20 -24)
Conclusion
The prior art made of record and not relied upon is considered pertinent to
applicant's disclosure.
Kobilarov; Marin (US 10671076 B1, Date Published 2020-06-02), is analogous to the invention, since Kobilarov teaches techniques for generating trajectories for autonomous vehicles and for predicting trajectories for third-party objects using temporal logic and tree search. They also use Static and dynamic object data as input.
Levinson; Jesse Sol (US 20170132334 A 1, Date Published 2017-05-11), is analogous to the invention since Levinson teaches systems, devices, and methods are configured to simulate navigation of autonomous vehicles in various simulated environments and simulating a predicted response of a data representation of a simulated autonomous vehicle.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABRHAM A. TAMIRU whose telephone number is (571)272-6987. The examiner can normally be reached Monday - Friday 8:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at 571 272 4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABRHAM ALEHEGN TAMIRU/Examiner, Art Unit 2188
/RYAN F PITARO/Supervisory Patent Examiner, Art Unit 2188
Prosecution Insights