Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Detailed Action
The following action is in response to the communication(s) received on 07/24/2026.
As of the claims filed 07/24/2026:
Claims 1, 9, and 17-20 have been amended.
Claims 1-20 are pending.
Claims 1 and 17 are independent claims.
Response to Arguments
Applicant’s arguments filed 07/24/2026 have been fully considered, but are not fully persuasive. Examiner’s response is set forth below.
With respect to the claim interpretation under 35 USC § 112(f), the amendments no longer invoke 35 USC § 112(f), thus the claim interpretation has been withdrawn.
With respect to the indefiniteness rejection under 35 USC § 112, the amendments have overcome the rejection, which thus has been withdrawn.
With respect to the patent eligibility rejections under 35 USC § 101:
Applicant asserts that the claims provide a technical solution for vehicle control using the identified abstract ideas of reducing the probability tree (p.11 last ¶). Examiner respectfully submits that autonomous control of the vehicle based on the automated decision is well-understood, routine, or conventional (see Chen et al., “Driving Maneuvers Prediction Based Autonomous Driving Control by Deep Monte Carlo Tree Search”, p.1 right last ¶) and thus cannot provide significantly more than the abstract ideas.
Applicant further asserts that calculating an updated entropy and removing at least one node cannot be practically performed in the human mind (p.12 ¶2). Examiner respectfully submits that the analysis requires the Examiner to consider whether the mental steps can be performed with aid of a pen and paper; as removing a node from a probability tree structure as currently recited can be performed with aid of pen and paper, these mental steps remain practical to perform in the human mind.
Applicant further asserts that the amended limitations “generating a probability tree structure based on information obtained by one or more vehicle sensors regarding an environment associated with a vehicle, associating entropy-based node structural values with nodes of the probability tree, selectively removing a least one node from the probability tree, recalculating entropy values for upstream nodes, determining an automated vehicle decision using an optimization algorithm based on calculated
trajectories through the reduced probability tree, and automatically controlling the vehicle based on that decision” cannot be practically performed in the human mind (p.12 last ¶) Examiner respectfully submits that the details regarding the vehicle environment are merely generally linked to the identified abstract idea of generating a probability tree structure; removing nodes, recalculating entropy, and determining a decision are abstract ideas (as explained above); the decision being based on automated vehicle control is also generally linked to the determining step; and controlling the vehicle based on the decision is well-understood, routine, or conventional (as explained above).
Applicant further asserts that the sensors of an autonomous vehicle architecture render the abstract ideas impractical to perform in the human mind. However, the sensors merely provide the data for the MCTS algorithm, which is an abstract idea. Additionally, vehicle control based on the MCTS algorithm is well-understood, routine, or conventional, as explained above.
Applicant further asserts that the abstract ideas being applied in automated vehicle architecture integrates the abstract ideas into a practical application (p.13 last ¶). Examiner respectfully submits that, as explained above, the sensors and vehicle control are generally linked to the abstract idea of pruning nodes in a MCTS algorithm in the instant claims.
Applicant further asserts that the instant claims improve autonomous vehicles with determining a reduced probability tree and thus an improvement to technology (p.14 ¶2). Examiner respectfully submits that the improvements are directed towards the abstract ideas, not an automated vehicle, of determining a pruned MCTS tree via a particular node removal step.
Applicant further asserts that the instant claims provide a real-world technological result and cannot be dismissed as extra-solution activities (p.14 last ¶), as the automated vehicle decision is using an optimization algorithm based on a calculation using the reduced probability tree structure (p.15 ¶1). Examiner respectfully disagrees, as the crux of the instant claims are not towards a particular vehicle control architecture, but merely reducing the size of a probability tree structure, which is an algorithm and thus an abstract idea.
Applicant further asserts that the automated vehicle decisions are integral aspects of the claims since the reduced probability tree directly drives the automate vehicle (p.15 ¶1b). However, as explained with the support provided above showing Berkheimer evidence, autonomous vehicle control using an algorithm is well-understood, routine, or conventional, and thus cannot be significantly more than the abstract idea itself.
Applicant further asserts that the claimed limitations in combination provide significantly more than the abstract ideas (p.16 last ¶). Examiner respectfully submits that, as explained with the support provided above showing Berkheimer evidence, autonomous vehicle control using an algorithm is clearly identified as well-understood, routine, or conventional, and thus cannot be significantly more than the abstract idea itself.
Thus, the claims remain ineligible.
With respect to the rejections under 35 USC § 102:
Applicant asserts that Dugueperoux does not teach the amended claims of vehicle control (p.17 ¶4). The argument has been considered but is moot in view of the new rejection under 35 USC § 103, where the claims are unpatentable under Dugueperoux in view of Ou further teaching applying a MCTS algorithm in autonomous vehicle control; see the new art rejection below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites a method, thus a process, one of the four statutory categories of patentable subject matter (Step 1). However, Claim 1 further recites:
generating a probability tree structure with a plurality of nodes…, wherein at least one node structural value is associated with each of the plurality of nodes, and the at least one node structural value quantifies an entropy of a subtree extending from a corresponding one of the plurality of nodes, which is an evaluation or judgement that can be performed in the human mind;
calculating an updated entropy for each of the plurality of nodes upstream from the removed node, which is an evaluation or judgement that can be performed in the human mind;
removing at least one node of the plurality of nodes from the probability tree structure according to the parameter, which is an evaluation or judgement that can be performed in the human mind;
and determining, with an optimization algorithm, an automated decision for the vehicle based on a plurality of calculated trajectories through the reduced probability tree structure when starting at a root node of the reduced probability tree structure…, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1.
Under Step 2A Prong 2, the claim recites:
based on information obtained by one or more vehicle sensors about an environment associated with a vehicle, which merely specifies the particular field of use or particular technological environment in which the abstract idea is to be performed, which by MPEP 2106.05(h) cannot integrate the abstract idea into a practical application;
receiving at least one parameter for removing one or more nodes of the probability tree structure, which is merely an insignificant extra-solution activity of data gathering, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application;
outputting a reduced probability tree structure without the removed node and with the updated entropy for each of the plurality of nodes upstream from the removed node, which is merely an insignificant extra-solution activity of data output, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application.
and automatically controlling the vehicle based on the automated decision, which is merely an insignificant extra-solution activity of autonomous control of a vehicle, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the activity of data gathering (MPEP 2106.05(g)) cannot provide significantly more, as storing and retrieving information in memory is well understood, routine, and conventional (MPEP 2106.05(d)(II)(iv)); the particular field of use or particular technological environment (MPEP 2106.05(h)) cannot provide significantly more; the activity of data output (MPEP 2106.05(g)) cannot provide significantly more, as receiving or transmitting data over a network is well understood, routine, and conventional (MPEP 2106.05(d)(II)(i), buySAFE, Inc. v. Google, Inc); and the activity of autonomous vehicle control is well-understood, routine, or conventional (Chen et al., “Driving Maneuvers Prediction Based Autonomous Driving Control by Deep Monte Carlo Tree Search”, p.1 right last ¶: “A typical…autonomous driving controller consists of two basic modules, a perception module and a control module. The perception module extracts useful features from the input images to locate the vehicle on the track. The control module generates control signals to keep the vehicle following a desired trajectory.”) The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 2, dependent on 1, further recites
calculating the updated entropy for each of the plurality of nodes upstream from the removed node includes updating a visit count for each of the plurality of nodes upstream from the removed node, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 3, dependent on 1, further recites
removing the at least one node of the plurality of nodes from the probability tree structure includes removing the at least one node and an entire subtree of the probability tree structure downstream of the at least one node, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 4, dependent on 1, further recite no additional abstract ideas. However: Under Step 2A Prong 2, the claim recites:
receiving at least one parameter for removing one or more nodes of the probability tree structure includes receiving at least one defined node to remove from the probability tree structure, which is merely an insignificant extra-solution activity of data gathering, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the activity of data gathering (MPEP 2106.05(g)) cannot provide significantly more, as storing and retrieving information in memory is well understood, routine, and conventional (MPEP 2106.05(d)(II)(iv)). The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 5, dependent on 1, further recite no additional abstract ideas. However: Under Step 2A Prong 2, the claim recites:
receiving at least one parameter for removing one or more nodes of the probability tree structure includes receiving at least one removal criterion for removing one or more nodes from the probability tree structure, which is merely an insignificant extra-solution activity of data gathering, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the activity of data gathering (MPEP 2106.05(g)) cannot provide significantly more, as storing and retrieving information in memory is well understood, routine, and conventional (MPEP 2106.05(d)(II)(iv)). The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 6, dependent on 1, further recite. no additional abstract ideas. However: Under Step 2A Prong 2, the claim recites:
the at least one removal criterion includes at least one of a removal of all single child nodes, a removal of each parent node having less than a defined number of child nodes, and a removal of each node having less than a defined number of branches extending therefrom, which is merely an insignificant extra-solution activity of data gathering, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the activity of data gathering (MPEP 2106.05(g)) cannot provide significantly more, as storing and retrieving information in memory is well understood, routine, and conventional (MPEP 2106.05(d)(II)(iv)). The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 7, dependent on 5, further recites
one of the plurality of nodes is a parent node with a plurality of child nodes downstream of the parent node, which is merely a detail of an abstract idea (removing at least one node of the plurality of nodes);
determining an optimal set of the plurality of child nodes to remove based on the at least one removal criterion, which is an evaluation or judgement that can be performed in the human mind;
and removing the at least one node of the plurality of nodes from the probability tree structure includes removing the optimal set of the plurality of child nodes from the probability tree structure, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 8, dependent on 7, further recites
the optimal set of the plurality of child nodes includes one or more child nodes downstream of the parent node, which is merely a detail of an abstract idea (removing the optimal set of the plurality of child nodes).
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 9, dependent on 7, further recites
for each possible set of the plurality of child nodes for removal, calculating an updated entropy for each of the plurality of nodes upstream from the possible set assuming that the possible set is removed and calculating a tradeoff score based on the updated entropy and a size of the probability tree structure assuming that the possible set is removed, which is an evaluation or judgement that can be performed in the human mind;
and selecting the possible set of the plurality of child nodes for removal having a highest value of the tradeoff score as the optimal set of the plurality of child nodes to remove, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 10, dependent on 5, further recites
generating a list including a plurality of possible sets of one or more child nodes for removal based on the at least one removal criterion, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1.Under Step 2A Prong 2, the claim recites:
wherein each possible set has a tradeoff score based on an updated entropy for each of the plurality of nodes upstream from the possible set assuming that the possible set is removed and a size of the probability tree structure assuming that the possible set is removed, which merely specifies the particular field of use or particular technological environment in which the abstract idea is to be performed, which by MPEP 2106.05(h) cannot integrate the abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the particular field of use or particular technological environment (MPEP 2106.05(h)) cannot provide significantly more. The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 11, dependent on 10, further recites
generating the list includes adding a possible set of one or more child nodes for removal to the list only if the tradeoff score for the possible set is greater than a reference value, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 12, dependent on 10, further recites
generating the list includes adding a possible set of one or more child nodes for removal to the list only if the tradeoff score for the possible set is greater than a tradeoff score for a parent node associated with the one or more child nodes, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 13, dependent on 10, further recites
generating the list includes prioritizing the plurality of possible sets of one or more child nodes in a descending order based on their tradeoff scores, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 14, dependent on 13, further recites
removing the at least one node of the plurality of nodes from the probability tree structure includes selecting the possible set of one or more child nodes from the prioritized list having a highest value of the tradeoff score and removing the selected set of one or more child nodes from the probability tree structure, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 15, dependent on 14, further recites
updating the tradeoff score for each possible set after the selected set of one or more child nodes is removed from the probability tree structure, which is an evaluation or judgement that can be performed in the human mind.
Thus, the claim recites an abstract idea under Step 2A Prong 1. Under Step 2A Prong 2 and Step 2B, the claim recites no additional elements which could integrate the abstract idea into a practical application or provide significantly more than the abstract idea itself; thus, the claim remains ineligible.
Claim 16, dependent on 1, further recite no additional abstract ideas.
However: Under Step 2A Prong 2, the claim recites:
storing the reduced probability tree structure in a memory circuit, which is merely an insignificant extra-solution activity of data transfer, which by MPEP 2106.05(g) cannot integrate an abstract idea into a practical application.
Thus, the claim is directed towards and abstract idea.
Further, the additional element(s), alone or in combination, do not provide significantly more than the abstract idea itself, because the activity of data transfer (MPEP 2106.05(g)) cannot provide significantly more, as receiving or transmitting data over a network is well understood, routine, and conventional (MPEP 2106.05(d)(II)(i), buySAFE, Inc. v. Google, Inc). The combination of these additional elements does not provide an inventive concept; thus, the claim remains ineligible.
Claim 17 recites An automated system, thus a machine, one of the four statutory categories of patentable subject matter. However, Claim 17 recites comprising: a control module configured to perform precisely the abstract ideas and additional elements of Claim 1. Therefore, Step 2A Prong 1 analysis remains the same. As for Step 2A Prong 2 and Step 2B: performance on a computer cannot integrate an abstract idea into a practical application (Step 2A Prong 2) nor provide significantly more than the abstract idea itself (Step 2B) (MPEP 2106.05(f)), and thus Claim 17 is rejected as subject-matter ineligible for reasons set forth in the rejections of Claim 1.
Claim 18, dependent on Claim 17, also recites the system configured to perform precisely the methods of Claims 16, respectively. Thus, Claim 18 is rejected for reasons set forth in Claims 16, respectively.
Claim 19, dependent on Claim 17, also recites the system configured to perform precisely the methods of Claims 5 and 7 combined. Thus, Claim 19 is rejected for reasons set forth in Claims 5 and 7 combined.
Claim 20, dependent on Claim 19, also recites the system configured to perform precisely the methods of Claims 10,13, and 14 combined. Thus, Claim 20 is rejected for reasons set forth in Claims 10,13, and 14 combined.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Dugueperoux et al., "Pruning playouts in Monte-Carlo Tree Search for the game of Havannah" (hereinafter Dugueperoux), in view of Ou et al., “An Auxiliary Decision-Making Method for Autonomous Driving via Monte Carlo Tree Search” (hereinafter Ou).
Regarding Claim 1, Dugueperoux teaches:
A method for probability tree reduction…, the method comprising: generating a probability tree structure with a plurality of nodes…, wherein at least one node structural value is associated with each of the plurality of nodes, and the at least one node structural value quantifies an entropy of a subtree extending from a corresponding one of the plurality of nodes; (Dugueperoux [p.4 last ¶] More precisely, let s be a node and mi the possible moves from s, leading to the child nodes si. For the classic MCTS algorithm, we already store, in s, the number of winning playouts ws and the total number of playouts ns (after s was selected). For the RAVE improvement, we also store, in s and for each move mi, the number of winning playouts ws,si and the total number of playouts ns,si obtained by choosing the move mi. These “RAVE statistics” are updated during the backpropagation step and indicate the estimated quality of the moves already considered in the subtree (see Fig. 3)…
PNG
media_image1.png
285
840
media_image1.png
Greyscale
[p.5 last ¶] PoolRave is an extension of RAVE [25,17]. The idea is to use the RAVE statistics to bias the simulation step (unlike the RAVE improvement which biases the selection step). More precisely, when a playout is performed, the PoolRave improvement firstly builds a pool of possible moves by selecting the N best moves according to the RAVE statistics.) (Note: the RAVE statistics which are used to determine the quality of the selected move corresponds to the entropy of a subtree)
receiving at least one parameter for removing one or more nodes of the probability tree structure; (Dugueperoux [p.6 ¶3] We propose a new improvement of the MCTS algorithm, called “Playout Pruning with Rave” (PPR). The idea is to prune bad moves in the simulation step in order to focus the simulation on good playouts (see Fig. 4, left). More precisely, before the playout, we compute a list of good moves by pruning the moves which have a winning rate lower than a given threshold Tw. The winning rate of a node j is computed using the RAVE statistics of a node sPPR, with wsPPR,j / nsPPR,j.
The node sPPR, giving the RAVE statistics, has to be chosen carefully. Indeed, the node s2, selected during the selection step of the MCTS algorithm, may still have very few playouts, hence inaccurate RAVE statistics. To solve this problem, we traverse the MCTS tree bottom-up, starting from s2, until we reach a node with a minimum ratio Tn, representing the current number of playouts for sPPR over the total number of playouts performed.
[p.7 algorithm 1, pruning step]
PNG
media_image2.png
653
852
media_image2.png
Greyscale
) (Note: the condition identifying lower threshold Tw corresponds to the parameter for removing the node)
removing at least one node of the plurality of nodes from the probability tree structure according to the parameter; (Dugueperoux [p.6 ¶3] We propose a new improvement of the MCTS algorithm, called “Playout Pruning with Rave” (PPR). The idea is to prune bad moves in the simulation step in order to focus the simulation on good playouts (see Fig. 4, left). More precisely, before the playout, we compute a list of good moves by pruning the moves which have a winning rate lower than a given threshold Tw. The winning rate of a node j is computed using the RAVE statistics of a node sPPR, with wsPPR,j / nsPPR,j.)
calculating an updated entropy for each of the plurality of nodes upstream from the removed node; (Dugueperoux [p.7 ¶2] After the PPR list is computed, the simulation step is performed. The idea is to use the moves in the PPR list, which are believed to be good, but we also have to choose other moves to explore other possible playouts. To this end, during the simulation step, each move is chosen in the PPR list with a probability p, or among the possible moves with a probability 1 − p. In the latter case, we have observed that considering only a part of all the possible moves gives better results; this can be seen as a default pruning with, in return, an additional bias (see Algorithm 1)) (Note: the PPR list and other moves corresponds to the updated entropy)
and outputting a reduced probability tree structure without the removed node and with the updated entropy for each of the plurality of nodes upstream from the removed node. (Dugueperoux [p.7 algorithm 1] “return best child of s0”) (Note: the resulting best child from the MCTS pruning and training corresponds to the output reduced probability tree structure)
Dugueperoux does not teach, but Ou further teaches:
…for… vehicle control…, based on information obtained by one or more vehicle sensors about an environment associated with a vehicle; (Ou [p.2 right last ¶] CARLA platform provides some manually designed decision making approaches, these approaches are designed directly from humans’ experiences. We use them as the basic methods to control the ego vehicle. [p.3 right ¶3] Besides, a state also contains the observed location information of other nearby traffic participants, which is from the RGB camera of CARLA.)
and determining, with an optimization algorithm, an automated decision for the vehicle based on a plurality of calculated trajectories through the reduced probability tree structure when starting at a root node of the reduced probability tree structure and automatically controlling the vehicle based on the automated decision (Ou [p.4 right ¶1] After reaching the time cap, our MCTS based method chooses the action 𝑎 with the maximum 𝑄(𝑠,𝑎). And the correction mechanism will give the final action to control the ego vehicle. [p.5 left ¶1] The results are shown in Figure 5, we can see that MCTS with pruning has the minimum collision times and the minimum time cost. Summarily, the proposed MCTS based auxiliary method can help the ego vehicle make a better decision, besides, the pruning rule can reduce the computation cost significantly, which makes the ego vehicle’s decision stronger. [p.3 left ¶1] In CARLA, road is represented as a mesh topology composed of many nodes and directed edges, as shown in Figure 3. The driving route of the car is a sequence with several such nodes, which named waypoints. The ego vehicle’s staying straight or changing lane actions can be approximately expressed as: drive towards the next waypoints on the current lane or the left/right lane.)
Ou and Dugueperoux are analogous to the present invention because both are from the same field of endeavor of MCTS-based methods. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to implement the MCTS-based control of an autonomous vehicle from Ou into Dugueperoux’s MCTS pruning method. The motivation would be to “MCTS with pruning has the minimum collision times and the minimum time cost” (Ou [p.5 left ¶1]).
Regarding Claim 2, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 1, wherein calculating the updated entropy for each of the plurality of nodes upstream from the removed node includes updating a visit count for each of the plurality of nodes upstream from the removed node. (Dugueperoux [p.6]
PNG
media_image3.png
817
858
media_image3.png
Greyscale
) (Note: discarding the moves using the PPR process corresponds to updating the entropy for the upstream nodes of the removed node; each playout corresponds to a visit count)
Regarding Claim 3, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 1, wherein removing the at least one node of the plurality of nodes from the probability tree structure includes removing the at least one node and an entire subtree of the probability tree structure downstream of the at least one node. (Dugueperoux [p.7 algorithm 1]
PNG
media_image4.png
697
657
media_image4.png
Greyscale
(Note: the pruning step and the returned best child of s0 corresponds to removing the entire subtree of the probability tree structure downstream of the node (s0))
Regarding Claim 4, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 1, wherein receiving at least one parameter for removing one or more nodes of the probability tree structure includes receiving at least one defined node to remove from the probability tree structure. (Dugueperoux [p.7 algorithm 1]
PNG
media_image4.png
697
657
media_image4.png
Greyscale
(Note: the pruning step removes a node of the expanded s1, thus corresponding to the algorithm receiving a defined node to remove from the structure)
Regarding Claim 5, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 1, wherein receiving at least one parameter for removing one or more nodes of the probability tree structure includes receiving at least one removal criterion for removing one or more nodes from the probability tree structure. (Dugueperoux [p.7 algorithm 1]
PNG
media_image4.png
697
657
media_image4.png
Greyscale
(Note: PPR keeping j only if the RAVE score is higher than the threshold Tw’ corresponds to the removal criterion)
Regarding Claim 6, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 5. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 5, wherein the at least one removal criterion includes at least one of a removal of all single child nodes, a removal of each parent node having less than a defined number of child nodes, and a removal of each node having less than a defined number of branches extending therefrom. (Dugueperoux [p.7 algorithm 1]
PNG
media_image4.png
697
657
media_image4.png
Greyscale
(Note: the pruning step removes a child node, thus corresponding to removing a single child node)
Regarding Claim 7, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 5. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 5, wherein: one of the plurality of nodes is a parent node with a plurality of child nodes downstream of the parent node; (Dugueperoux [p.7 algorithm 1]
PNG
media_image5.png
697
593
media_image5.png
Greyscale
(Note: the child nodes of s1 correspond to the plurality of child nodes downstream of the parent node)
the method further comprises determining an optimal set of the plurality of child nodes to remove based on the at least one removal criterion; and removing the at least one node of the plurality of nodes from the probability tree structure includes removing the optimal set of the plurality of child nodes from the probability tree structure. (Dugueperoux, Alg. 1, {pruning step}
PNG
media_image6.png
160
348
media_image6.png
Greyscale
) (Note: the inequality for the updated PPR removes the node from the structure, thus corresponding to the removal criterion; the total number of child nodes pruned from sPPR corresponds to the optimal set of the plurality of child nodes to remove)
Regarding Claim 8, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 7. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 7, wherein the optimal set of the plurality of child nodes includes one or more child nodes downstream of the parent node. (Dugueperoux, Alg. 1, {expansion}; {pruning step}; the expansion of s1 corresponds to the set of pruned child nodes from sPPR being downstream of the parent node (s1))
Regarding Claim 9, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 7. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 7, wherein determining the optimal set of the plurality of child nodes to remove includes: for each possible set of the plurality of child nodes for removal, calculating the updated entropy for each of the plurality of nodes upstream from the possible set assuming that the possible set is removed… (Dugueperoux [p.7 ¶2] After the PPR list is computed, the simulation step is performed. The idea is to use the moves in the PPR list, which are believed to be good, but we also have to choose other moves to explore other possible playouts. To this end, during the simulation step, each move is chosen in the PPR list with a probability p, or among the possible moves with a probability 1 − p. In the latter case, we have observed that considering only a part of all the possible moves gives better results; this can be seen as a default pruning with, in return, an additional bias (see Algorithm 1)) (Note: the PPR list and other moves corresponds to the updated entropy)
… and calculating a tradeoff score based on the updated entropy and a size of the probability tree structure assuming that the possible set is removed; (Dugueperoux [p.4 last ¶] For the RAVE improvement, we also store, in s and for each move mi, the number of winning playouts ws,si and the total number of playouts ns,si obtained by choosing the move mi.
[p.6 ¶3] More precisely, before the playout, we compute a list of good moves by pruning the moves which have a winning rate lower than a given threshold Tw. The winning rate of a node j is computed using the RAVE statistics of a node sPPR, with w’sPPR,j / n’sPPR,j .
[Alg.1]
PNG
media_image7.png
693
594
media_image7.png
Greyscale
) (Note: (w’sPPR,j / n’sPPR,j) (hereinafter RAVE) > Tw’ implies 1- RAVE > 1-Tw’. Thus, corresponds to the tradeoff score since it is a ratio of winning playouts and total playouts, and the node is removed only if this 1- RAVE is higher than the set threshold 1-Tw’ (thus assuming that the possible set is removed))
and selecting the possible set of the plurality of child nodes for removal having a highest value of the tradeoff score as the optimal set of the plurality of child nodes to remove. (Dugueperoux [Alg.1]
PNG
media_image7.png
693
594
media_image7.png
Greyscale
) (Note: the final PPR removes all the values less than the threshold, thus corresponding to the optimal set of the plurality of child nodes removed)
Regarding Claim 10, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim . Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 5, further comprising generating a list including a plurality of possible sets of one or more child nodes for removal based on the at least one removal criterion, (Dugueperoux, Alg. 1, {pruning step}
PNG
media_image6.png
160
348
media_image6.png
Greyscale
) (Note: the inequality for the updated PPR removes the node from the structure, thus corresponding to the removal criterion; the total number of child nodes pruned from sPPR corresponds to the optimal set of the plurality of child nodes to remove)
wherein each possible set has a tradeoff score based on an updated entropy for each of the plurality of nodes upstream from the possible set assuming that the possible set is removed and a size of the probability tree structure assuming that the possible set is removed. (Dugueperoux [p.4 last ¶] For the RAVE improvement, we also store, in s and for each move mi, the number of winning playouts ws,si and the total number of playouts ns,si obtained by choosing the move mi.
[p.6 ¶3] More precisely, before the playout, we compute a list of good moves by pruning the moves which have a winning rate lower than a given threshold Tw. The winning rate of a node j is computed using the RAVE statistics of a node sPPR, with w’sPPR,j / n’sPPR,j .
[Alg. 1, {pruning step}]
PNG
media_image6.png
160
348
media_image6.png
Greyscale
) (Note: each iteration of the expanded child node has their respective RAVE statistics and thus corresponds to each possible set having their respective tradeoff score)
Regarding Claim 11, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 10. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 10, wherein generating the list includes adding a possible set of one or more child nodes for removal to the list only if the tradeoff score for the possible set is greater than a reference value. (Dugueperoux [p.4 last ¶] For the RAVE improvement, we also store, in s and for each move mi, the number of winning playouts ws,si and the total number of playouts ns,si obtained by choosing the move mi.
[p.6 ¶3] More precisely, before the playout, we compute a list of good moves by pruning the moves which have a winning rate lower than a given threshold Tw. The winning rate of a node j is computed using the RAVE statistics of a node sPPR, with w’sPPR,j / n’sPPR,j .
[Alg. 1, {pruning step}]
PNG
media_image6.png
160
348
media_image6.png
Greyscale
) (Note: 1-RAVE corresponds to the tradeoff score; the node is added for removal if 1-RAVE > 1-Tw’; thus, 1-Tw’ corresponds to the reference value)
Regarding Claim 12, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 10. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 10, wherein generating the list includes adding a possible set of one or more child nodes for removal to the list only if the tradeoff score for the possible set is greater than a tradeoff score for a parent node associated with the one or more child nodes. (Dugueperoux [p.4 last ¶] For the RAVE improvement, we also store, in s and for each move mi, the number of winning playouts ws,si and the total number of playouts ns,si obtained by choosing the move mi.
[p.6 ¶3] More precisely, before the playout, we compute a list of good moves by pruning the moves which have a winning rate lower than a given threshold Tw. The winning rate of a node j is computed using the RAVE statistics of a node sPPR, with w’sPPR,j / n’sPPR,j .
[Alg. 1, {pruning step}]
PNG
media_image6.png
160
348
media_image6.png
Greyscale
) (Note: each expanded s2 corresponds to each additional set of child node; 1-RAVE corresponds to the tradeoff score; the node is added for removal if 1-RAVE > 1-Tw’; pruning removes the child nodes and thus the total playouts (n’sPPR,j), thus resulting in a greater tradeoff score than not removing the child node.)
Regarding Claim 13, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 10. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 10, wherein generating the list includes prioritizing the plurality of possible sets of one or more child nodes in a descending order based on their tradeoff scores. (Dugueperoux [p.5 last ¶] PoolRave is an extension of RAVE [25,17]. The idea is to use the RAVE statistics to bias the simulation step (unlike the RAVE improvement which biases the selection step). More precisely, when a playout is performed, the PoolRave improvement firstly builds a pool of possible moves by selecting the N best moves according to the RAVE statistics.) (Note: selecting the N best moves is equivalent to selecting All moves – N best moves (i.e. M worst moves) to prioritizing the nodes in a descending order based on the tradeoff scores)
Regarding Claim 14, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 13. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 13, wherein removing the at least one node of the plurality of nodes from the probability tree structure includes selecting the possible set of one or more child nodes from the prioritized list having a highest value of the tradeoff score and removing the selected set of one or more child nodes from the probability tree structure. (Dugueperoux [p.5 last ¶] PoolRave is an extension of RAVE [25,17]. The idea is to use the RAVE statistics to bias the simulation step (unlike the RAVE improvement which biases the selection step). More precisely, when a playout is performed, the PoolRave improvement firstly builds a pool of possible moves by selecting the N best moves according to the RAVE statistics.
[p.4 last ¶] For the RAVE improvement, we also store, in s and for each move mi, the number of winning playouts ws,si and the total number of playouts ns,si obtained by choosing the move mi.
[p.6 ¶3] More precisely, before the playout, we compute a list of good moves by pruning the moves which have a winning rate lower than a given threshold Tw. The winning rate of a node j is computed using the RAVE statistics of a node sPPR, with w’sPPR,j / n’sPPR,j .
[Alg. 1, {pruning step}]
PNG
media_image6.png
160
348
media_image6.png
Greyscale
) (Note: each expanded s2 corresponds to each additional set of child node; 1-RAVE corresponds to the tradeoff score; the node is added for removal if 1-RAVE > 1-Tw’; pruning removes the child nodes and thus the total playouts (n’sPPR,j), thus resulting in a greater tradeoff score than not removing the child node; selecting the N best moves is equivalent to selecting All moves – N best moves (i.e. M worst moves) to prioritizing the nodes in a descending order based on the tradeoff scores)
Regarding Claim 15, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 14. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 14, further comprising updating the tradeoff score for each possible set after the selected set of one or more child nodes is removed from the probability tree structure. (Dugueperoux [p.4 last ¶] For the RAVE improvement, we also store, in s and for each move mi, the number of winning playouts ws,si and the total number of playouts ns,si obtained by choosing the move mi.
[p.6 ¶3] More precisely, before the playout, we compute a list of good moves by pruning the moves which have a winning rate lower than a given threshold Tw. The winning rate of a node j is computed using the RAVE statistics of a node sPPR, with w’sPPR,j / n’sPPR,j .
[Alg. 1, {pruning step}]
PNG
media_image6.png
160
348
media_image6.png
Greyscale
) (Note: each expanded s2 after pruning j results in an updated total playout (n’sPPR,j), thus corresponding to updating the tradeoff score for the successive possible set)
Regarding Claim 16, Dugueperoux/Ou respectively teaches and incorporates the claimed limitations and rejections of Claim 1. Dugueperoux, via Dugueperoux/Ou, further teaches:
The method of claim 1, further comprising storing the reduced probability tree structure in a memory circuit. (Dugueperoux [p.11 ¶3] Experiments presented in this paper were carried out using the CALCULCO computing platform, supported by SCOSI/ULCO (Service Commun du Syst`eme d’Information de l’Universit´e du Littoral Cˆote d’Opale).) (Note: carrying out experiments on a computing platform requires storing the probability tree structure in a memory circuit)
Independent Claim 17 recites An automated system comprising: a control module configured (Dugueperoux [p.11 ¶3] Experiments presented in this paper were carried out using the CALCULCO computing platform, supported by SCOSI/ULCO (Service Commun du Syst`eme d’Information de l’Universit´e du Littoral Cˆote d’Opale).) (Note: carrying out experiments on a computing platform requires storing the probability tree structure in a memory circuit and thus a control module) to perform precisely the methods of Claim 1. Thus, Claim 17 is rejected for reasons set forth in Claim 1.
Claim 18, dependent on Claim 17, also recites the system configured to perform precisely the methods of Claims 16, respectively. Thus, Claim 18 is rejected for reasons set forth in Claims 16, respectively.
Claim 19, dependent on Claim 17, also recites the system configured to perform precisely the methods of Claims 5 and 7 combined. Thus, Claim 19 is rejected for reasons set forth in Claims 5 and 7 combined.
Claim 20, dependent on Claim 19, also recites the system configured to perform precisely the methods of Claims 10,13, and 14 combined. Thus, Claim 20 is rejected for reasons set forth in Claims 10,13, and 14 combined.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSEP HAN whose telephone number is (703)756-1346. The examiner can normally be reached Mon-Fri 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached on (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.H./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights