DETAILED ACTION
This action is responsive to the amendment filed on 07/13/2026. Claims 10-14, 16-18 and 21-25 are pending in the case. Claims 10, 16, and 17 are independent claims. Claims 21-26 have been added. Claims 11-14 and 16-18 have been amended and claims 1-9, 15 and 19-20 have been cancelled.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 04/10/2026 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 11-13, 21-22 and 24 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Dependent claims must be further limiting. Claims recite the limitation of when making part of the limitations contingent limitation with no guarantee that the limitations after when are going to be present when later referenced. This causes claims that reference the limitations cited after when to be improper as the references limitations may not exist in all embodiments under the broadest reasonable interpretation making the dependent claims not further limiting. Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements
Claim 11 recites the limitation when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes. Specifically, there has not been a respective current state of the respective control pass was not previously reached declared and, as the claims are written, there is no guarantee that there will be the required control passes determined to not have been reached. This renders the cited claim language for the dependent claim as not further limiting.
Claim 12 recites the limitation when the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes. Specifically, there has not been a when the respective current state of the respective control pass was previously reached declared and, as the claims are written, there is no guarantee that there will be the required control passes determined to have been reached. This renders the cited claim language for the dependent claim as not further limiting.
Claim 13 recites the limitation wherein for each of at least some of the control passes for which the respective current state was determined not to have been previously reached during any of the multiple reinforcement learning training passes and the at least some of the control passes. Specifically, there has not been a control passes for which the respective current state was determined not to have been previously reached declared and, as the claims are written, there is no guarantee that there will be the required control passes determined to not have been reached. This renders the cited claim language for the dependent claim as not further limiting.
Claim 21 and 22 recites the limitation for each of the one or more control passes for which the respective current state was determined to have been previously reached during any of the multiple reinforcement learning training passes. Specifically, there has not been a one or more control passes for which the respective current state was determined to have been previously reached declared and, as the claims are written, there is no guarantee that there will be the required control passes determined to have been reached. This renders the cited claim language for the dependent claim as not further limiting.
Claim 24 recites the limitation when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes. Specifically, there has not been a respective control pass was not previously reached declared and, as the claims are written, there is no guarantee that there will be the required control passes determined to not have been reached. This renders the cited claim language for the dependent claim as not further limiting.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 10-14, 16-18, 21-22 and 24 is/are rejected under 35 U.S.C. 103 as being unpatentable over Leonetti et al.(“A synthesis of automated planning and reinforcement learning for efficient, robust decision-making”, henceforth known as Leonetti) in view of Martinez et al, (“Relational Reinforcement Learning with Guided Demonstration”, henceforth known as Martinez) and further in view of Yang et al.(“PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making”, henceforth known as Yang).
Regarding claim 10:
Leonetti discloses A method of a robotic device for self-training a control strategy for itself using reinforcement learning performed by the robotic device navigating within an environment(Leonetti, Page 103, Abstract, “We demonstrate the effectiveness of the proposed method on a service robot, carrying out a variety of tasks in an office building” where the service robot is considered a robotic device and the method is considered reinforcement learning)
Leonetti discloses carrying out multiple reinforcement learning training passes(Leonetti, Page 111, Paragraph 7, “It has an incremental version of the solver designed specifically for planning, which efficiently concatenates these queries for different plan lengths retaining the fluents grounded at the previous iteration”) that each includes the robotic device transitioning from a respective starting state of the robotic device in the environment(Leonetti, Page 111, Paragraph 5, “Given an initial state and a goal, we want to compute all the plans of length less than a certain threshold” where the planning from initial state to goal is considered transitioning from a respecting starting state to a target state), to a respective target state of the robotic device in the environment, and one or more of which includes one or more respective intermediary states between the starting state and the target state(Leonetti, Page 109, Paragraph 5, “Lastly, planning has been combined with RL through reward shaping... Reward shaping is a technique to hasten Reinforcement Learning when the reward is sparse, and the agent has to execute a long sequence of actions before getting any feedback about its choices. The reward function is enriched by adding a term which provides feedback for intermediate states, helping guide the agent towards the goal” where Leonetti’s intermediate states between initial state and goal is considered intermediary states.)
Leonetti discloses for each respective one of the states of each of the reinforcement learning training passes prior to reaching the respective target state of the respective reinforcement learning training pass, the respective one of the states being a respective current state, a respective control pass is performed, while the robotic device is in the respective current state, for selecting a respective action for the robotic device to perform to transition out of the respective current state(Leonetti, Page 115, Paragraph 1, “The partial policy computed in the previous phase is used during the execution to constrain the agent’s behavior. At each state in the original MDP D, the agent chooses an action from among the ones returned by the partial policy for that state. Making an informed choice at this stage is the last step of DARLING, and is based on reinforcement learning” where a partial policy is a set-valued action constraint(i.e. for each state, it returns the set of actions) and a control pass is a reinforcement-learning episode/iteration (See also Leonetti, Page 110, Paragraph 5, “execution and learning: during execution, choose only actions that are returned by the partial policy, and learn their expected cumulative long-term reward in order to converge to the optimal solution”))
Leonetti discloses …the selection of the respective action for the respective control pass includes applying an answer set programming solver to an answer set programming program(Leonetti, Page 111, Paragraph 2, “We used Answer Set Programming to represent our models, and the answer set solver Clingo” where Clingo is considered applying an answer set programming solver) which models a relationship between actions and successor states that are reached by the actions to ascertain a respective plurality of sequences of states reachable from the respective current state(Leonetti, Page 111, Paragraph 8, “The answer set solver returns sequences <si, ai> where si is a state, that is a set of fluents, and ai is the action executed in state si, at time step i” where the modeling of states si being executed at time i with action ai is considered modeling sets of actions to sequences) according to a specified planning horizon that predefines a maximum number of states of each of the sequences(Leonetti, Page 111, Paragraph 5, “Given an initial state and a goal, we want to compute all the plans of length less than a certain threshold.” Where the length threshold of Leonetti is considered a predefined maximum number of states)
Leonetti discloses …the selection of the respective action for the respective control pass includes selecting, according to a predefined rate of random exploration, between (i) random action selection and (ii) selecting the respective action according to a previously trained target control policy(Leonetti, Page 105, Paragraph 7, “The exploration will be an ϵ-greedy strategy. With the ϵ-greedy strategy, the agent chooses the current optimal action according to qπ with probability 1 − ϵ, and a random action with probability ϵ” where the probability of choosing the current optimal action and random action is considered selecting between a random action and target control policy based on random exploration)
Leonetti discloses and for each of the control passes, the robotic device performs the respective selected action of the respective control pass(Leonetti, Page 104, Paragraph 5, “The agent interacts with the environment by choosing an action At” where interacting with the environment after choosing an action is considered a robotic device performing an action of a control pass)
Leonetti does not disclose the following limitations:
for each of the control passes, determining, by the robotic device during the respective control pass, whether the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes…when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes…when the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes
selecting, from the respective plurality of sequences ascertained for the respective control pass, that sequence that delivers a maximum return compared to returns of all other sequences of the respective plurality of sequences, each of the returns being a respective sum of rewards that are obtained upon reaching a respective final one of the states of the respective sequence and selecting, as the respective action for the robotic device to perform in the respective control pass, an action that, when performed while the robotic device is in the respective current state, results in a first state of the selected sequence
Martinez discloses for each of the control passes, determining, by the robotic device during the respective control pass, whether the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes(Martinez, Page 7, Paragraph 3, “The E3 algorithm is based on the concept of known states: a state is considered to be known when all the state-action pairs for that state have been visited at least the confidence threshold ζ number of times.”)…when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes,( Martinez, Page 7, Paragraph 3, “Whenever the robot enters an unknown state…” where Martinez’s algorithm checking if it is an unknown state corresponds to checking whether the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes)…when the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes(Martinez, Page 7, Paragraph 3, “If it enters a known state, and a valid plan through known states is found… If it enters a known state but no plan is found” where the algorithm checking if it is a known state corresponds to checking whether the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes)
References Leonetti and Martinez are analogous art because they are from the field of endeavor of using autonomous decision-making systems with reinforcement learning
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Leonetti and Martinez before him or her, to modify decision mechanism in Leonetti to include the ζ parameter for evaluation of known/unknown states as stated in Martinez to minimize the exploration to avoid the large sample cost of exploring the entire state space and determine when ordinary exploration has supplied enough experience to stop locally exploring using the learned model for planning.. The suggestion/motivation for doing so would have been Martinez, page 8, Paragraph 1-2 “Executing actions takes a long time for robots, so it is recommended to request help from the teacher to save large periods of learning time. Exploration is performed until a known state is reached. However, once a known state is entered, the decision maker has to decide where to explore. Exploring the whole state space requires a large number of samples; thus, a demonstration may be requested to guide the system through the optimal path”
Leonetti-Martinez does not disclose the following limitations:
selecting, from the respective plurality of sequences ascertained for the respective control pass, that sequence that delivers a maximum return compared to returns of all other sequences of the respective plurality of sequences, each of the returns being a respective sum of rewards that are obtained upon reaching a respective final one of the states of the respective sequence and selecting, as the respective action for the robotic device to perform in the respective control pass, an action that, when performed while the robotic device is in the respective current state, results in a first state of the selected sequence
Yang discloses selecting, from the respective plurality of sequences ascertained for the respective control pass, that sequence that delivers a maximum return compared to returns of all other sequences of the respective plurality of sequences, each of the returns being a respective sum of rewards that are obtained upon reaching a respective final one of the states of the respective sequence(Yang, Page 5, Col. 2, Paragraph 2, “In Scenario 2 … It starts by trying the shortest plan but during exploration of longer alternatives, it discovers the extra reward, and finally converge to the optimal” where converging to an optimal plan is considered selecting a sequence that delivers the maximum return compared to other sequences (See also Yang, Page 3, Col. 2, Paragraph 2, “A plan Π … is optimal iff…is maximal among all plans” and Yang, Page 4, Col. 2, Paragraph 3, “Given a plan Π, the quality of Π is defined by summing up all gain rewards for the transitions in Π”)); and selecting, as the respective action for the robotic device to perform in the respective control pass, an action that, when performed while the robotic device is in the respective current state, results in a first state of the selected sequence(Yang, Page 3, Col. 1, Paragraph 1, “An option (I, π , β ) is available in state st iff st ϵ I. After the option is taken, a course of actions is selected according to π until the option is terminated stochastically according to the termination condition β” where selecting an option in the state from the set of options and mapping transition maps is considered selecting an action that results in a first state of a selected sequence(See also Yang, Page 3, Col. 2, Paragraph 5, “We further build one more deterministic layer by mapping a transition path defined by a symbolic plan to a set of options… It is easy to see that the execution of a symbolic plan is deterministically realized by executing their corresponding options sequentially”)
References Leonetti-Martinez and Yang are analogous art because they are from the same field of technology of using autonomous decision-making systems combining symbolic reasoning and reinforcement learning
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Leonetti-Martinez and Yang before him or her, to modify the planner of DARLING in Leonetti-Martinez to include the choosing the path with the maximum cumulative reward of Yang to make the planning objective explicitly reward-optimal and reduce the risk of being optimal only in a reduced space. The suggestion/motivation for doing so would have been page 5, Col 2, Paragraph 2 “It starts by trying the shortest plan but during exploration of longer alternatives, it discovers the extra reward, and finally converge to the optimal.” and Page 5, Col. 1, Algorithm 1, Line 13 updating planning goal checking quality vs current optimal quality to tighten the goal.
Regarding claim 11:
The rejection of claim 10 is incorporated and further claim recites further additional
elements/limitations:
As the claims are currently presented claim 11 is dealing exclusively with the contingent limitation of when the respective current state of the respective control pass was not previously reached and does not positively recite that a respective current state of the respective control pass was not previously reached ever happening and therefore the broadest reasonable interpretation reading of when is conditional and the broadest reasonable interpretation embodiment of when is the condition is not met. Thus there is no requirement for this claim to be mapped as the condition has not been met, however, Examiner is citing the following for the claim:
Leonetti discloses when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes(Leonetti, Page 115, Paragraph 2, “Thus, an action can make the current state transition into a state that is not what Dm predicts…In that case, there will be no action available in s′ , and the system replans, computing a new partial policy from s′” where planning occurs when no action is available and triggers a planning and formulation of new partial policies that are now be used for exploration), the respective action is selected as the action that results in the first state of the selected sequence without selecting the respective action according to the previously trained target control strategy(Leonetti, Page 105, Paragraph 7, “The exploration will be an ϵ-greedy strategy. With the ϵ-greedy strategy, the agent chooses the current optimal action according to qπ with probability 1 − ϵ, and a random action with probability ϵ” where selecting random exploration instead of the optimal action of corresponds to a choosing a first state of a sequence without selecting an action according to a previously trained target control strategy as the ϵ-greedy is not part of the partial policy)
Regarding claim 12:
The rejection of claim 10 is incorporated and further claim recites further additional
elements/limitations:
As the claims are currently presented claim 11 is dealing exclusively with the contingent limitation of when the respective current state of the respective control pass was previously reached and does not positively recite that a respective current state of the respective control pass was previously reached ever happening and therefore the broadest reasonable interpretation of when is conditional and the broadest reasonable interpretation embodiment of when is the condition is not met. Thus there is no requirement for this claim to be mapped as the condition has not been met, however, Examiner is citing the following for the claim:
Leonetti discloses wherein, when the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes, the respective action is selected without applying the answer set programming solver for the respective control pass(Leonetti, Page 115, Paragraph 1, “At each state in the original MDP D, the agent chooses an action from among the ones returned by the partial policy for that state. Making an informed choice at this stage is the last step of DARLING, and is based on reinforcement learning” where the informed choice determined using reinforcement learning is performed without applying the answer set programming solver)
Regarding claim 13:
The rejection of claim 10 is incorporated and further claim recites further additional
elements/limitations:
As the claims are currently presented claim 13 is dealing exclusively with the contingent limitation of wherein for each of at least some of the control passes for which the respective current state was determined not to have been previously reached with the at least some of the control passes and does not positively recite that a respective current state of the respective control pass was not previously reached ever happening and therefore the broadest reasonable interpretation of when is conditional and the broadest reasonable interpretation embodiment of when is the condition is not met. Thus there is no requirement for this claim to be mapped as the condition has not been met, however, Examiner is citing the following for the claim:
Leonetti discloses selecting between different predefined numbers of states as the planning horizon according to a condition (Leonetti, Page 111, Paragraph 7, “Given an encoding of the initial state and the goal, all the answer sets returned for the maximum time step incrementally increased up to L are all the plans of length at most L”)
Leonetti discloses when, for ascertaining the respective action for a respective one of the at least some of the control passes, predefined available computation budget is depleted(Leonetti, Page 110, Paragraph 5, “plan generation: generate all the plans that have a cost lower than a given threshold in the model” where DARLING looking at plans with cost lower than a threshold is considered checking if a predefined computation budget is depleted(Leonetti, Page 128, Paragraph 5, “As a consequence, in the real-world domains the number of plans was often limited by planning time, rather than plan length” where the planning time is equated to computation budget)),
Leonetti discloses and when, for ascertaining the respective action for the respective one of the at least some of the control passes, the predefined available computation budget is not yet depleted, second number of states is used as the planning horizon, the second number of states being greater than the first number of states(Leonetti, Page 111, Paragraph 4, “plans are computed in order of length. We take advantage of this feature to define the threshold of plan length as a function of the length of the shortest plans” where the longer DARLING runs, the longer the plans in order of length(states), up to the maximum defined threshold is considered checking if an available computation budget is depleted and, if not, the plan will continue to the next length count(state) which will now have a higher count of states)
Leonetti discloses the method then comprising ascertaining multiple sequences of states that are reachable from the respective current state of the respective one of the at least some of the control passes using the second number of states by applying the answer set programming solver to the answer set programming program which models the relationship between actions and the successor states that are reached by the actions(Leonetti, Page 107, Paragraph 3, “We want to represent a transition system Dm=<Sm, A, fm>(where the subscript stands for model) in answer set programming, where Smis a set of states, Ais a set of actions and fm: Sm×A →Smis the transition function. This representation has to allow for fast planning, therefore in our models the set Smis always discrete, and the function fm is always deterministic.”)
Leonetti does not disclose, however Yang does disclose a first number of states of the sequence is used as the planning horizon and selecting from the ascertained sequences, using the second number of states, that sequence among the ascertained sequences that delivers the maximum return(Yang, Page 5, Col. 2, Paragraph 2, “In Scenario 2 … It starts by trying the shortest plan but during exploration of longer alternatives, it discovers the extra reward, and finally converge to the optimal” where the finding of better optimal routes is considered finding sequence that deliver the maximum return and finding the maximum return is done whether computation is depleted or not with the difference being what states are considered for the maximum return)
Regarding claim 14:
The rejection of claim 10 is incorporated and further claim recites further additional
elements/limitations:
Leonetti discloses wherein the answer set programming solver assists with multi-shot solving, and the multiple sequences for successive respective current states in each reinforcement learning training pass are ascertained by multi-shot solving using the answer set programming solver by solving each of one or more control passes of a sequence of the control passes by reusing information from prior control passes(Leonetti, Page 111, Paragraph 7, “Given an encoding of the initial state and the goal, all the answer sets returned for the maximum time step incrementally increased up to L are …All the queries with different maximum lengths up to L would be separate, but the answer set solver Clingo is particularly efficient at this computation. It has an incremental version of the solver designed specifically for planning, which efficiently concatenates these queries for different plan lengths retaining the fluents grounded at the previous iteration” where Clingo’s incremental mode is considered multi-shot solving as the increase the maximum step up to L and “all the answer sets returned” across this incremental steps are all steps of length <= L, considered many “shots” (solve calls) and the time step is grounded up to a constant n, and the solver increments n to expand the horizon)
Regarding claim 16:
Leonetti discloses a robotic device that includes a control device configured to self-train a control strategy for the robotic device using reinforcement learning performed by the robotic device navigating within an environment (Leonetti, Page 103, Abstract, “We demonstrate the effectiveness of the proposed method on a service robot, carrying out a variety of tasks in an office building” where the service robot is considered a robotic device with a control device and the method is considered reinforcement learning)
Leonetti discloses carrying out multiple reinforcement learning training passes(Leonetti, Page 111, Paragraph 7, “It has an incremental version of the solver designed specifically for planning, which efficiently concatenates these queries for different plan lengths retaining the fluents grounded at the previous iteration”) that each includes the robotic device transitioning from a respective starting state of the robotic device in the environment(Leonetti, Page 111, Paragraph 5, “Given an initial state and a goal, we want to compute all the plans of length less than a certain threshold” where the planning from initial state to goal is considered transitioning from a respecting starting state to a target state), to a respective target state of the robotic device in the environment, and one or more of which includes one or more respective intermediary states between the starting state and the target state(Leonetti, Page 109, Paragraph 5, “Lastly, planning has been combined with RL through reward shaping... Reward shaping is a technique to hasten Reinforcement Learning when the reward is sparse, and the agent has to execute a long sequence of actions before getting any feedback about its choices. The reward function is enriched by adding a term which provides feedback for intermediate states, helping guide the agent towards the goal” where Leonetti’s intermediate states between initial state and goal is considered intermediary states.)
Leonetti discloses for each respective one of the states of each of the reinforcement learning training passes prior to reaching the respective target state of the respective reinforcement learning training pass, the respective one of the states being a respective current state, a respective control pass is performed, while the robotic device is in the respective current state, for selecting a respective action for the robotic device to perform to transition out of the respective current state(Leonetti, Page 115, Paragraph 1, “The partial policy computed in the previous phase is used during the execution to constrain the agent’s behavior. At each state in the original MDP D, the agent chooses an action from among the ones returned by the partial policy for that state. Making an informed choice at this stage is the last step of DARLING, and is based on reinforcement learning” where a partial policy is a set-valued action constraint(i.e. for each state, it returns the set of actions) and a control pass is a reinforcement-learning episode/iteration) (See also Leonetti, Page 110, Paragraph 5, “execution and learning: during execution, choose only actions that are returned by the partial policy, and learn their expected cumulative long-term reward in order to converge to the optimal solution”))
Leonetti discloses …the selection of the respective action for the respective control pass includes applying an answer set programming solver to an answer set programming program(Leonetti, Page 111, Paragraph 2, “We used Answer Set Programming to represent our models, and the answer set solver Clingo” where Clingo is considered applying an answer set programming solver) which models a relationship between actions and successor states that are reached by the actions to ascertain a respective plurality of sequences of states reachable from the respective current state(Leonetti, Page 111, Paragraph 8, “The answer set solver returns sequences <si, ai> where si is a state, that is a set of fluents, and ai is the action executed in state si, at time step i” where the modeling of states si being executed at time i with action ai is considered modeling sets of actions to sequences) according to a specified planning horizon that predefines a maximum number of states of each of the sequences(Leonetti, Page 111, Paragraph 5, “Given an initial state and a goal, we want to compute all the plans of length less than a certain threshold.” Where the length threshold of Leonetti is considered a predefined maximum number of states)
Leonetti discloses …the selection of the respective action for the respective control pass includes selecting, according to a predefined rate of random exploration, between (i) random action selection and (ii) selecting the respective action according to a previously trained target control policy(Leonetti, Page 105, Paragraph 7, “The exploration will be an ϵ-greedy strategy. With the ϵ-greedy strategy, the agent chooses the current optimal action according to qπ with probability 1 − ϵ, and a random action with probability ϵ” where the probability of choosing the current optimal action and random action is considered selecting between a random action and target control policy based on random exploration)
Leonetti discloses and for each of the control passes, the robotic device performs the respective selected action of the respective control pass(Leonetti, Page 104, Paragraph 5, “The agent interacts with the environment by choosing an action At” where interacting with the environment after choosing an action is considered a robotic device performing an action of a control pass)
Leonetti does not disclose the following limitations:
for each of the control passes, determining, by the robotic device during the respective control pass, whether the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes…when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes…when the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes
selecting, from the respective plurality of sequences ascertained for the respective control pass, that sequence that delivers a maximum return compared to returns of all other sequences of the respective plurality of sequences, each of the returns being a respective sum of rewards that are obtained upon reaching a respective final one of the states of the respective sequence and selecting, as the respective action for the robotic device to perform in the respective control pass, an action that, when performed while the robotic device is in the respective current state, results in a first state of the selected sequence
Martinez discloses for each of the control passes, determining, by the robotic device during the respective control pass, whether the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes(Martinez, Page 7, Paragraph 3, “The E3 algorithm is based on the concept of known states: a state is considered to be known when all the state-action pairs for that state have been visited at least the confidence threshold ζ number of times.”)…when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes,( Martinez, Page 7, Paragraph 3, “Whenever the robot enters an unknown state…” where Martinez’s algorithm checking if it is an unknown state corresponds to checking whether the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes)…when the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes(Martinez, Page 7, Paragraph 3, “If it enters a known state, and a valid plan through known states is found… If it enters a known state but no plan is found” where the algorithm checking if it is a known state corresponds to checking whether the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes)
References Leonetti and Martinez are analogous art because they are from the field of endeavor of using autonomous decision-making systems with reinforcement learning
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Leonetti and Martinez before him or her, to modify decision mechanism in Leonetti to include the ζ parameter for evaluation of known/unknown states as stated in Martinez to minimize the exploration to avoid the large sample cost of exploring the entire state space and determine when ordinary exploration has supplied enough experience to stop locally exploring using the learned model for planning.. The suggestion/motivation for doing so would have been Martinez, page 8, Paragraph 1-2 “Executing actions takes a long time for robots, so it is recommended to request help from the teacher to save large periods of learning time. Exploration is performed until a known state is reached. However, once a known state is entered, the decision maker has to decide where to explore. Exploring the whole state space requires a large number of samples; thus, a demonstration may be requested to guide the system through the optimal path”
Leonetti-Martinez does not disclose the following limitations:
selecting, from the respective plurality of sequences ascertained for the respective control pass, that sequence that delivers a maximum return compared to returns of all other sequences of the respective plurality of sequences, each of the returns being a respective sum of rewards that are obtained upon reaching a respective final one of the states of the respective sequence and selecting, as the respective action for the robotic device to perform in the respective control pass, an action that, when performed while the robotic device is in the respective current state, results in a first state of the selected sequence
Yang discloses selecting, from the respective plurality of sequences ascertained for the respective control pass, that sequence that delivers a maximum return compared to returns of all other sequences of the respective plurality of sequences, each of the returns being a respective sum of rewards that are obtained upon reaching a respective final one of the states of the respective sequence(Yang, Page 5, Col. 2, Paragraph 2, “In Scenario 2 … It starts by trying the shortest plan but during exploration of longer alternatives, it discovers the extra reward, and finally converge to the optimal” where converging to an optimal plan is considered selecting a sequence that delivers the maximum return compared to other sequences (See also Yang, Page 3, Col. 2, Paragraph 2, “A plan Π … is optimal iff…is maximal among all plans” and Yang, Page 4, Col. 2, Paragraph 3, “Given a plan Π, the quality of Π is defined by summing up all gain rewards for the transitions in Π”)); and selecting, as the respective action for the robotic device to perform in the respective control pass, an action that, when performed while the robotic device is in the respective current state, results in a first state of the selected sequence(Yang, Page 3, Col. 1, Paragraph 1, “An option (I, π , β ) is available in state st iff st ϵ I. After the option is taken, a course of actions is selected according to π until the option is terminated stochastically according to the termination condition β” where selecting an option in the state from the set of options and mapping transition maps is considered selecting an action that results in a first state of a selected sequence(See also Yang, Page 3, Col. 2, Paragraph 5, “We further build one more deterministic layer by mapping a transition path defined by a symbolic plan to a set of options… It is easy to see that the execution of a symbolic plan is deterministically realized by executing their corresponding options sequentially”)
References Leonetti-Martinez and Yang are analogous art because they are from the same field of technology of using autonomous decision-making systems combining symbolic reasoning and reinforcement learning
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Leonetti-Martinez and Yang before him or her, to modify the planner of DARLING in Leonetti-Martinez to include the choosing the path with the maximum cumulative reward of Yang to make the planning objective explicitly reward-optimal and reduce the risk of being optimal only in a reduced space. The suggestion/motivation for doing so would have been page 5, Col 2, Paragraph 2 “It starts by trying the shortest plan but during exploration of longer alternatives, it discovers the extra reward, and finally converge to the optimal.” and Page 5, Col. 1, Algorithm 1, Line 13 updating planning goal checking quality vs current optimal quality to tighten the goal.
Regarding claim 17:
Leonetti discloses A non-transitory computer-readable memory medium on is stored a computer program including program instructions that are executable by a robotic device and that, when executed by the robotic device, cause the robotic device to perform a self-training of a control strategy for the robotic device using reinforcement learning performed by the robotic device navigating within an environment(Leonetti, Page 103, Abstract, “We demonstrate the effectiveness of the proposed method on a service robot, carrying out a variety of tasks in an office building” where the service robot is considered a robotic device with computer-readable memory medium and the method is considered reinforcement learning)
Leonetti discloses carrying out multiple reinforcement learning training passes(Leonetti, Page 111, Paragraph 7, “It has an incremental version of the solver designed specifically for planning, which efficiently concatenates these queries for different plan lengths retaining the fluents grounded at the previous iteration”) that each includes the robotic device transitioning from a respective starting state of the robotic device in the environment(Leonetti, Page 111, Paragraph 5, “Given an initial state and a goal, we want to compute all the plans of length less than a certain threshold” where the planning from initial state to goal is considered transitioning from a respecting starting state to a target state), to a respective target state of the robotic device in the environment, and one or more of which includes one or more respective intermediary states between the starting state and the target state(Leonetti, Page 109, Paragraph 5, “Lastly, planning has been combined with RL through reward shaping... Reward shaping is a technique to hasten Reinforcement Learning when the reward is sparse, and the agent has to execute a long sequence of actions before getting any feedback about its choices. The reward function is enriched by adding a term which provides feedback for intermediate states, helping guide the agent towards the goal” where Leonetti’s intermediate states between initial state and goal is considered intermediary states.)
Leonetti discloses for each respective one of the states of each of the reinforcement learning training passes prior to reaching the respective target state of the respective reinforcement learning training pass, the respective one of the states being a respective current state, a respective control pass is performed, while the robotic device is in the respective current state, for selecting a respective action for the robotic device to perform to transition out of the respective current state(Leonetti, Page 115, Paragraph 1, “The partial policy computed in the previous phase is used during the execution to constrain the agent’s behavior. At each state in the original MDP D, the agent chooses an action from among the ones returned by the partial policy for that state. Making an informed choice at this stage is the last step of DARLING, and is based on reinforcement learning” where a partial policy is a set-valued action constraint(i.e. for each state, it returns the set of actions) and a control pass is a reinforcement-learning episode/iteration) (See also Leonetti, Page 110, Paragraph 5, “execution and learning: during execution, choose only actions that are returned by the partial policy, and learn their expected cumulative long-term reward in order to converge to the optimal solution”))
Leonetti discloses …the selection of the respective action for the respective control pass includes applying an answer set programming solver to an answer set programming program(Leonetti, Page 111, Paragraph 2, “We used Answer Set Programming to represent our models, and the answer set solver Clingo” where Clingo is considered applying an answer set programming solver) which models a relationship between actions and successor states that are reached by the actions to ascertain a respective plurality of sequences of states reachable from the respective current state(Leonetti, Page 111, Paragraph 8, “The answer set solver returns sequences <si, ai> where si is a state, that is a set of fluents, and ai is the action executed in state si, at time step i” where the modeling of states si being executed at time i with action ai is considered modeling sets of actions to sequences) according to a specified planning horizon that predefines a maximum number of states of each of the sequences(Leonetti, Page 111, Paragraph 5, “Given an initial state and a goal, we want to compute all the plans of length less than a certain threshold.” Where the length threshold of Leonetti is considered a predefined maximum number of states)
Leonetti discloses …the selection of the respective action for the respective control pass includes selecting, according to a predefined rate of random exploration, between (i) random action selection and (ii) selecting the respective action according to a previously trained target control policy(Leonetti, Page 105, Paragraph 7, “The exploration will be an ϵ-greedy strategy. With the ϵ-greedy strategy, the agent chooses the current optimal action according to qπ with probability 1 − ϵ, and a random action with probability ϵ” where the probability of choosing the current optimal action and random action is considered selecting between a random action and target control policy based on random exploration)
Leonetti discloses and for each of the control passes, the robotic device performs the respective selected action of the respective control pass(Leonetti, Page 104, Paragraph 5, “The agent interacts with the environment by choosing an action At” where interacting with the environment after choosing an action is considered a robotic device performing an action of a control pass)
Leonetti does not disclose the following limitations:
for each of the control passes, determining, by the robotic device during the respective control pass, whether the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes…when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes…when the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes
selecting, from the respective plurality of sequences ascertained for the respective control pass, that sequence that delivers a maximum return compared to returns of all other sequences of the respective plurality of sequences, each of the returns being a respective sum of rewards that are obtained upon reaching a respective final one of the states of the respective sequence and selecting, as the respective action for the robotic device to perform in the respective control pass, an action that, when performed while the robotic device is in the respective current state, results in a first state of the selected sequence
Martinez discloses for each of the control passes, determining, by the robotic device during the respective control pass, whether the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes(Martinez, Page 7, Paragraph 3, “The E3 algorithm is based on the concept of known states: a state is considered to be known when all the state-action pairs for that state have been visited at least the confidence threshold ζ number of times.”)…when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes,( Martinez, Page 7, Paragraph 3, “Whenever the robot enters an unknown state…” where Martinez’s algorithm checking if it is an unknown state corresponds to checking whether the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes)…when the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes(Martinez, Page 7, Paragraph 3, “If it enters a known state, and a valid plan through known states is found… If it enters a known state but no plan is found” where the algorithm checking if it is a known state corresponds to checking whether the respective current state of the respective control pass was previously reached during any of the multiple reinforcement learning training passes)
References Leonetti and Martinez are analogous art because they are from the field of endeavor of using autonomous decision-making systems with reinforcement learning
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Leonetti and Martinez before him or her, to modify decision mechanism in Leonetti to include the ζ parameter for evaluation of known/unknown states as stated in Martinez to minimize the exploration to avoid the large sample cost of exploring the entire state space and determine when ordinary exploration has supplied enough experience to stop locally exploring using the learned model for planning.. The suggestion/motivation for doing so would have been Martinez, page 8, Paragraph 1-2 “Executing actions takes a long time for robots, so it is recommended to request help from the teacher to save large periods of learning time. Exploration is performed until a known state is reached. However, once a known state is entered, the decision maker has to decide where to explore. Exploring the whole state space requires a large number of samples; thus, a demonstration may be requested to guide the system through the optimal path”
Leonetti-Martinez does not disclose the following limitations:
selecting, from the respective plurality of sequences ascertained for the respective control pass, that sequence that delivers a maximum return compared to returns of all other sequences of the respective plurality of sequences, each of the returns being a respective sum of rewards that are obtained upon reaching a respective final one of the states of the respective sequence and selecting, as the respective action for the robotic device to perform in the respective control pass, an action that, when performed while the robotic device is in the respective current state, results in a first state of the selected sequence
Yang discloses selecting, from the respective plurality of sequences ascertained for the respective control pass, that sequence that delivers a maximum return compared to returns of all other sequences of the respective plurality of sequences, each of the returns being a respective sum of rewards that are obtained upon reaching a respective final one of the states of the respective sequence(Yang, Page 5, Col. 2, Paragraph 2, “In Scenario 2 … It starts by trying the shortest plan but during exploration of longer alternatives, it discovers the extra reward, and finally converge to the optimal” where converging to an optimal plan is considered selecting a sequence that delivers the maximum return compared to other sequences (See also Yang, Page 3, Col. 2, Paragraph 2, “A plan Π … is optimal iff…is maximal among all plans” and Yang, Page 4, Col. 2, Paragraph 3, “Given a plan Π, the quality of Π is defined by summing up all gain rewards for the transitions in Π”)); and selecting, as the respective action for the robotic device to perform in the respective control pass, an action that, when performed while the robotic device is in the respective current state, results in a first state of the selected sequence(Yang, Page 3, Col. 1, Paragraph 1, “An option (I, π , β ) is available in state st iff st ϵ I. After the option is taken, a course of actions is selected according to π until the option is terminated stochastically according to the termination condition β” where selecting an option in the state from the set of options and mapping transition maps is considered selecting an action that results in a first state of a selected sequence(See also Yang, Page 3, Col. 2, Paragraph 5, “We further build one more deterministic layer by mapping a transition path defined by a symbolic plan to a set of options… It is easy to see that the execution of a symbolic plan is deterministically realized by executing their corresponding options sequentially”)
References Leonetti-Martinez and Yang are analogous art because they are from the same field of technology of using autonomous decision-making systems combining symbolic reasoning and reinforcement learning
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Leonetti-Martinez and Yang before him or her, to modify the planner of DARLING in Leonetti-Martinez to include the choosing the path with the maximum cumulative reward of Yang to make the planning objective explicitly reward-optimal and reduce the risk of being optimal only in a reduced space. The suggestion/motivation for doing so would have been page 5, Col 2, Paragraph 2 “It starts by trying the shortest plan but during exploration of longer alternatives, it discovers the extra reward, and finally converge to the optimal.” and Page 5, Col. 1, Algorithm 1, Line 13 updating planning goal checking quality vs current optimal quality to tighten the goal.
Regarding claim 18:
The rejection of claim 10 is incorporated and further claim recites further additional
elements/limitations:
Leonetti discloses for each of at least one of the performed actions: observing, by the robotic device, a respective reward and a respective subsequent state achieved by the respective action(Leonetti, Page 104, Paragraph 5, “We consider the system at discrete time steps. Let t ∈ N be the current time, and St be the state at time t. The agent interacts with the environment by choosing an action At and perceiving the next state St+1…It also receives a reward Rt+1:”) and updating the target control policy based on the observation(Leonetti, Page 115, Paragraph 1, “At each state in the original MDP D, the agent chooses an action from among the ones returned by the partial policy for that state. Making an informed choice at this stage is the last step of DARLING, and is based on reinforcement learning.”)
Regarding claim 21:
The rejection of claim 18 is incorporated and further claim recites further additional
elements/limitations:
As the claims are currently presented claim 11 is dealing exclusively with the contingent limitation of for each of the one or more control passes for which the respective current state was determined to have been previously reached and does not positively recite that a respective current state of the respective control pass was previously reached ever happening and therefore the broadest reasonable interpretation of when is conditional and the broadest reasonable interpretation embodiment of when is the condition is not met. Thus there is no requirement for this claim to be mapped as the condition has not been met, however, Examiner is citing the following for the claim:
Leonetti discloses the selecting according to the predefined rate of random exploration includes selecting the respective action using the random action selection with a probability corresponding to the predefined rate of random exploration and selecting the respective action using the target control policy with a probability corresponding to one minus the predefined rate of random exploration(Leonetti, Page 105, Paragraph 7, “The exploration will be an ϵ-greedy strategy. With the ϵ-greedy strategy, the agent chooses the current optimal action according to qπ with probability 1 − ϵ, and a random action with probability ϵ” where 1 – ϵ corresponds to one minus the predefined rate of random exploration and ϵ corresponds to a predefined rate of random exploration that is used to select the respective action using the target control policy )
Regarding claim 22:
The rejection of claim 21 is incorporated and further claim recites further additional
elements/limitations:
As the claims are currently presented claim 11 is dealing exclusively with the contingent limitation of for each of the one or more control passes for which the respective current state was determined to have been previously reached and does not positively recite that a respective current state of the respective control pass was previously reached ever happening and therefore the broadest reasonable interpretation of when is conditional and the broadest reasonable interpretation embodiment of when is the condition is not met. Thus there is no requirement for this claim to be mapped as the condition has not been met, however, Examiner is citing the following for the claim:
Leonetti discloses the respective action is selected without applying the answer set programming solver to the answer set programming program for ascertaining, for the respective current state, a plurality of sequences of states reachable from the respective current state(Leonetti, Page 115, Paragraph 1, “At each state in the original MDP D, the agent chooses an action from among the ones returned by the partial policy for that state. Making an informed choice at this stage is the last step of DARLING, and is based on reinforcement learning” where the informed choice determined using reinforcement learning is performed without applying the answer set programming solver)
Regarding claim 24:
The rejection of claim 10 is incorporated and further claim recites further additional
elements/limitations:
As the claims are currently presented claim 11 is dealing exclusively with the contingent limitation of when the respective current state of the respective control pass was not previously reached and does not positively recite that a respective current state of the respective control pass was not previously reached ever happening and therefore the broadest reasonable interpretation of when is conditional and the broadest reasonable interpretation embodiment of when is the condition is not met. Thus there is no requirement for this claim to be mapped as the condition has not been met, however, Examiner is citing the following for the claim:
Leonetti discloses when the respective current state of the respective control pass was not previously reached during any of the multiple reinforcement learning training passes and the answer set programming solver outputs no action in a set of available actions for the respective current state(Leonetti, Page 115, Paragraph 2, “Thus, an action can make the current state transition into a state that is not what Dm predicts…In that case, there will be no action available in s′ , and the system replans, computing a new partial policy from s′” where planning occurs when no action is available and triggers a planning and formulation of new partial policies that are now be used for exploration), the selection of the respective action for the respective control pass includes selecting, according to the predefined rate of random exploration, between (i) the random action selection and (ii) selecting the respective action according to the previously trained target control policy(Leonetti, Page 105, Paragraph 7, “The exploration will be an ϵ-greedy strategy. With the ϵ-greedy strategy, the agent chooses the current optimal action according to qπ with probability 1 − ϵ, and a random action with probability ϵ” where the probability of choosing the current optimal action and random action is considered selecting between a random action and target control policy based on random exploration)
Regarding claim(s) 23 and 25:
Claim(s) 23 and 25 is/are rejected under 35 U.S.C. 103 as being unpatentable over Leonetti et al.(“A synthesis of automated planning and reinforcement learning for efficient, robust decision-making”, henceforth known as Leonetti) in view of Martinez et al, (“Relational Reinforcement Learning with Guided Demonstration”, henceforth known as Martinez) and further in view of Yang et al.(“PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making”, henceforth known as Yang) with further in view of Bejjani et al.(“ Planning with a Receding Horizon for Manipulation in Clutter using a Learned Value Function” henceforth known as Bejjani)
Regarding claim 23:
The rejection of claim 10 is incorporated and further claim recites further additional
elements/limitations:
Leonetti discloses the answer set programming solver(Leonetti, Page 111, Paragraph 2, “We used Answer Set Programming to represent our models, and the answer set solver Clingo” where Clingo is considered applying an answer set programming solver) is applied for a plurality of respective current states(Leonetti, Page 111, Paragraph 8, “The answer set solver returns sequences <si, ai> where si is a state, that is a set of fluents, and ai is the action executed in state si, at time step i” where looking at multiple sequences of states corresponds to a plurality of respective current states ) during the at least one reinforcement learning training pass before the respective target state is reached(Leonetti, Page 110, Paragraph 8, “The plan generation step requires a planner able to return all plans that can reach the goal from the initial state”)
Leonetti-Martinez-Yang does not explicitly disclose, however Bejjani discloses specified planning horizon used for the respective current state is insufficient to reach the respective target state from the respective current state(Bejjani, Page 3, Col 2, Paragraph 6,“Therefore, in this paper, we propose to plan for only a short horizon, and not necessarily until the goal. G”)
References Leonetti-Martinez-Yang and Bejjani are analogous art because they are from the same field of endeavor of using autonomous robot with imperfect planning models and efficiently adapting it’s behavior.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art, having the teachings of Leonetti-Martinez-Yang and Bejjani before him or her, to modify horizon planning in Leonetti-Martinez-Yang to include the short horizon of Bejjani to explore maximizing the reward. The suggestion/motivation for doing so would have been Bejjani, Page 3, Col 2, Paragraph 6 “Therefore, in this paper, we propose to plan for only a short horizon, and not necessarily until the goal. Given such a horizon h, we are then interested in maximizing the reward for h steps”
Regarding claim 25:
The rejection of claim 10 is incorporated and further claim recites further additional
elements/limitations:
Leonetti-Martinez-Yang does not explicitly disclose, however Bejjani wherein the reinforcement learning is performed according to an off-policy reinforcement learning method(Bejjani, Page 5, Col 1, Paragraph 3,“We implement the Deep Q-Learning (DQN) algorithm” where Deep Q-Learning is an off-policy reinforcement learning algorithm)
Relevant art:
While not used in the current rejection the following prior art was found to be relevant::
Nickles, “A System for the Use of Answer Set Programming in Reinforcement Learning” as it discusses using ASP in reinforcement learning
Lin et al, “Self-Improving Reactive Agents Based On Reinforcement Learning, Planning and Teaching” as it discuss planning and reinforcement learning
Response to Arguments
Applicant's arguments filed 07/13/2026 have been fully considered but they are not persuasive. A breakdown of arguments can be found below.
35 U.S.C. § 112:
Applicant has added amendments to highlight the differences in states and control passes that have overcome previous 112 rejections.
35 U.S.C. § 103:
Applicant appears to argue on pages 12-15 that Leonetti and Yang do not implement or cover the per-control-pass state-history determination of the claims and does not suggest the features in which the ASP local look-ahead and previously reached current states use of greedy based algorithms. Further, Applicant argues that the combination would not provide the claimed state-by-state branching in which a robot determines, for each control pass, whether the current state was previously reached and Neither Leonetti nor Yang discloses or suggests a state-history-triggered switching system and Leonetti’s greedy algorithm does not match the claim language as the claims require Leonetti’s random-or-target-policy selection is used in the previously reached-state branch, after the robotic device has made a determination that the current state of the control pass was previously reached during one of the training passes.
Examiner respectfully disagrees as a control pass is not clearly defined in the specification or claims. The most specific information Examiner could locate is in Page 21, Line 20, that merely states a control pass has an initial state and Page 2, Lines 10-13 that states that a reinforcement learning pass begins with an initial state of a control pass . Applicant appears to be interpreting a narrower claim as the current claims do not positively recite characteristics of a control pass that would preclude the BRI interpretation that a control pass is simply a reinforcement learning pass and/or a part of reinforcement learning pass that exists at least at the beginning of the learning pass or part of, in some fashion, an Answer Set Programming process/pass/decision making process. Further, Applicant asserts that there is a specific state-history-triggered switching system, however the claims lack the specific positively recited language, however Examiner has brought in prior art Martinez for the decision mechanism in claims as Martinez’s controller classifies the current state on past experience of other states. Examiner also notes that the claimed language does not state after a robotic device has made a determination and only uses the language of when which is conditional and not sequential.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHARLES JEFFREY JONES JR whose telephone number is (703)756-1414. The examiner can normally be reached Monday - Friday 8:00 - 5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at 571-272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/C.J.J./ Examiner, Art Unit 2122
/KAKALI CHAKI/ Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights