DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Pending
1-31
Rejected – 35 U.S.C. 102
1-5, 7-12, 14-20, 22-28, 30-31
Rejected – 35 U.S.C. 103
6, 13, 21, 29
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 08/25/2025 has been entered.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-5, 7-12, 14-20, 22-28, 30-31 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Kalakrishnan et al. (US 2022/0105624 A1, hereinafter “Kalakrishnan”).
Regarding claim 1: Kalakrishnan teaches: One or more processors, comprising: circuitry to ([0070] processors; [0132] robot, control system, wired computing device):
cause one or more neural networks to generate one or more predicates corresponding to a logical state of an environment ([0053] use neural networks; [0055] predict actions, using embedding and control networks, sharing convolutional layers between them); and
cause the one or more neural networks to generate one or more actions to be performed by one or more robotic devices based, at least in part, on the one or more predicates ([0053] use neural networks; [0132] robot controls movement of end effector to change environment state).
Regarding claim 2: Kalakrishnan further teaches: The one or more processors of claim 1, wherein: the one or more neural networks comprise a first neural network and a second neural network ([0053] use of neural networks; [0065] captures info regarding current environment of robot, using vision component; [0055] control network processes image of current scene, including robot state, end-effector pose, to generate robot action; [0069] combination of environment state embedding, robot state data, and context embedding provided to actor network to generate robot action to perform robot task);
the first neural network is to generate the one or more predicates ([0065] environment state data captures info regarding the current environment of the robot; captured using vision component; [0065] environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment; [0053] use of neural networks; [0064] binary indication); and
the second neural network is to generate the one or more actions to be performed by the one or more robotic devices ([0132] vision component of the sensors capture environment state data, which is processed, along with robot state data, using policy network of the meta-learning model to generate end effector control commands for controlling the movement and/or grasping of an end effector of the robot; [0053]-[0055] using neural networks to control actions of the robot).
Regarding claim 3: Kalakrishnan further teaches: The one or more processors of claim 1, wherein the logical state comprises one or more predicates describing one or more environmental states ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component; environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment).
Regarding claim 4: Kalakrishnan further teaches: The one or more processors of claim 1, wherein the one or more predicates describe one or more environmental states that are obtained by one or more cameras ([0058] sensors to generate images (camera) to visualize robot and environment; [0129] sensors).
Regarding claim 5: Kalakrishnan further teaches: The one or more processors of claim 1, wherein the one or more robotic devices are to perform the one or more actions to change the logical state ([0132] vision component of the sensors capture environment state data, which is processed, along with robot state data, using policy network of the meta-learning model to generate end effector control commands for controlling the movement and/or grasping of an end effector of the robot; [0057] robot, end effector, actuation controls motion of effector).
Regarding claim 7: Kalakrishnan further teaches: The one or more processors of claim 1, wherein the one or more neural networks are based, at least in part, on temporal convolutional networks ([0053] meta-policy can consist of two neural networks: embedding network and control network; embedding network can consist of a convolutional neural network followed by 1-D temporal convolutions).
Regarding claim 8: Kalakrishnan discloses: A system, comprising ([0070] system): one or more processors to ([0132] robot, control system, wired computing device):
cause one or more neural networks to generate one or more predicates corresponding to a logical state of an environment ([0053] use of neural networks; [0055] predict actions, using embedding and control networks, sharing convolutional layers between them); and
cause the one or more neural networks to generate one or more actions to be performed by one or more robotic devices based, at least in part, on the one or more predicates ([0053] use of neural networks; [0132] robot controls movement of end effector to change environment state).
Regarding claim 9: Kalakrishnan further teaches: The system of claim 8, wherein: the one or more robotic devices comprise mechanical and electrical components to perform one or more tasks ([0094] robot, computing device, processors; [0131] processors, CPU, GPU; [0137] memory, RAM, ROM);
the one or more neural networks determine one or more environmental states ([0065] environment state data captures info regarding the current environment of the robot; captured using vision component; [0065] environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment; [0053] use of neural networks; [0064] binary indication);
the one or more environmental states comprise observations about an environment ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component);
the observations about an environment are obtained from one or more sensors ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component); and
the one or more neural networks determine the one or more actions to help control the one or more robotic devices ([0132] vision component of the sensors capture environment state data, which is processed, along with robot state data, using policy network of the meta-learning model to generate end effector control commands for controlling the movement and/or grasping of an end effector of the robot; [0053]-[0055] using neural networks to control actions of the robot).
Regarding claim 10: Kalakrishnan further teaches: The system of claim 9, wherein the one or more sensors comprise cameras ([0058] sensors to generate images (camera) to visualize robot and environment; [0129] sensors).
Regarding claim 11: Kalakrishnan further teaches: The system of claim 9, wherein the logical state specifies a configuration of the environment ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component; environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment).
Regarding claim 12: Kalakrishnan further teaches: The system of claim 9, wherein the one or more neural networks are to determine the one or more actions based at least in part on the logical state and an operator ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component; environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment).
Regarding claim 14: Kalakrishnan further teaches: The system of claim 8, wherein the one or more robotic devices are to perform one or more tasks that comprise one or more steps to be performed in order to accomplish a goal ([0132] vision component of the sensors capture environment state data, which is processed, along with robot state data, using policy network of the meta-learning model to generate end effector control commands for controlling the movement and/or grasping of an end effector of the robot; [0057] robot, end effector, actuation controls motion of effector).
Regarding claim 15: Kalakrishnan further teaches: The system of claim 8, wherein the one or more neural networks are based, at least in part, on convolutional neural networks ([0053] meta-policy can consist of two neural networks: embedding network and control network; embedding network can consist of a convolutional neural network followed by 1-D temporal convolutions).
Regarding claim 16: Kalakrishnan discloses: A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause one or more processors to at least ([0094] robot, computing device, processors; [0131] processors, CPU, GPU; [0137] memory, RAM, ROM):
causing one or more neural networks to generate one or more predicates corresponding to a logical state of an environment ([0053] use neural networks; [0132] robot, control system, wired computing device; robot controls movement of end effector to change environment state; [0055] predict actions, using embedding and control networks, sharing convolutional layers between them; [0065] environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment; [0064] binary indication); and
causing the one or more neural networks to generate one or more actions to be performed by one or more robotic devices based, at least in part, on the one or more predicates ([0053] use neural networks; [0055] predict actions, using embedding and control networks, sharing convolutional layers between them; [0132] robot controls movement of end effector to change environment state).
Regarding claim 17: Kalakrishnan further teaches: The non-transitory machine-readable medium of claim 16, wherein: the logical state describes one or more environmental states ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component; environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment);
the one or more environmental states are obtained by one or more sensors ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component);
the one or more neural networks determine the one or more actions that, when performed, help control the one or more robotic devices ([0132] vision component of the sensors capture environment state data, which is processed, along with robot state data, using policy network of the meta-learning model to generate end effector control commands for controlling the movement and/or grasping of an end effector of the robot; [0053] – [0055] using neural networks to control actions of the robot); and
the one or more actions are determined based at least in part on the logical state and an operator ([0055] control network processes image of current scene, including robot state such as end-effector pose, to generate robot action; [0069] combination of environment state embedding, robot state data, and context embedding provided to actor network to generate robot action to perform robot task).
Regarding claim 18: Kalakrishnan further teaches: The non-transitory machine-readable medium of claim 17, wherein the logical state comprises one or more predicates describing the one or more environmental states ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component; environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment).
Regarding claim 19: Kalakrishnan further teaches: The non-transitory machine-readable medium of claim 17, wherein the one or more actions instruct the one or more robotic devices to change the logical state ([0132] vision component of the sensors capture environment state data, which is processed, along with robot state data, using policy network of the meta-learning model to generate end effector control commands for controlling the movement and/or grasping of an end effector of the robot; [0057] robot, end effector, actuation controls motion of effector).
Regarding claim 20: Kalakrishnan further teaches: The non-transitory machine-readable medium of claim 17, wherein the operator contains conditions and effects ([0065] environment state data captures info regarding the current environment of the robot; captured using vision component; environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment; [0132] vision component of the sensors capture environment state data, which is processed, along with robot state data, using policy network of the meta-learning model to generate end effector control commands for controlling the movement and/or grasping of an end effector of the robot; [0057] robot, end effector, actuation controls motion of effector).
Regarding claim 22: Kalakrishnan further teaches: The non-transitory machine-readable medium of claim 16, wherein the one or more neural networks are based, at least in part, on a temporal convolutional network ([0053] meta-policy can consist of two neural networks: embedding network and control network; embedding network can consist of a convolutional neural network followed by 1-D temporal convolutions).
Regarding claim 23: Kalakrishnan further teaches: The non-transitory machine-readable medium of claim 16, wherein the one or more neural networks are to generate one or more loss values used to train the one or more neural networks ([0033] minimize task loss with respect to model parameters following steps of gradient descent of the same task loss; [0055] neural network used and loss function minimized via optimization).
Regarding claim 24: Kalakrishnan discloses: A method, comprising ([0027] method):
causing one or more neural networks to generate one or more predicates corresponding to a logical state of an environment ([0053] use neural networks; [0132] robot, control system, wired computing device; robot controls movement of end effector to change environment state; [0055] predict actions, using embedding and control networks, sharing convolutional layers between them; [0065] environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment; [0064] binary indication); and
cause the one or more neural networks to generate one or more actions to be performed by one or more robotic devices based, at least in part, on the one or more predicates ([0053] use neural networks; [0055] predict actions, using embedding and control networks, sharing convolutional layers between them; [0132] robot controls movement of end effector to change environment state).
Regarding claim 25: Kalakrishnan further teaches: The method of claim 24, wherein: the logical state describes one or more environmental states ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component; environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment);
the one or more environmental states are obtained by one or more sensors ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component);
the one or more neural networks determine the one or more actions that, when performed, help control the one or more robotic devices ([0132] vision component of the sensors capture environment state data, which is processed, along with robot state data, using policy network of the meta-learning model to generate end effector control commands for controlling the movement and/or grasping of an end effector of the robot; [0053]-[0055] using neural networks to control actions of the robot); and
the one or more actions are determined based at least in part on the logical state and an operator ([0055] control network processes image of current scene, including robot state such as end-effector pose, to generate robot action; [0069] combination of environment state embedding, robot state data, and context embedding provided to actor network to generate robot action to perform robot task).
Regarding claim 26: Kalakrishnan further teaches: The method of claim 25, wherein the sensors comprise hardware and software devices that gather information about the one or more environmental states ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component; [0129] sensors; [0131] processors, CPU, GPU).
Regarding claim 27: Kalakrishnan further teaches: The method of claim 25, wherein the logical state comprises one or more predicates describing the one or more environmental states ([0058] sensors to generate images (camera) to visualize robot and environment; [0065] environment state data captures info regarding the current environment of the robot; captured using vision component; environment state includes predicted state based on vision/other sensor data; poses and classification of objects in environment).
Regarding claim 28: Kalakrishnan further teaches: The method of claim 25, wherein the one or more actions are to instruct the one or more robotic devices to change state ([0132] vision component of the sensors capture environment state data, which is processed, along with robot state data, using policy network of the meta-learning model to generate end effector control commands for controlling the movement and/or grasping of an end effector of the robot; [0057] robot, end effector, actuation controls motion of effector).
Regarding claim 30: Kalakrishnan further teaches: The method of claim 24, wherein the one or more neural networks are to generate one or more loss values used to train the one or more neural networks ([0033] minimize task loss with respect to model parameters following steps of gradient descent of the same task loss; [0055] neural network used and loss function minimized via optimization).
Regarding claim 31: Kalakrishnan further teaches: The method of claim 24, wherein the one or more neural networks are based, at least in part, on temporal convolutional neural networks ([0053] meta-policy can consist of two neural networks: embedding network and control network; embedding network can consist of a convolutional neural network followed by 1D temporal convolutions).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 6, 13, 21, 29 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kalakrishnan et al. (US 2022/0105624 A1, hereinafter “Kalakrishnan”) and further in view of Sabe et al. (US 2006/0195227 A1, hereinafter “Sabe”).
Regarding claim 6: Kalakrishnan discloses: The one or more processors of claim 1 ([0094] robot, computing device, processors; [0131] processors, CPU, GPU; [0137] memory, RAM, ROM).
However, Kalakrishnan does not explicitly teach, but Sabe teaches: wherein the one or more predicates describe one or more environmental states that comprise one or more unexpected events ([0131] autonomous agent 1 executes the plan generated on the basis of the wrong prediction; as a result, the autonomous agent 1 may change to an unexpected state; even in such a case, the Fwd model 131 learns this execution result; that is, the Fwd model 131 learns a new input/output relationship (a present environment and prediction of an environment of next time to an action), and thus a prediction error is corrected as the learning progresses; accordingly, when the planner 133 plans the behavior of the autonomous agent 1 again, the planner 133 can generate a plan that is different from the plan previously generated on the basis of the wrong prediction).
Kalakrishnan and Sabe are analogous art to the claimed invention since they are from the similar field of robot controls and environment states. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Kalakrishnan with the unexpected events of Sabe to create, with a reasonable expectation for success, a system, method, and process wherein the one or more environmental states comprises one or more unexpected events. The motivation for modification would have been to have improved prediction performance (Sabe, [0170]) and improve an action approximate to a learned path in control of actions taken thereafter (Sabe, [0146]).
Regarding claim 13: Kalakrishnan discloses: The system of claim 9 ([0094] robot, computing device, processors; [0131] processors, CPU, GPU; [0137] memory, RAM, ROM).
However, Kalakrishnan does not explicitly teach, but Sabe teaches: wherein the one or more environmental states comprise unexpected events ([0131] autonomous agent 1 executes the plan generated on the basis of the wrong prediction; as a result, the autonomous agent 1 may change to an unexpected state; even in such a case, the Fwd model 131 learns this execution result; that is, the Fwd model 131 learns a new input/output relationship (a present environment and prediction of an environment of next time to an action), and thus a prediction error is corrected as the learning progresses; accordingly, when the planner 133 plans the behavior of the autonomous agent 1 again, the planner 133 can generate a plan that is different from the plan previously generated on the basis of the wrong prediction).
Kalakrishnan and Sabe are analogous art to the claimed invention since they are from the similar field of robot controls and environment states. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Kalakrishnan with the unexpected events of Sabe to create, with a reasonable expectation for success, a system, method, and process wherein the one or more environmental states comprises one or more unexpected events. The motivation for modification would have been to have improved prediction performance (Sabe, [0170]) and improve an action approximate to a learned path in control of actions taken thereafter (Sabe, [0146]).
Regarding claim 21: Kalakrishnan discloses: The non-transitory machine-readable medium of claim 17 ([0094] robot, computing device, processors; [0131] processors, CPU, GPU; [0137] memory, RAM, ROM).
However, Kalakrishnan does not explicitly disclose teach, but Sabe teaches: wherein the one or more environmental states comprise one or more unexpected events ([0131] autonomous agent 1 executes the plan generated on the basis of the wrong prediction; as a result, the autonomous agent 1 may change to an unexpected state; even in such a case, the Fwd model 131 learns this execution result; that is, the Fwd model 131 learns a new input/output relationship (a present environment and prediction of an environment of next time to an action), and thus a prediction error is corrected as the learning progresses; accordingly, when the planner 133 plans the behavior of the autonomous agent 1 again, the planner 133 can generate a plan that is different from the plan previously generated on the basis of the wrong prediction).
Kalakrishnan and Sabe are analogous art to the claimed invention since they are from the similar field of robot controls and environment states. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Kalakrishnan with the unexpected events of Sabe to create, with a reasonable expectation for success, a system, method, and process wherein the one or more environmental states comprises one or more unexpected events. The motivation for modification would have been to have improved prediction performance (Sabe, [0170]) and improve an action approximate to a learned path in control of actions taken thereafter (Sabe, [0146]).
Regarding claim 29: Kalakrishnan discloses: The method of claim 25 ([0094] robot, computing device, processors; [0131] processors, CPU, GPU; [0137] memory, RAM, ROM).
However, Kalakrishnan does not explicitly teach, but Sabe teaches: wherein the one or more environmental states comprise unexpected events ([0131] autonomous agent 1 executes the plan generated on the basis of the wrong prediction; as a result, the autonomous agent 1 may change to an unexpected state; even in such a case, the Fwd model 131 learns this execution result; that is, the Fwd model 131 learns a new input/output relationship (a present environment and prediction of an environment of next time to an action), and thus a prediction error is corrected as the learning progresses; accordingly, when the planner 133 plans the behavior of the autonomous agent 1 again, the planner 133 can generate a plan that is different from the plan previously generated on the basis of the wrong prediction).
Kalakrishnan and Sabe are analogous art to the claimed invention since they are from the similar field of robot controls and environment states. It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the invention of Kalakrishnan with the unexpected events of Sabe to create, with a reasonable expectation for success, a system, method, and process wherein the one or more environmental states comprises one or more unexpected events. The motivation for modification would have been to have improved prediction performance (Sabe, [0170]) and improve an action approximate to a learned path in control of actions taken thereafter (Sabe, [0146]).
Response to Arguments
Applicant's arguments filed 08/25/2025 have been fully considered but they are not persuasive.
Applicant argues:
Applicant respectfully submits that Kalakrishnan does not teach circuitry to cause one or more neural networks to "generate one or more predicates corresponding to a logical state of an environment," as recited in claim 1.
While Kalakrishnan may disclose using a neural network to generate environment state embeddings representing visual features of an environment, it does not disclose using a neural network to generate predicates corresponding to a logical state of an environment. This is apparent at least because an embedding is not a predicate.
Additionally, Kalakrishnan does not teach circuitry to cause the one or more neural networks to "generate one or more actions to be performed by one or more robotic devices based, at least in part, on the one or more predicates," as recited in claim 1.
Kalakrishnan discloses that "[t]his environment state data may be processed, along with robot state data, using a policy network of the meta-learning model to generate the one or more end effector control commands for controlling the movement and/or grasping of an end effector of the robot." See Kalakrishnan 132.
Therefore, while Kalakrishnan may disclose generating control commands to control the movement of a robot using the environment state, which may be in the form of embeddings, but does not disclose using predicates to generate actions to be performed by robotic devices, as recited in claim 1.
Examiner responds:
Examiner respectfully disagrees. The “one or more predicates” as claimed are described in Applicant’s specification as follows:
[0076]
In at least one embodiment, an ideal logical state function L() 404 determines an ideal logical state lt406 based on an underlying world state st 402. In at least one embodiment, an ideal logical state lt 406 is a set of predicates p, where each p has different arguments consisting of symbols. In at least one embodiment, an ideal logical state lt 406 is a set of predicates p that specify a logical environment or world state based on a total underlying world state 402. In at least one embodiment, an ideal logical state function L() 404 is defined as i = L(st), where an ideal logical state lt 406 is computed from a total underlying world state st 402.
[0079]
In at least one embodiment, given a list of operators 414, a current operator 420 can be determined from an approximated logical state 416. In at least one embodiment, an approximated logical state is a set of predicates p, where each p has different arguments consisting of symbols.
As such, these “predicates” are described as a logical state that specifies a logical environment or world state based on a total underlying world state.
Kalakrishnan recites:
[0005]
In some implementations, the meta-learning model can be trained using Q-learning. Q-learning learns a policy network which can be used to determine what action for an agent (e.g., a robot) to take under what circumstances (e.g., based on current robot state data and/or current environmental state data). In some implementations, tasks can be represented using a finite Markov decision process. Q-learning can be used to find the optimal policy which maximizes the expected value of a reward over successive steps starting from the current state for the finite Markov decision process.
[0055]
In some implementations, the control network can process an image of the current scene (along with other parts of the robot's state such as end-effector pose), to generate a robot action.
[0061]
Data from robot 100 (e.g., state data) can be utilized to train a meta-learning model 114 using meta-learning model training engine 108. For example, meta-learning model training engine 108 can train a trial policy and/or an adapted trial policy of meta-learning model 114 using meta-learning. Meta-learning model training engine 108 can include imitation learning training engine 110, reinforcement learning training engine 112, and/or additional engine(s) (not depicted).
[0065]
Environment state data 202 captures information regarding the current environment of the robot. In some implementations, environment state data can be captured using a vision component, such as vision component 106 illustrated in FIG. 1. […] For instance, environment state data 202 can include pose(s) and/or classification(s) of object(s) in the environment, determined based on vision data.
[0066]
The environment state embedding can be combined with robot state data 208. For example, the environment state embedding 206 can be concatenated with robot state data 208. In some implementations, the robot state data 208 can include a representation of a current end-effector pose, a current end effector angle, a current end effector velocity, and/or additional information about the current position of the robot and/or one or more components of the robot.
[0067]
As described above, environment state data 202 can be processed using vision network 204 to generate environment state embedding 206. Environment state embedding 206 can be combined with robot state data 208. In the illustrated example, environment state embedding 206 and robot state data 208 are additionally combined with a context embedding 260.
[0132]
As described herein, in some implementations all or aspects of the control commands generated by control system 1060 in positioning an end effector to grasp an object may be based on end effector commands generated using a meta-learning model. For example, a vision component of the sensors 1042a-m may capture environment state data. This environment state data may be processes, along with robot state data, using a policy network of the meta-learning model to generate the one or more end effector control commands for controlling the movement and/or grasping of an end effector of the robot.
[0136]
Storage subsystem 1124 stores programming and data constructs that provide the functionality of some or all of the modules described herein. For example, the storage subsystem 1124 may include the logic to perform selected aspects of the processes of FIGS. 3-9, and/or other methods described herein.
Here, Kalakrishnan describes the process of a robot using sensors to determine the state of the environment and the state of itself within the environment. This is described in a logical state such that it can be useful and processed by the robot system. This “logic” is a predicate. Thus, Kalakrishnan discloses sensing and processing data to generate predicates that correspond to the current logical state of the environment.
Then, Kalakrishnan teaches creation of motion, movement, and action plans for the robot based on the predicate generated for the logical state of the environment. This means the actions to be performed by the robot are based on the generated predicates. Kalakrishnan also teaches the circuitry required to perform these robotic tasks, such as the processors, GPU, CPU, and controllers (see [0131]), and the software modules, memory, RAM, ROM, hard drive, file storage, and processors (see [0137]). Therefore, Kalakrishnan teaches the entirety of claim 1, and those claims similar in scope to claim 1, and thus the prior art rejections are maintained.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MADISON B EMMETT whose telephone number is (303)297-4231. The examiner can normally be reached Monday - Friday 9:00 - 5:00 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tommy Worden can be reached at (571)272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MADISON B EMMETT/Examiner, Art Unit 3658