DETAILED ACTION
Status of Claims
Claims 1-14 and 16-20 are currently pending and have been examined in this application. This Final Rejection is in response to the amendment submitted on 5/5/2026.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments and Amendments
Applicant’s arguments, filed on 5/5/2026, with respect to interpretation of claims 1-20 under 35 USC 101 have been fully considered and are persuasive. The claims are now directed to a practical application and the interpretation pertaining to 35 USC 101 has been withdrawn.
Applicant’s arguments, filed on 5/5/2026, with respect to the rejection of Claims 1-20 under 35 USC 103 have been fully considered but they are moot in view of the new grounds of rejection provided below, which was necessitated based on Applicant’s amendments to the claims, which changed the scope of the claims. Examiner notes wherein Applicant’s arguments are directed towards the newly amended claim limitation(s), which are addressed by the newly found prior art, as indicated below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-14 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ijiri (US 20210283771 A1) as modified by Oleynik (US 20230031545 A1)
Claim 1:
Ijiri teaches the following limitations:
A method for operating a robot system including a robot body, a robot controller, and at least one sensor, the method comprising: (Ijiri – [0011] … a controller configured to control the robot so as to output an action to be performed by the robot to perform predetermined work on the object, in response to receiving the state observation data, … ; [0037] To observe the state of the robot 10, for example, encoders of the joints of the robot 10, a visual sensor (camera), motion capture, and a force-related sensor can be used. …) accessing, by the robot controller, a first reusable work primitive from [[ (Ijiri - [0014] In one or more embodiments, the predetermined work may include a plurality of motion primitives, and the controller may include a plurality of learning models corresponding to the plurality of motion primitives. The motion primitives are also called operation sections, MPs, or the like. Each of the motion primitives may be an operation with a defined goal, and unique restrictions may be applied to state focused-on variables and actions to be performed.)
accessing, by the robot controller, a first percept associated with the first reusable work primitive, wherein the first percept comprises a first metric for evaluating a state representation in relation to the first reusable work primitive; (Ijiri - [0057] The data obtaining unit 21 obtains data indicating the state of the robot 10. Here, the data to be obtained is called “state observation data”. … ; [0058] The success judgment unit 22 judges whether or not operations performed by the robot 10 meet a predetermined success condition. As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. …) capturing, by the at least one sensor, first sensor data at a first time; determining, by at least one processor of the robot controller, a first state representation for the first time, based on the first sensor data; (Ijiri – [0057] … While described in detail below, the data obtaining unit 21 obtains state observation data from, for example, a joint encoder, a visual sensor (camera), motion capture, a force-related sensor (force sensor, torque sensor, and tactile sensor), and a displacement sensor. ; [0058] … The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.) applying, by the at least one processor, the first percept to the first state representation to determine whether the first metric is satisfied; and (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.)
Examiner Note:
Success Condition corresponds to First Percept
Ijiri does not explicitly teach the following limitations, however Oleynik teaches:
reusable work primitive from a library of reusable work primitives;
(Oleynik - [0297] The minimanipulation library is a command-software repository, where motion behaviors and processes are stored based on an off-line learning process, where the arm/wrist/finger motions and sequences to successfully complete a particular abstract task …)
in response to determining that the first metric is not satisfied, causing the robot
body to perform the first reusable work primitive.
(Oleynik - [0613] Note that should a minimanipulation (MM) sub-routine action fail (such as needing to re-grasp), all the minimanipulation sequencer has to do is to jump back backwards to a prior phase and repeat the same actions (possibly with a modified set of parameters to ensure success, if needed). …)
Examiner’s Note:
(MM) sub-routine action corresponds to first reusable work primitive
to Fail corresponds to first metric is not satisfied
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Ijiri to include a method of storing motion primitives in an accessible library and to further include a means of repeating a work primitive (MM) if it fails as taught in Oleynik. Having the ability to store and access motion primitives in a library system allows the user to easily create programs based on stored sub-programs, update the parameters associated with each motion primitive file and also easily access motion primitives that do not satisfy an expected metric and need to be repeated.
Claim 2:
Ijiri teaches the following limitations:
The method of claim 1, further comprising: if the first metric is satisfied, outputting an indication of metric satisfaction; and if the first metric is not satisfied, outputting an indication of metric non-satisfaction.
(Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.)
Claim 3:
Ijiri teaches the following limitations:
The method of claim 1, further comprising, prior to the first time: capturing, by the at least one sensor, second sensor data at a second time before the first time; determining, by the at least one processor, a goal state based on the second sensor data; (Ijiri - [0088] When learning for one MP has been completed, learning is performed for the next MP. Note that, while the policy parameters θ need to be initialized for each MP, the state transition model used in the learning for the previous MP can be used as an initial model. ; [0089] Thus, by using the above-described processing, a policy corresponding to a plurality of MPs can be acquired.) selecting, by the at least one processor, the first percept and the associated first reusable work primitive by identifying the first metric as indicative of whether the robot system is in the goal state; and (Ijiri - [0094] The success judgment unit 82 judges whether or not an action taken by the robot 10 meets a predetermined success condition. The success judgment unit 82, similarly to when learning is being performed, determines the success condition for each MP.) performing the first reusable work primitive to attempt to transition the robot system towards the goal state. (Ijiri - [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Examiner Note:
A successfully completed task corresponds to goal state
Claim 4:
Ijiri teaches the following limitations:
The method of claim 1, further comprising: determining, by the at least one processor, a goal state; and selecting, by the at least one processor, the first percept and the associated first reusable work primitive based on the goal state.
(Ijiri - [0058] The success judgment unit 22 judges whether or not operations performed by the robot 10 meet a predetermined success condition. As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. Examples of a success condition include, for example, a peg being located in the vicinity of a hole (non-contact), a peg being in contact with a surface near a hole, the leading end of a peg entering a hole, a peg entering a hole and being parallel with the hole, and a peg being completely fitted into a hole. The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.)
Claim 5:
Ijiri teaches the following limitations:
The method of claim 1, wherein the first percept is a Boolean function which returns either true or false.
(Ijiri - [0086] In step S74, a determination is made as to whether the learning apparatus has completed learning of the policy 28. … If the termination condition is not met (S74: NO), the processing proceeds to step S75. If the termination condition is met (S74: YES), learning is terminated.)
Claim 6:
Ijiri teaches the following limitations:
The method of claim 1, wherein the first metric indicates a success state representation, and the first metric is satisfied if the first state representation matches the success state representation.
(Ijiri – [0058] … The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. Examples of a success condition include, for example, a peg being located in the vicinity of a hole (non-contact), a peg being in contact with a surface near a hole, the leading end of a peg entering a hole, a peg entering a hole and being parallel with the hole, and a peg being completely fitted into a hole.)
Claim 7:
Ijiri teaches the following limitations:
The method of claim 1, wherein the first metric indicates at least one failure condition, and the first metric is satisfied if the first state representation does not fulfill any of the at least one failure condition.
(Ijiri - [0098] Note that, in cases where the termination conditions for each MP are not met within a predetermined time step, the robot 10 is subjected to an excessive force, or the robot moves out of a designated area, the task is interrupted and the initial state is returned to.)
Claim 8:
Ijiri teaches the following limitations:
The method of claim 1, wherein: accessing a first reusable work primitive from a library of reusable work primitives comprises: accessing a plurality of reusable work primitives from the [[(Ijiri - [0014] In one or more embodiments, the predetermined work may include a plurality of motion primitives, and the controller may include a plurality of learning models corresponding to the plurality of motion primitives. The motion primitives are also called operation sections, MPs, or the like. Each of the motion primitives may be an operation with a defined goal, and unique restrictions may be applied to state focused-on variables and actions to be performed.) accessing the first percept comprises: accessing a plurality of percepts including the first percept, each percept of the plurality of percepts associated with a respective reusable work primitive of the plurality of reusable work primitives, wherein each percept comprises a respective metric for evaluating a respective state representation in relation to the respective reusable work primitive; (Ijiri – [0014] … the controller may include a plurality of learning models corresponding to the plurality of motion primitives. … ; [0057] … While described in detail below, the data obtaining unit 21 obtains state observation data from, for example, a joint encoder, a visual sensor (camera), motion capture, a force-related sensor (force sensor, torque sensor, and tactile sensor), and a displacement sensor. ; [0058] … The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.) applying the first percept to the first state representation comprises, for each percept in the plurality of percepts: applying the respective percept to at least one respective aspect of the first state representation to determine whether the respective metric is satisfied; and (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.) the method further comprises determining, by the at least one processor, whether the first state representation satisfies a combined metric represented by the plurality of percepts, based on whether the respective metrics are satisfied. (Ijiri - [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Ijiri does not explicitly teach the following limitations, however Oleynik teaches:
work primitives from the library of reusable work primitives, including the first reusable work primitive;
(Oleynik - [0297] The minimanipulation library is a command-software repository, where motion behaviors and processes are stored based on an off-line learning process, where the arm/wrist/finger motions and sequences to successfully complete a particular abstract task …)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Ijiri to include a method of storing motion primitives in an accessible library as taught in Oleynik. Having the ability to store and access motion primitives in a library system allows the user to easily create programs based on stored sub-programs and also allows the user to easily update the parameters associated with each motion primitive file.
Claim 9:
Ijiri teaches the following limitations:
The method of claim 8, wherein determining whether the first state representation satisfies the combined metric comprises: determining that the first state representation satisfies the combined metric if each respective aspect of the first state representation satisfies the respective metric of the respective percept applied to the respective aspect of the first state representation; and (Ijiri - [0096] [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.) determining that the first state representation does not satisfy the combined metric if at least one respective aspect of the first state representation does not satisfy the respective metric of the respective percept applied to the respective aspect of the first state representation. (Ijiri - [0098] Note that, in cases where the termination conditions for each MP are not met within a predetermined time step, the robot 10 is subjected to an excessive force, or the robot moves out of a designated area, the task is interrupted and the initial state is returned to.)
Examiner Note:
The combined policies for the actions of contact, fit, align, and insert correspond to a combined metric.
Claim 10:
Ijiri teaches the following limitations:
The method of claim 8, wherein: applying the first percept to the first state representation comprises, for each percept in the plurality of percepts: if the at least one respective aspect of the first state representation satisfies the respective metric, outputting an indication of metric satisfaction; and if the at least one respective aspect of the first state representation does not satisfy the respective metric, outputting an indication of metric non-satisfaction; and (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.) determining whether the first state representation satisfies a combined metric represented by the plurality of percepts comprises determining whether the first state representation satisfies the combined metric based on respective indications of metric satisfaction or metric non-satisfaction for each percept. (Ijiri - [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Examiner Note:
Positive reward and negative reward correspond to an output indicating that the metric is satisfied or not satisfied
Claim 11:
Ijiri teaches the following limitations:
The method of claim 10, wherein: determining whether the first state representation satisfies the combined metric comprises: evaluating a Boolean logic function which takes the indications of metric satisfaction or metric non-satisfaction for each respective aspect of the first state representation as input.
(Ijiri - [0086] In step S74, a determination is made as to whether the learning apparatus has completed learning of the policy 28. … If the termination condition is not met (S74: NO), the processing proceeds to step S75. If the termination condition is met (S74: YES), learning is terminated. ; [0098] Note that, in cases where the termination conditions for each MP are not met within a predetermined time step, the robot 10 is subjected to an excessive force, or the robot moves out of a designated area, the task is interrupted and the initial state is returned to.)
Claim 12:
Ijiri teaches the following limitations:
The method of claim 1, further comprising: capturing, by the at least one sensor, second sensor data at a second time subsequent to the first time; determining, by the at least one processor, a second state representation for the second time, based on the second sensor data; and applying, by the at least one processor, the first percept to the second state representation to determine whether the first metric is satisfied.
(Ijiri - [0087] In step S75, the operation determination unit 27 applies the mid-learning policy 28 and determines the next movement u(t+1), and the data obtaining unit 21 observes the resulting state. The processing returns to step S72, and learning using the thus obtained state observation data (updating of state transition model and updating of policy) is repeated.; [0088] When learning for one MP has been completed, learning is performed for the next MP. Note that, while the policy parameters θ need to be initialized for each MP, the state transition model used in the learning for the previous MP can be used as an initial model. ; [0089] Thus, by using the above-described processing, a policy corresponding to a plurality of MPs can be acquired.)
Claim 13:
Ijiri teaches the following limitations:
A method for operating a robot system including a robot body, a robot controller, and at least one sensor, the method comprising: (Ijiri – [0011] … a controller configured to control the robot so as to output an action to be performed by the robot to perform predetermined work on the object, in response to receiving the state observation data, … ; [0037] To observe the state of the robot 10, for example, encoders of the joints of the robot 10, a visual sensor (camera), motion capture, and a force-related sensor can be used. …) identifying, by the at least one processor, a workflow to complete a work objective, the workflow comprising (Ijiri – [0047] … The robot 10 performs fitting work which involves fitting a part (a peg, etc.) gripped by the gripper 12 into a hole.) a plurality of reusable work primitives (Ijiri – [0058] … As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). …)
accessing, by the robot controller, each reusable work primitive in the plurality of reusable work primitives; (Ijiri - [0014] In one or more embodiments, the predetermined work may include a plurality of motion primitives, and the controller may include a plurality of learning models corresponding to the plurality of motion primitives. The motion primitives are also called operation sections, MPs, or the like. Each of the motion primitives may be an operation with a defined goal, and unique restrictions may be applied to state focused-on variables and actions to be performed.) accessing, by the robot controller, a plurality of percepts, each percept in the plurality of percepts associated with a respective reusable work primitive in the plurality of reusable work primitives, (Ijiri - [0058] The success judgment unit 22 judges whether or not operations performed by the robot 10 meet a predetermined success condition. As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. Examples of a success condition include, for example, a peg being located in the vicinity of a hole (non-contact), a peg being in contact with a surface near a hole, the leading end of a peg entering a hole, a peg entering a hole and being parallel with the hole, and a peg being completely fitted into a hole. …) each percept in the plurality of percepts comprising a respective metric for evaluating a state representation in relation to the respective reusable work primitive; and (Ijiri – [0058] … The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.) performing the workflow, wherein for each reusable work primitive in the plurality of reusable work primitives, performing the workflow comprises: capturing, by the at least one sensor, respective sensor data at a respective time; determining, by the at least one processor, the respective state representation of for the respective time based on the respective sensor data; and (Ijiri – [0061] … That is, a state transition model updating unit 25 of the learning unit 24 learns a state transition model of the robot 10, and a policy updating unit 26 of the learning unit 24 uses the learned state transmission model to learn the policy 28 of the robot 10. The state transition model receives input of the state x(t) and the action u(t) at the time of the state x(t) and outputs a post-action state x(t+1). The policy 28 is a model that receives input of the state x(t) and outputs an action u(t) that is to be taken. …) applying the respective percept to the respective state representation to determine whether the respective metric is satisfied; and (Ijiri - [0096] [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Ijiri does not explicitly teach the following limitations, however Oleynik teaches:
available in a library of reusable work primitives;
(Oleynik - [0297] The minimanipulation library is a command-software repository, where motion behaviors and processes are stored based on an off-line learning process, where the arm/wrist/finger motions and sequences to successfully complete a particular abstract task …)
Ijiri does not explicitly teach the following limitations, however Oleynik teaches:
in response to determining that the respective metric is not satisfied,
causing the robot body to perform the reusable work primitive.
(Oleynik - [0613] Note that should a minimanipulation (MM) sub-routine action fail (such as needing to re-grasp), all the minimanipulation sequencer has to do is to jump back backwards to a prior phase and repeat the same actions (possibly with a modified set of parameters to ensure success, if needed). …)
Examiner’s Note:
(MM) sub-routine action corresponds to first reusable work primitive
to Fail corresponds to first metric is not satisfied
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Ijiri to include a method of storing motion primitives in an accessible library and to further include a means of repeating a work primitive (MM) if it fails as taught in Oleynik. Having the ability to store and access motion primitives in a library system allows the user to easily create programs based on stored sub-programs, update the parameters associated with each motion primitive file and also easily access motion primitives that do not satisfy an expected metric and need to be repeated.
Claim 14:
Ijiri teaches the following limitations:
The method of claim 13, further comprising, for each reusable work primitive in the plurality of reusable work primitives, prior to each respective time: determining, by the at least one processor, a respective goal state for the respective reusable work primitive based on the workflow; and (Ijiri - [0058] The success judgment unit 22 judges whether or not operations performed by the robot 10 meet a predetermined success condition. As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. Examples of a success condition include, for example, a peg being located in the vicinity of a hole (non-contact), a peg being in contact with a surface near a hole, the leading end of a peg entering a hole, a peg entering a hole and being parallel with the hole, and a peg being completely fitted into a hole. The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.) performing the respective reusable work primitive to attempt to transition the robot system towards the respective goal state as part of performing the workflow. (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met. ; [0060] The learning unit 24 learns a policy 28 (learning model) for controlling the robot 10. When state observation data regarding the robot 10 is input, the learning unit 24 outputs the appropriate action of the next step to perform predetermined work, based on the policy 28.)
Claim 16:
Ijiri teaches the following limitations:
The method of claim 13, wherein each percept is a Boolean function which returns either true or false.
(Ijiri - [0086] In step S74, a determination is made as to whether the learning apparatus has completed learning of the policy 28. … If the termination condition is not met (S74: NO), the processing proceeds to step S75. If the termination condition is met (S74: YES), learning is terminated.)
Claim 17:
Ijiri teaches the following limitations:
The method of claim 13, wherein performing the workflow further comprises:
determining whether a combined metric is satisfied based on whether the respective metrics are satisfied; and the combined metric is represented by the plurality of percepts and is for evaluating whether a combined state representation in relation to the work objective is satisfied.
(Ijiri - [0094] The success judgment unit 82 judges whether or not an action taken by the robot 10 meets a predetermined success condition. The success judgment unit 82, similarly to when learning is being performed, determines the success condition for each MP. ; [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Claim 18:
Ijiri teaches the following limitations:
The method of claim 17, wherein determining whether the combined metric is satisfied comprises: determining that the combined metric is satisfied if each respective state representation satisfies the respective metric of the respective percept applied to the respective state representation; and (Ijiri - [0094] The success judgment unit 82 judges whether or not an action taken by the robot 10 meets a predetermined success condition. The success judgment unit 82, similarly to when learning is being performed, determines the success condition for each MP. determining that the combined metric is not satisfied if at least one respective state representation does not satisfy the respective metric of the respective percept applied to the respective state representation. (Ijiri - [0098] Note that, in cases where the termination conditions for each MP are not met within a predetermined time step, the robot 10 is subjected to an excessive force, or the robot moves out of a designated area, the task is interrupted and the initial state is returned to.)
Claim 19:
Ijiri teaches the following limitations:
The method of claim 17, wherein: for each reusable work primitive in the plurality of reusable work primitives, applying the respective percept to the respective state representation comprises, for each reusable work primitive in the plurality of reusable work primitives: if the respective state representation satisfies the respective metric, outputting an indication of metric satisfaction for the respective percept; and (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.) if the respective state representation does not satisfy the respective metric, outputting an indication of metric non-satisfaction for the respective percept; and determining whether the combined metric is satisfied comprises: determining whether the combined metric is satisfied based on respective indications of metric satisfaction or metric non-satisfaction for each percept. (Ijiri - [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Examiner Note:
Positive reward and negative reward correspond to an output indicating that the metric is satisfied or not satisfied
Claim 20:
Ijiri does not explicitly teach the following limitations, however Oleynik teaches:
The method of claim 13, further comprising: capturing, by the at least one sensor, additional sensor data subsequent to the respective times; (Oleynik - [0659] FIG. 121 is a block diagram illustrating the real-time parameter adjustment during the execution phase of minimanipulations in accordance with the present disclosure. … As seen in FIG. 120, each parameter to complete the minimanipulation to achieve the functional result may require different parameters for the left hand. Specifically, each change in a parameter sensed by the right hand as a result of a parameter in the first object make impact the parameters used by the left hand and the parameters of the object in the left had.) determining, by the at least one processor, an additional state representation based on the additional sensor data; and (Oleynik - [0660] In an embodiment, in order to complete minimanipulations 1-.1-1.3, to yield the functional result, right hand and left hand must sense and receive feedback on the object and the state change of the object in the hand or palm, or leg. …) applying, by the at least one processor, a final percept of the plurality of percepts to the additional state representation to determine whether the respective metric for the final percept is satisfied. (Oleynik – [0660] … This sensed state change may result in an adjustment to the parameters that comprise the minimanipulation. Each change in one parameter may yield in a change to each subsequent parameter and each subsequent required minimanipulation until the desired tasks result is achieved.)
Examiner Note:
Since the specification does not seem have a definitive meaning for the term “Additional”, the examiner is interpreting “Additional” to mean “real-time” (i.e. data handled during robot operation) with respect to Additional sensor data and Additional state representation.
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Ijiri to include a method of incorporating real-time sensor data in order to update robot state observations and therefore also update the parameters associated with the motion primitives as taught in Oleynik. Having the ability to update the parameters of the motion primitives while work tasks are in process, provides the robot with greater accuracy when evaluating the results of robot tasks which are a sequence of combined motion primitives.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure or directed to the state of the art is listed on the enclosed PTO-892.
The following is a brief description for relevant prior art that was cited but not applied:
Natarajan (US 20210229281 A1) describes methods, systems, and use cases include techniques for training or using a model to control a robot, including a set of action primitives applicable to a set of robots, receiving information corresponding to a task (e.g., a collaborative task), and determining at least one action primitive based on the received information.
Jha (US 20240131698 A1) describes a robotic controller which generates sequences of movement primitives for sequential tasks of a robot having a manipulator. The controller includes a method for storing a dictionary of movement primitives, a pretrained learning module, and a graph-search based planning module having instructions stored thereon. The controller acquires a planned task provided by an interface device operated by a user, wherein the planned task is represented by an initial state and a goal state with respect to an object.
Bhat (US 20210331316 A1) describes methods and systems for improved control of robotic arms. In one embodiment, a method is presented that includes predefining a plurality of motion primitives, which may include one or more preconditions and effects. A target state for a plurality of workpieces may be determined as well as an initial state of the plurality of workpieces. A sequence of operations may be generated based on the preconditions and/or effects of the motion primitives, as well as the target state and the initial state. Executing the sequence of operations may be capable of changing the plurality of workpieces from the initial state to the target state.
Dong (US 20230136402 A1) describes an industrial robot motion accuracy compensation method includes: establishing a motion parameter database, wherein the motion parameter database includes a plurality of different reference operating conditions and a motion parameter of the industrial robot corresponding to each reference operating condition, and each reference operating condition is formed by combining each element in each set in a total set of operation conditions; acquiring a current operating condition of the industrial robot; determining whether there is a reference operating condition matched with the current operating condition in the motion parameter database
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALAN LINDSAY OSTROW whose telephone number is (703)756-1854. The examiner can normally be reached M-F 8 - 5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Mott can be reached on (571) 270 5376. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALAN LINDSAY OSTROW/
Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights