DETAILED ACTION
Status of Claims
Claims 1-20 are currently pending and have been examined in this application. This Non-final communication is the first action on the merits.
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
The claims are directed to a system or method, which is one of the statutory categories of invention. (Step 1: YES)
The examiner has identified system Claim 1 as the claim that represents the claimed invention for analysis and is similar to Claim 13. Claim 1 recites the limitations of (additional elements emphasized in bold are considered to be parsed from the remaining abstract idea):
A method for operating a robot system including a robot body, a robot controller, and at least one sensor, the method comprising: accessing, by the robot controller, a first reusable work primitive from a library of reusable work primitives; accessing, by the robot controller, a first percept associated with the first reusable work primitive, wherein the first percept comprises a first metric for evaluating a state representation in relation to the first reusable work primitive; capturing, by the at least one sensor, first sensor data at a first time; determining, by at least one processor of the robot controller, a first state representation for the first time, based on the first sensor data; and applying, by the at least one processor, the first percept to the first state representation to determine whether the first metric is satisfied.
which under its broadest reasonable interpretation, covers performance of the limitation(s) as a mental process (concept performed in the human mind) to accessing a first reusable work primitive from a library , accessing a first percept associated with the first reusable work primitive, capturing first sensor data at a first time, determining a first state representation and applying the first percept to the first state representation to determine whether the first metric is satisfied. One of ordinary skill in the art could obtain (“access”) work primitive information form a library, obtain (access) a first percept, which is a file containing robot metrics for comparison, collect (capture) a set of sensor data, determine the first state of the robot from the sensor data and compare (apply) this sensor data to the robot metrics information and the current state of the robot to see if the metrics have been satisfied.
Similarly, if a claim limitation under its BRI, covers performance of the limitation in the human mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. (Claims can recite a mental process even if they are claimed as being performed on a computer Gottschalk v. Benson, 409 U.S. 63; “Courts have examined claims that required the use of a computer and still found that the underlying, patent-ineligible invention could be performed via pen and paper or in a person’s mind.” Versata Dev. Group v. SAP Am., Inc., 793 F. 3d 1306, 1335, 115 USPQ2d 1681, 1702. (Fed. Cir. 2015.))
Accordingly, the claim recites an abstract idea (Step 2A- Prong 1: YES. The claims are abstract).
This judicial exception is not integrated into a practical application. Limitations that are not indicative of integration into a practical application include: (1) Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea (MPEP 2106.05.f), (2) Adding insignificant extra-solution activity to the judicial exception (MPEP 2106.05.g), (3) Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05.h).
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”):
A method for operating a robot system including a robot body, a robot controller, and at least one sensor, the method comprising: accessing, by the robot controller, a first reusable work primitive from a library of reusable work primitives; accessing, by the robot controller, a first percept associated with the first reusable work primitive, wherein the first percept comprises a first metric for evaluating a state representation in relation to the first reusable work primitive; capturing, by the at least one sensor, first sensor data at a first time; determining, by at least one processor of the robot controller, a first state representation for the first time, based on the first sensor data; and applying, by the at least one processor, the first percept to the first state representation to determine whether the first metric is satisfied.
The robot body, robot controller, and sensor in Claim 1 is just using generic computer components. The computer hardware is recited at a high level of generality such that it amounts to no more than mere instructions to implement an abstract idea by adding the words “apply it” (or an equivalent) with the judicial exception. Accordingly, these additional elements, when considered separately and as an ordered combination, do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore claim 1 is directed to an abstract idea without a practical application. (Step 2A-Prong 2: NO. The additional claimed elements are not integrated into a practical application)
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because, when considered separately and as an ordered combination, they do not add significantly more (also known as an “inventive concept”) to the exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using computer hardware amounts to no more than mere instructions to implement an abstract idea by adding the words “apply it” (or an equivalent) with the judicial exception. Mere instructions to implement an abstract idea on or with the use of generic computer components, cannot provide an inventive concept - rendering the claim patent ineligible. Thus claim 1 is not patent eligible. (Step 2B: NO. The claims do not provide significantly more).
The dependent claims further define the abstract idea that is present in their respective independent claims and hence are abstract for at least the reasons presented above. The dependent claims do not include any additional elements that integrate the abstract idea into a practical application or are sufficient to amount to significantly more than the judicial exception when considered both individually and as an ordered combination. Therefore, the dependent claims are directed to an abstract idea. Thus, the aforementioned claims are not patent-eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-19 are rejected under 35 U.S.C. 103 as being unpatentable over Ijiri (US 20210283771 A1) as modified by Natarajan (US 20210229281 A1)
Claim 1:
Ijiri teaches the following limitations:
A method for operating a robot system including a robot body, a robot controller, and at least one sensor, the method comprising: (Ijiri – [0011] … a controller configured to control the robot so as to output an action to be performed by the robot to perform predetermined work on the object, in response to receiving the state observation data, … ; [0037] To observe the state of the robot 10, for example, encoders of the joints of the robot 10, a visual sensor (camera), motion capture, and a force-related sensor can be used. …) accessing, by the robot controller, a first reusable work primitive from [[ (Ijiri - [0014] In one or more embodiments, the predetermined work may include a plurality of motion primitives, and the controller may include a plurality of learning models corresponding to the plurality of motion primitives. The motion primitives are also called operation sections, MPs, or the like. Each of the motion primitives may be an operation with a defined goal, and unique restrictions may be applied to state focused-on variables and actions to be performed.)
accessing, by the robot controller, a first percept associated with the first reusable work primitive, wherein the first percept comprises a first metric for evaluating a state representation in relation to the first reusable work primitive; (Ijiri - [0057] The data obtaining unit 21 obtains data indicating the state of the robot 10. Here, the data to be obtained is called “state observation data”. … ; [0058] The success judgment unit 22 judges whether or not operations performed by the robot 10 meet a predetermined success condition. As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. …) capturing, by the at least one sensor, first sensor data at a first time; determining, by at least one processor of the robot controller, a first state representation for the first time, based on the first sensor data; and (Ijiri – [0057] … While described in detail below, the data obtaining unit 21 obtains state observation data from, for example, a joint encoder, a visual sensor (camera), motion capture, a force-related sensor (force sensor, torque sensor, and tactile sensor), and a displacement sensor. ; [0058] … The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.) applying, by the at least one processor, the first percept to the first state representation to determine whether the first metric is satisfied. (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.)
Ijiri does not explicitly teach the following limitations, however Natarajan teaches:
reusable work primitive from a library of reusable work primitives;
(Natarajan - [0019] … These primitives may be stored in a library. Previously learned action or interaction primitives may be combined or sequenced by a coordination algorithm to achieve a complex collaborative robotic task (e.g., rather than learning a task entirely from scratch without primitives). This technique may significantly reduce the state-action space. The systems and methods described herein may significantly simplify the creation of collaborative multi-agent applications.)
Examiner Note:
Success Condition corresponds to First Percept
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Ijiri to include a method of storing motion primitives in an accessible library as taught in Natarajan. Having the ability to store and access motion primitives in a library system allows the user to easily create programs based on stored sub-programs and also allows the user to easily update the parameters associated with each motion primitive file.
Claim 2:
Ijiri teaches the following limitations:
The method of claim 1, further comprising: if the first metric is satisfied, outputting an indication of metric satisfaction; and if the first metric is not satisfied, outputting an indication of metric non-satisfaction.
(Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.)
Claim 3:
Ijiri teaches the following limitations:
The method of claim 1, further comprising, prior to the first time: capturing, by the at least one sensor, second sensor data at a second time before the first time; determining, by the at least one processor, a goal state based on the second sensor data; (Ijiri - [0088] When learning for one MP has been completed, learning is performed for the next MP. Note that, while the policy parameters θ need to be initialized for each MP, the state transition model used in the learning for the previous MP can be used as an initial model. ; [0089] Thus, by using the above-described processing, a policy corresponding to a plurality of MPs can be acquired.) selecting, by the at least one processor, the first percept and the associated first reusable work primitive by identifying the first metric as indicative of whether the robot system is in the goal state; and (Ijiri - [0094] The success judgment unit 82 judges whether or not an action taken by the robot 10 meets a predetermined success condition. The success judgment unit 82, similarly to when learning is being performed, determines the success condition for each MP.) performing the first reusable work primitive to attempt to transition the robot system towards the goal state. (Ijiri - [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Examiner Note:
A successfully completed task corresponds to goal state
Claim 4:
Ijiri teaches the following limitations:
The method of claim 1, further comprising: determining, by the at least one processor, a goal state; selecting, by the at least one processor, the first percept and the associated first reusable work primitive by identifying the first metric as indicative of whether the robot system is in the goal state; (Ijiri - [0058] The success judgment unit 22 judges whether or not operations performed by the robot 10 meet a predetermined success condition. As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. Examples of a success condition include, for example, a peg being located in the vicinity of a hole (non-contact), a peg being in contact with a surface near a hole, the leading end of a peg entering a hole, a peg entering a hole and being parallel with the hole, and a peg being completely fitted into a hole. The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.) if the first state representation satisfies the first metric, determining that the robot system is in the goal state; and (Ijiri – [0059] … The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, …)
if the first state representation does not satisfy the first metric, performing the first reusable work primitive to attempt to transition the robot system towards the goal state. (Ijiri – [0059] … and a negative reward is given when a state is far from one where a success condition is met.)
Claim 5:
Ijiri teaches the following limitations:
The method of claim 1, wherein the first percept is a Boolean function which returns either true or false.
(Ijiri - [0086] In step S74, a determination is made as to whether the learning apparatus has completed learning of the policy 28. … If the termination condition is not met (S74: NO), the processing proceeds to step S75. If the termination condition is met (S74: YES), learning is terminated.)
Claim 6:
Ijiri teaches the following limitations:
The method of claim 1, wherein the first metric indicates a success state representation, and the first metric is satisfied if the first state representation matches the success state representation.
(Ijiri – [0058] … The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. Examples of a success condition include, for example, a peg being located in the vicinity of a hole (non-contact), a peg being in contact with a surface near a hole, the leading end of a peg entering a hole, a peg entering a hole and being parallel with the hole, and a peg being completely fitted into a hole.)
Claim 7:
Ijiri teaches the following limitations:
The method of claim 1, wherein the first metric indicates at least one failure condition, and the first metric is satisfied if the first state representation does not fulfill any of the at least one failure condition.
(Ijiri - [0098] Note that, in cases where the termination conditions for each MP are not met within a predetermined time step, the robot 10 is subjected to an excessive force, or the robot moves out of a designated area, the task is interrupted and the initial state is returned to.)
Claim 8:
Ijiri teaches the following limitations:
The method of claim 1, wherein: accessing a first reusable work primitive from a library of reusable work primitives comprises: accessing a plurality of reusable work primitives from the [[(Ijiri - [0014] In one or more embodiments, the predetermined work may include a plurality of motion primitives, and the controller may include a plurality of learning models corresponding to the plurality of motion primitives. The motion primitives are also called operation sections, MPs, or the like. Each of the motion primitives may be an operation with a defined goal, and unique restrictions may be applied to state focused-on variables and actions to be performed.) accessing the first percept comprises: accessing a plurality of percepts including the first percept, each percept of the plurality of percepts associated with a respective reusable work primitive of the plurality of reusable work primitives, wherein each percept comprises a respective metric for evaluating a respective state representation in relation to the respective reusable work primitive; (Ijiri – [0014] … the controller may include a plurality of learning models corresponding to the plurality of motion primitives. … ; [0057] … While described in detail below, the data obtaining unit 21 obtains state observation data from, for example, a joint encoder, a visual sensor (camera), motion capture, a force-related sensor (force sensor, torque sensor, and tactile sensor), and a displacement sensor. ; [0058] … The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.) applying the first percept to the first state representation comprises, for each percept in the plurality of percepts: applying the respective percept to at least one respective aspect of the first state representation to determine whether the respective metric is satisfied; and (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.) the method further comprises determining, by the at least one processor, whether the first state representation satisfies a combined metric represented by the plurality of percepts, based on whether the respective metrics are satisfied. (Ijiri - [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Ijiri does not explicitly teach the following limitations, however Natarajan teaches:
work primitives from the library of reusable work primitives, including the first reusable work primitive;
(Natarajan - [0019] … These primitives may be stored in a library. Previously learned action or interaction primitives may be combined or sequenced by a coordination algorithm to achieve a complex collaborative robotic task (e.g., rather than learning a task entirely from scratch without primitives). This technique may significantly reduce the state-action space. The systems and methods described herein may significantly simplify the creation of collaborative multi-agent applications.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Ijiri to include a method of storing motion primitives in an accessible library as taught in Natarajan. Having the ability to store and access motion primitives in a library system allows the user to easily create programs based on stored sub-programs and also allows the user to easily update the parameters associated with each motion primitive file.
Claim 9:
Ijiri teaches the following limitations:
The method of claim 8, wherein determining whether the first state representation satisfies the combined metric comprises: determining that the first state representation satisfies the combined metric if each respective aspect of the first state representation satisfies the respective metric of the respective percept applied to the respective aspect of the first state representation; and (Ijiri - [0096] [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.) determining that the first state representation does not satisfy the combined metric if at least one respective aspect of the first state representation does not satisfy the respective metric of the respective percept applied to the respective aspect of the first state representation. (Ijiri - [0098] Note that, in cases where the termination conditions for each MP are not met within a predetermined time step, the robot 10 is subjected to an excessive force, or the robot moves out of a designated area, the task is interrupted and the initial state is returned to.)
Examiner Note:
The combined policies for the actions of contact, fit, align, and insert correspond to a combined metric.
Claim 10:
Ijiri teaches the following limitations:
The method of claim 8, wherein: applying the first percept to the first state representation comprises, for each percept in the plurality of percepts: if the at least one respective aspect of the first state representation satisfies the respective metric, outputting an indication of metric satisfaction; and if the at least one respective aspect of the first state representation does not satisfy the respective metric, outputting an indication of metric non-satisfaction; and (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.) determining whether the first state representation satisfies a combined metric represented by the plurality of percepts comprises determining whether the first state representation satisfies the combined metric based on respective indications of metric satisfaction or metric non-satisfaction for each percept. (Ijiri - [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Examiner Note:
Positive reward and negative reward correspond to an output indicating that the metric is satisfied or not satisfied
Claim 11:
Ijiri teaches the following limitations:
The method of claim 10, wherein: determining whether the first state representation satisfies the combined metric comprises: evaluating a Boolean logic function which takes the indications of metric satisfaction or metric non-satisfaction for each respective aspect of the first state representation as input.
(Ijiri - [0086] In step S74, a determination is made as to whether the learning apparatus has completed learning of the policy 28. … If the termination condition is not met (S74: NO), the processing proceeds to step S75. If the termination condition is met (S74: YES), learning is terminated. ; [0098] Note that, in cases where the termination conditions for each MP are not met within a predetermined time step, the robot 10 is subjected to an excessive force, or the robot moves out of a designated area, the task is interrupted and the initial state is returned to.)
Claim 12:
Ijiri teaches the following limitations:
The method of claim 1, further comprising: capturing, by the at least one sensor, second sensor data at a second time subsequent the first time; determining, by the at least one processor, a second state representation for the second time, based on the second sensor data; and applying, by the at least one processor, the first percept to the second state representation to determine whether the first metric is satisfied.
(Ijiri - [0087] In step S75, the operation determination unit 27 applies the mid-learning policy 28 and determines the next movement u(t+1), and the data obtaining unit 21 observes the resulting state. The processing returns to step S72, and learning using the thus obtained state observation data (updating of state transition model and updating of policy) is repeated.; [0088] When learning for one MP has been completed, learning is performed for the next MP. Note that, while the policy parameters θ need to be initialized for each MP, the state transition model used in the learning for the previous MP can be used as an initial model. ; [0089] Thus, by using the above-described processing, a policy corresponding to a plurality of MPs can be acquired.)
Claim 13:
Ijiri teaches the following limitations:
A method for operating a robot system including a robot body, a robot controller, and at least one sensor, the method comprising: (Ijiri – [0011] … a controller configured to control the robot so as to output an action to be performed by the robot to perform predetermined work on the object, in response to receiving the state observation data, … ; [0037] To observe the state of the robot 10, for example, encoders of the joints of the robot 10, a visual sensor (camera), motion capture, and a force-related sensor can be used. …) identifying, by the at least one processor, a workflow to complete a work objective, the workflow comprising (Ijiri – [0047] … The robot 10 performs fitting work which involves fitting a part (a peg, etc.) gripped by the gripper 12 into a hole.)
a plurality of reusable work primitives (Ijiri – [0058] … As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). …)
accessing, by the robot controller, each reusable work primitive in the plurality of reusable work primitives; (Ijiri - [0014] In one or more embodiments, the predetermined work may include a plurality of motion primitives, and the controller may include a plurality of learning models corresponding to the plurality of motion primitives. The motion primitives are also called operation sections, MPs, or the like. Each of the motion primitives may be an operation with a defined goal, and unique restrictions may be applied to state focused-on variables and actions to be performed.) accessing, by the robot controller, a plurality of percepts, each percept in the plurality of percepts associated with a respective reusable work primitive in the plurality of reusable work primitives, (Ijiri - [0058] The success judgment unit 22 judges whether or not operations performed by the robot 10 meet a predetermined success condition. As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. Examples of a success condition include, for example, a peg being located in the vicinity of a hole (non-contact), a peg being in contact with a surface near a hole, the leading end of a peg entering a hole, a peg entering a hole and being parallel with the hole, and a peg being completely fitted into a hole. …) each percept in the plurality of percepts comprising a respective metric for evaluating a state representation in relation to the respective reusable work primitive; and (Ijiri – [0058] … The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.) performing the workflow, wherein for each reusable work primitive in the plurality of reusable work primitives, the method further comprises: capturing, by the at least one sensor, respective sensor data at a respective time; determining, by the at least one processor, the respective state representation of for the respective time, based on the respective sensor data; and (Ijiri – [0061] … That is, a state transition model updating unit 25 of the learning unit 24 learns a state transition model of the robot 10, and a policy updating unit 26 of the learning unit 24 uses the learned state transmission model to learn the policy 28 of the robot 10. The state transition model receives input of the state x(t) and the action u(t) at the time of the state x(t) and outputs a post-action state x(t+1). The policy 28 is a model that receives input of the state x(t) and outputs an action u(t) that is to be taken. …) applying the respective percept to the respective state representation to determine whether the respective metric is satisfied. (Ijiri - [0096] [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Ijiri does not explicitly teach the following limitations, however Natarajan teaches:
available in a library of reusable work primitives;
(Natarajan - [0019] … These primitives may be stored in a library. Previously learned action or interaction primitives may be combined or sequenced by a coordination algorithm to achieve a complex collaborative robotic task (e.g., rather than learning a task entirely from scratch without primitives). This technique may significantly reduce the state-action space. The systems and methods described herein may significantly simplify the creation of collaborative multi-agent applications.)
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Ijiri to include a method of storing motion primitives in an accessible library as taught in Natarajan. Having the ability to store and access motion primitives in a library system allows the user to easily create programs based on stored sub-programs and also allows the user to easily update the parameters associated with each motion primitive file.
Claim 14:
Ijiri teaches the following limitations:
The method of claim 13, further comprising, for each reusable work primitive in the plurality of reusable work primitives, prior to each respective time: determining, by the at least one processor, a respective goal state for the respective reusable work primitive based on the workflow; and (Ijiri - [0058] The success judgment unit 22 judges whether or not operations performed by the robot 10 meet a predetermined success condition. As described below, in one or more embodiments, one kind of work (skill) such as peg fitting work is learned by being divided into a plurality of motion primitives (MP). The success judgment unit 22 judges whether or not the success condition defined for each MP has been met. Examples of a success condition include, for example, a peg being located in the vicinity of a hole (non-contact), a peg being in contact with a surface near a hole, the leading end of a peg entering a hole, a peg entering a hole and being parallel with the hole, and a peg being completely fitted into a hole. The success judgment unit 22 may make a judgment based on state observation data or based on data different from state observation data.) performing the respective reusable work primitive to attempt to transition the robot system towards the respective goal state as part of performing the workflow. (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met. ; [0060] The learning unit 24 learns a policy 28 (learning model) for controlling the robot 10. When state observation data regarding the robot 10 is input, the learning unit 24 outputs the appropriate action of the next step to perform predetermined work, based on the policy 28.)
Claim 15:
Ijiri teaches the following limitations:
The method of claim 13, further comprising, for each reusable work primitive in the plurality of reusable work primitives: determining, by the at least one processor, a respective goal state; if the respective state representation satisfies the respective metric, determining that the robot system is in the respective goal state; and (Ijiri - [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.) if the respective state representation does not satisfy the respective metric, performing the respective reusable work primitive to attempt to transition the robot system towards the respective goal state as part of performing the workflow. (Ijiri - [0104] From these results, regardless of whether dimension reduction is performed or not, it can be understood that each operation has been learned with sufficient accuracy. However, it can be appreciated that performing dimension reduction means that learning can be performed with few trials and with little error. ; [0106] FIGS. 10A to 10C are graphs showing, for each of the [Fit], [Align], and [Insert] MPs, an error at each trial. The horizontal axis shows the number of trials, and the vertical axis shows the error from the target. Learning was performed over 10 sessions, and the polygonal line graphs show the averages of the 10 sessions and the error bars show the standard deviation. It can be understood that, for each of the [Fit], [Align], and [Insert] MPs, the errors were sufficiently reduced within five trials).
Claim 16:
Ijiri teaches the following limitations:
The method of claim 13, wherein each percept is a Boolean function which returns either true or false.
(Ijiri - [0086] In step S74, a determination is made as to whether the learning apparatus has completed learning of the policy 28. … If the termination condition is not met (S74: NO), the processing proceeds to step S75. If the termination condition is met (S74: YES), learning is terminated.)
Claim 17:
Ijiri teaches the following limitations:
The method of claim 13, wherein: The method further comprises determining whether a combined metric is satisfied based on whether the respective metrics are satisfied; and the combined metric is represented by the plurality of percepts and is for evaluating whether a combined state representation in relation to the work objective is satisfied. (Ijiri - [0094] The success judgment unit 82 judges whether or not an action taken by the robot 10 meets a predetermined success condition. The success judgment unit 82, similarly to when learning is being performed, determines the success condition for each MP. ; [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Claim 18:
Ijiri teaches the following limitations:
The method of claim 17, wherein determining whether the combined metric is satisfied comprises: determining that the combined metric is satisfied if each respective state representation satisfies the respective metric of the respective percept applied to the respective state representation; and (Ijiri - [0094] The success judgment unit 82 judges whether or not an action taken by the robot 10 meets a predetermined success condition. The success judgment unit 82, similarly to when learning is being performed, determines the success condition for each MP. determining that the combined metric is not satisfied if at least one respective state representation does not satisfy the respective metric of the respective percept applied to the respective state representation. (Ijiri - [0098] Note that, in cases where the termination conditions for each MP are not met within a predetermined time step, the robot 10 is subjected to an excessive force, or the robot moves out of a designated area, the task is interrupted and the initial state is returned to.)
Claim 19:
Ijiri teaches the following limitations:
The method of claim 17, wherein: for each reusable work primitive in the plurality of reusable work primitives, applying the respective percept to the respective state representation comprises, for each reusable work primitive in the plurality of reusable work primitives: if the respective state representation satisfies the respective metric, outputting an indication of metric satisfaction for the respective percept; and (Ijiri - [0059] The reward determination unit 23 determines a reward r(t) relative to an action u(t) in a state x(t). For example, the reward determination unit 23 only gives a positive reward when success has been judged by the success judgment unit 22 and otherwise gives no reward. The reward determination unit 23 may be set so that a positive reward is given when a state is close to one that meets a success condition, and a negative reward is given when a state is far from one where a success condition is met.) if the respective state representation does not satisfy the respective metric, outputting an indication of metric non-satisfaction for the respective percept; and determining whether the combined metric is satisfied comprises: determining whether the combined metric is satisfied based on respective indications of metric satisfaction or metric non-satisfaction for each percept. (Ijiri - [0096] If it is determined that the success condition(s) up to the current MP have been met by the success judgment unit 82, the controller 84 makes a switch to the policy 88 corresponding to the next MP. Specifically, if [Approach] is successful, the policy is switched to one corresponding to [Contact], if [Contact] is successful, the policy is switched to one corresponding to [Fit], if [Fit] is successful, the policy is switched to one corresponding to [Align], and if [Align] is successful, the policy is switched to one corresponding to [Insert] . If [Insert] is successful, the peg fitting work is judged as being complete.)
Examiner Note:
Positive reward and negative reward correspond to an output indicating that the metric is satisfied or not satisfied
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Ijiri (US 20210283771 A1) as modified by Natarajan (US 20210229281 A1) in view of Oleynik (US 20230031545 A1)
Claim 20:
Ijiri in combination with Natarajan does not explicitly teach the following limitations, however Oleynik teaches:
The method of claim 13, further comprising: capturing, by the at least one sensor, additional sensor data subsequent the respective times; (Oleynik - [0659] FIG. 121 is a block diagram illustrating the real-time parameter adjustment during the execution phase of minimanipulations in accordance with the present disclosure. … As seen in FIG. 120, each parameter to complete the minimanipulation to achieve the functional result may require different parameters for the left hand. Specifically, each change in a parameter sensed by the right hand as a result of a parameter in the first object make impact the parameters used by the left hand and the parameters of the object in the left had.) determining, by the at least one processor, an additional state representation based on the additional sensor data; and (Oleynik - [0660] In an embodiment, in order to complete minimanipulations 1-.1-1.3, to yield the functional result, right hand and left hand must sense and receive feedback on the object and the state change of the object in the hand or palm, or leg. …) applying, by the at least one processor, a final percept of the plurality of percepts to the additional state representation to determine whether the respective metric for the final percept is satisfied. (Oleynik – [0660] … This sensed state change may result in an adjustment to the parameters that comprise the minimanipulation. Each change in one parameter may yield in a change to each subsequent parameter and each subsequent required minimanipulation until the desired tasks result is achieved.)
Examiner Note:
Since the specification does not seem have a definitive meaning for the term “Additional”, the examiner is interpreting “Additional” to mean “real-time” (i.e. data handled during robot operation) with respect to Additional sensor data and Additional state representation.
Therefore, prior to the effective filing date of the claimed invention, it would have been
obvious to one of ordinary skill in the art to modify Ijiri and Natarajan to include a method of incorporating real-time sensor data in order to update robot state observations and therefore also update the parameters associated with the motion primitives as taught in Oleynik. Having the ability to update the parameters of the motion primitives while work tasks are in process, provides the robot with greater accuracy when evaluating the results of robot tasks which are a sequence of combined motion primitives.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure or directed to the state of the art is listed on the enclosed PTO-892.
The following is a brief description for relevant prior art that was cited but not applied:
Jha (US 20240131698 A1) describes a robotic controller which generates sequences of movement primitives for sequential tasks of a robot having a manipulator. The controller includes a method for storing a dictionary of movement primitives, a pretrained learning module, and a graph-search based planning module having instructions stored thereon. The controller acquires a planned task provided by an interface device operated by a user, wherein the planned task is represented by an initial state and a goal state with respect to an object.
Bhat (US 20210331316 A1) describes methods and systems for improved control of robotic arms. In one embodiment, a method is presented that includes predefining a plurality of motion primitives, which may include one or more preconditions and effects. A target state for a plurality of workpieces may be determined as well as an initial state of the plurality of workpieces. A sequence of operations may be generated based on the preconditions and/or effects of the motion primitives, as well as the target state and the initial state. Executing the sequence of operations may be capable of changing the plurality of workpieces from the initial state to the target state.
Dong (US 20230136402 A1) describes an industrial robot motion accuracy compensation method includes: establishing a motion parameter database, wherein the motion parameter database includes a plurality of different reference operating conditions and a motion parameter of the industrial robot corresponding to each reference operating condition, and each reference operating condition is formed by combining each element in each set in a total set of operation conditions; acquiring a current operating condition of the industrial robot; determining whether there is a reference operating condition matched with the current operating condition in the motion parameter database.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALAN LINDSAY OSTROW whose te