DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of Claims
Claims 1-9 are now pending.
Priority
Acknowledgement is made of applicant’s claim for foreign priority under 35 USC 119 (a)-(d) to application JP2024-056408 filed 03/29/2024. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55. As such, the effective filing date of the application is 03/29/2024.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are:
“Management unit” in claims 1-5.
“Designation unit” in claim 6.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Presented claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1, it recites:
“A robot motion learning device comprising:
a plurality of first learning models that receive motion information at a certain time and convert the motion information into motion features, and also receive external information at the time and convert the external information into external features, for a plurality of types of robots;
a shared learning model that converts the motion features and external features output by the first learning models into predicted motion features at a next time that are common to the plurality of types of robots;
a plurality of second learning models that convert the predicted motion features at the next time into predicted motion information, for the plurality of types of robots;
and a management unit that uses teaching data related to motion of each of the robots to train either the first learning model and the second learning model related to the robot or the shared learning model.”
The limitations as drafted, under their broadest reasonable interpretation, cover performance of the limitations in the human mind. That is, other than reciting that the steps are implemented on “a plurality of first learning models,” nothing in the claim precludes the steps from practically being performed in the mind. For example, but for the “learning models” language, the limitation of “receive motion information at a certain time and convert the motion information into motion features,” in the context of this claim, given the broadest reasonable interpretation, encompasses the act of a user viewing motion information and determining motion features based on that data. Similarly, the limitations of “converts the motion features and external features output by the first learning models into predicted motion features at a next time that are common to the plurality of types of robots…” and “convert the predicted motion features at the next time into predicted motion information, for the plurality of types of robots…” can be performed mentally or through pen and paper calculations. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, claim 1 presents the additional elements of “a plurality of first learning models,” “a plurality of different types of robots,” “a shared learning model,” and “a management unit.” The components are recited at a high-level of generality (i.e., a generic learning model, generic robots and a generic management unit) such that they amount to no more than mere instructions to apply the exception using generic computer components. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional generic computer components to perform the claimed steps amount to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. The claim is not patent eligible.
Dependent claims 2-8 are similarly rejected under 35 U.S.C. 101 because the
claimed invention is directed to an abstract idea without significantly more. The dependent claims have
been given the full two-part analysis including analyzing the additional limitations both individually and
in combination. The dependent claims, when analyzed individually, and in combination, are also held to
be patent ineligible under 35 U.S.C. 101. The additional recited limitations of the dependent claims fail
to establish that the claims do not recite an abstract idea because the additional recited limitations of
the dependent claims merely further narrow the abstract idea. Accordingly, these elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
Regarding claim 9, it recites:
“A robot motion learning method for learning motions of a plurality of types of robots, comprising the steps of:
causing a first learning model corresponding to a robot to learn processing for converting motion information and external information of the robot at a certain time into common motion features using teaching data related to the motion of the robot;
causing a shared learning model to learn a time-series relationship of the common motion features related to motions common to the plurality of types of robots using the teaching data related to the motions of the plurality of types of robots;
and causing a second learning model corresponding to the robot to learn processing for converting predicted values at a next time of the common motion features output by the shared learning model into predicted motion information of the robot at the next time using the teaching data related to the motion of the robot.”
The limitations as drafted, under their broadest reasonable interpretation, cover performance of the limitations in the human mind. That is, other than reciting that the steps are implemented by a “learning model,” nothing in the claim precludes the steps from practically being performed in the mind. For example, but for the “learning model” language, the limitation of “converting motion information and external information of the robot at a certain time into common motion features using teaching data related to the motion of the robot,” in the context of this claim, given the broadest reasonable interpretation, encompasses the act of a user viewing motion information and determining motion features based on that data. Similarly, the limitations of “causing a shared learning model to learn a time-series relationship of the common motion features related to motions common to the plurality of types of robots using the teaching data related to the motions of the plurality of types of robots; and causing a second learning model corresponding to the robot to learn processing for converting predicted values at a next time of the common motion features output by the shared learning model into predicted motion information of the robot at the next time using the teaching data related to the motion of the robot,” can be performed mentally or through pen and paper calculations. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind, it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, claim 9 presents the additional elements of “a plurality of types of robots,” “a first learning model,” and “a shared learning model.” The components are recited at a high-level of generality (i.e., as generic computer components performing a generic computer function) such that it amounts to no more than mere instructions to apply the exception using generic computer components. Accordingly, these additional elements does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional generic computer components to perform the claimed steps amount to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. The claim is not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kolluri et al. (US 20210362330 A1), hereinafter Kolluri.
Regarding claim 1, Kolluri discloses:
A robot motion learning device comprising:
a plurality of first learning models that receive motion information at a certain time and convert the motion information into motion features, and also receive external information at the time and convert the external information into external features, for a plurality of types of robots (see at least [0058]: “A skill template can also include, for each demonstration subtask, the software modules that are required to tune the demonstration subtask using local demonstration data. Each demonstration subtask can rely on a different type of machine learning model and can use different techniques for tuning… Therefore, the tuning procedure for the insertion demonstration subtask can more heavily tune the machine learning models that deal with force perception and corresponding feedback. In other words, even when the underlying models for subtasks in a skill template are the same, each subtask can have its own respective tuning procedures for incorporating local demonstration data in differing ways.”)
a shared learning model that converts the motion features and external features output by the first learning models into predicted motion features at a next time that are common to the plurality of types of robots (see at least [0044]: “In some implementations, the training system generates the base control policy from generalized training data 165. While the local demonstration data 115 collected by the online execution system 110 is typically specific to one particular robot or one particular robot model, the generalized training data 165 can in contrast be generated from one or more other robots, which need not be the same model, located at the same site, or built by the same manufacturer. For example, the generalized training data 165 can be generated offsite from tens or hundreds or thousands of different robots having different characteristics and being different models. In addition, the generalized training data 165 does not even need to be generated from physical robots. For example, the generalized training data can include data generated from simulations of physical robots.”)
a plurality of second learning models that convert the predicted motion features at the next time into predicted motion information, for the plurality of types of robots (see at least [0047]: “Adapting a base control policy using local demonstration data has the highly desirable effect that it is relatively fast compared to generating the base control policy, e.g., either by collecting system demonstration data or by training using the generalized training data 165. For example, the size of the generalized training data 165 for a particular task tends to be orders of magnitude larger than the local demonstration data 115 and thus training the base control policy is expected to take much longer than adapting it for a particular robot. For example, training the base control policy can require vast computing resources, in some instances, a datacenter having hundreds or thousands of machines working for days or weeks to train the base control policy from generalized training data. In contrast, adapting the base control policy using local demonstration data 115 can take just a few hours.”)
and a management unit that uses teaching data related to motion of each of the robots to train either the first learning model and the second learning model related to the robot or the shared learning model (see at least Figure 1, item 110, the Online Execution System.)
Regarding claim 2, Kolluri discloses:
The robot motion learning device according to claim 1, wherein the management unit uses teaching data related to motion unlearned by each of the robots, to train the shared learning model while keeping parameters of the first learning model and the second learning model fixed (See at least [0046]: “The base control policy can also be defined using system demonstration data that is collected during the process of developing the skill template. For example, a team of engineers associated with the entity that generates skill templates can perform demonstrations using one or more robots at a facility that is remote from and/or unassociated with the system 100. The robots used to generate the system demonstration data also need not be the same robots or the same robot models as the robots 170a-n in the workcell 170. In this case, the system demonstration data can be used to bootstrap the actions of the base control policy. The base control policy can then be adapted into a customized control policy using more computationally expensive and sophisticated learning methods.”)
Regarding claim 3, Kolluri discloses:
The robot motion learning device according to claim 1, wherein the management unit newly adds the first learning model and the second learning model corresponding to a robot to be newly added as an object to be controlled (see at least [0044]: “In some implementations, the training system generates the base control policy from generalized training data 165. While the local demonstration data 115 collected by the online execution system 110 is typically specific to one particular robot or one particular robot model, the generalized training data 165 can in contrast be generated from one or more other robots, which need not be the same model, located at the same site, or built by the same manufacturer. For example, the generalized training data 165 can be generated offsite from tens or hundreds or thousands of different robots having different characteristics and being different models. In addition, the generalized training data 165 does not even need to be generated from physical robots. For example, the generalized training data can include data generated from simulations of physical robots.”)
Regarding claim 4, Kolluri discloses:
The robot motion learning device according to claim 1, wherein the management unit uses teaching data for the robot to be added, related to motion already learned by the shared learning model, to train the first learning model and the second learning model while keeping parameters of the shared learning model fixed (see at least [0044-0045]: “In some implementations, the training system generates the base control policy from generalized training data 165. While the local demonstration data 115 collected by the online execution system 110 is typically specific to one particular robot or one particular robot model, the generalized training data 165 can in contrast be generated from one or more other robots, which need not be the same model, located at the same site, or built by the same manufacturer. For example, the generalized training data 165 can be generated offsite from tens or hundreds or thousands of different robots having different characteristics and being different models. In addition, the generalized training data 165 does not even need to be generated from physical robots. For example, the generalized training data can include data generated from simulations of physical robots.
Thus, the local demonstration data 115 is local in the sense that it is specific to a particular robot that a user can access and manipulate. The local demonstration data 115 thus represents data that is specific to a particular robot, but can also represent local variables, e.g., specific characteristics of the particular task as well as specific characteristics of the particular working environment.”)
Regarding claim 5, Kolluri discloses:
The robot motion learning device according to claim 1, wherein upon newly obtaining teaching data for a robot, related to a shared motion already learned by the shared learning model, the management unit uses the teaching data to perform either first training in which to only the first learning model and the second learning model corresponding to the robot are trained or second training in which only the shared learning model corresponding to the shared motion is trained, or alternate between the first training and the second training (see at least [0046]: “The base control policy can also be defined using system demonstration data that is collected during the process of developing the skill template. For example, a team of engineers associated with the entity that generates skill templates can perform demonstrations using one or more robots at a facility that is remote from and/or unassociated with the system 100. The robots used to generate the system demonstration data also need not be the same robots or the same robot models as the robots 170a-n in the workcell 170. In this case, the system demonstration data can be used to bootstrap the actions of the base control policy. The base control policy can then be adapted into a customized control policy using more computationally expensive and sophisticated learning methods.”)
Regarding claim 6, Kolluri discloses:
The robot motion learning device according to claim 5, wherein the shared learning model is provided for each of a plurality of shared motions, and the robot motion learning device further comprises a motion designation unit that selects and executes one of the shared learning models provided for each of the plurality of shared motions (see at least [0050]: “In execution mode, an execution engine 130 can use the customized control policy 125 to automatically perform the task without any user intervention. The online execution system 110 can use the customized control policy 125 to generate commands 155 to be provided to the robot interface subsystem 160, which drives one or more robots, e.g., robots 170a-n, in a workcell 170. The online execution system 110 can consume status messages 135 generated by the robots 170a-n and online observations 145 made by one or more sensors 171a-n making observations within the workcell 170. As illustrated in FIG. 1, each sensor 171 is coupled to a respective robot 170. However, the sensors need not have a one-to-one correspondence with robots and need not be coupled to the robots. In fact, each robot can have multiple sensors, and the sensors can be mounted on stationary or movable surfaces in the workcell 170.”)
Regarding claim 7, Kolluri discloses:
The robot motion learning device according to claim 1, wherein the shared learning model converts the external features output by the first learning models into predicted external features at the next time that are common to the plurality of robots (see at least [0046]: “The base control policy can also be defined using system demonstration data that is collected during the process of developing the skill template. For example, a team of engineers associated with the entity that generates skill templates can perform demonstrations using one or more robots at a facility that is remote from and/or unassociated with the system 100. The robots used to generate the system demonstration data also need not be the same robots or the same robot models as the robots 170a-n in the workcell 170. In this case, the system demonstration data can be used to bootstrap the actions of the base control policy. The base control policy can then be adapted into a customized control policy using more computationally expensive and sophisticated learning methods.”)
Regarding claim 8, Kolluri discloses:
A robot motion learning system comprising: the robot motion learning device according to claim 1; and a plurality of types of robots (see at least [0046]: “The base control policy can also be defined using system demonstration data that is collected during the process of developing the skill template. For example, a team of engineers associated with the entity that generates skill templates can perform demonstrations using one or more robots at a facility that is remote from and/or unassociated with the system 100. The robots used to generate the system demonstration data also need not be the same robots or the same robot models as the robots 170a-n in the workcell 170. In this case, the system demonstration data can be used to bootstrap the actions of the base control policy. The base control policy can then be adapted into a customized control policy using more computationally expensive and sophisticated learning methods.”)
Regarding claim 9, Kolluri discloses:
A robot motion learning method for learning motions of a plurality of types of robots, comprising the steps of:
causing a first learning model corresponding to a robot to learn processing for converting motion information and external information of the robot at a certain time into common motion features using teaching data related to the motion of the robot (see at least [0058]: “A skill template can also include, for each demonstration subtask, the software modules that are required to tune the demonstration subtask using local demonstration data. Each demonstration subtask can rely on a different type of machine learning model and can use different techniques for tuning… Therefore, the tuning procedure for the insertion demonstration subtask can more heavily tune the machine learning models that deal with force perception and corresponding feedback. In other words, even when the underlying models for subtasks in a skill template are the same, each subtask can have its own respective tuning procedures for incorporating local demonstration data in differing ways.”)
causing a shared learning model to learn a time-series relationship of the common motion features related to motions common to the plurality of types of robots using the teaching data related to the motions of the plurality of types of robots (see at least [0044]: “In some implementations, the training system generates the base control policy from generalized training data 165. While the local demonstration data 115 collected by the online execution system 110 is typically specific to one particular robot or one particular robot model, the generalized training data 165 can in contrast be generated from one or more other robots, which need not be the same model, located at the same site, or built by the same manufacturer. For example, the generalized training data 165 can be generated offsite from tens or hundreds or thousands of different robots having different characteristics and being different models. In addition, the generalized training data 165 does not even need to be generated from physical robots. For example, the generalized training data can include data generated from simulations of physical robots.”)
and causing a second learning model corresponding to the robot to learn processing for converting predicted values at a next time of the common motion features output by the shared learning model into predicted motion information of the robot at the next time using the teaching data related to the motion of the robot (see at least [0047]: “Adapting a base control policy using local demonstration data has the highly desirable effect that it is relatively fast compared to generating the base control policy, e.g., either by collecting system demonstration data or by training using the generalized training data 165. For example, the size of the generalized training data 165 for a particular task tends to be orders of magnitude larger than the local demonstration data 115 and thus training the base control policy is expected to take much longer than adapting it for a particular robot. For example, training the base control policy can require vast computing resources, in some instances, a datacenter having hundreds or thousands of machines working for days or weeks to train the base control policy from generalized training data. In contrast, adapting the base control policy using local demonstration data 115 can take just a few hours.”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
“Learning and Generalization of Dynamic Movement Primitives by Hierarchical Deep Reinforcement Learning from Demonstration” by Kim et al.
“Collective Robot Reinforcement Learning with Distributed Asynchronous Guided Policy Search” by Yahya et al.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ELIZABETH NELESKI whose telephone number is (571)272-6064. The examiner can normally be reached 10 - 6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THOMAS WORDEN can be reached at (571) 272-4876. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.R.N./Examiner, Art Unit 3658 /JASON HOLLOWAY/ Primary Examiner, Art Unit 3658