DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The Information Disclosure Statements filed 23 August 2024, and 07 April 2025 have been fully considered by the examiner. Signed copies are attached
Acknowledgement is made of the preliminary amendment to the specification and claims filed on 23 August 2024, and the application is being examined on the basis of the amended disclosure.
Claims 1-6 and 8 are pending.
Claims 1-6 and 8 are rejected, grounds follow.
Priority
Application’s status as a 35 USC 371 national stage application of PCT application PCT/JP2022/008700 is acknowledged.
Claim Interpretation
Following the preliminary amendment to the claims cancelling the “means” verbiage; all claim limitations in the presently pending claims have been interpreted under the “Broadest Reasonable Interpretation” standard of claim construction (see MPEP 2111).
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-6 and 8 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, because the specification, while being enabling for control targets which comprise articulated robots, grasping end effectors, and the like, the disclosure does not appear to reasonably provide adequate guidance to one of ordinary skill to make and use the invention (enablement) for machine learning related to control targets which use fundamentally different operational modes, such as (non-limiting examples) Power production facilities, HVAC equipment, or unmanned autonomous-vehicles. The specification does not enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to practice the invention commensurate in scope with these claims. In particular the claims recite the extremely broad and very generic “learning device”, “control device”, “control target”, and “control of the control target”, which under broadest reasonable interpretation appear to cover any and all control plants. However the disclosure only provides guidance regarding articulated robots, particularly those with grasping end effectors (see e.g. fig. 2). Correspondingly the Disclosure does not provide any direction for making and using the claims with control targets other than said articulated robots; and there are no working examples other than articulated robots discussed in the overall disclosure.
Considering these factors together (See MPEP 2164.01(a)), Examiner concludes that the scope of enablement provided by the disclosure is exceeded by the claims and accordingly the claims are rejected under 35 USC 112(a) for lacking enablement of the full scope of the claimed invention.
In the interest of compact prosecution, examiner notes that limitations which narrow the scope of the independent claims by reciting e.g. “the control target comprising an articulated robot” or “the control target comprising a grasping end effector” (claim 1 representative) or similar language which comports with the examples and modes set forth in the original disclosure would overcome this rejection.
Claims 2 and 3 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. The meaning of the verb “set” in Claim 2 line 8 “set a prediction accuracy evaluation function…” is not clear. In particular it is not clear how applicant intends the use of “set” in this instance to differ in scope from the more conventional terminology such as e.g., causing the processor to execute instructions “defining” or “evaluating” or “calculating” an evaluation function; correspondingly, one of ordinary skill in the art would not be apprised of the precise scope of the claim limitation.
Where applicant acts as his or her own lexicographer to specifically define a term of a claim contrary to its ordinary meaning, the written description must clearly redefine the claim term and set forth the uncommon definition so as to put one reasonably skilled in the art on notice that the applicant intended to so redefine that claim term. Process Control Corp. v. HydReclaim Corp., 190 F.3d 1350, 1357, 52 USPQ2d 1029, 1033 (Fed. Cir. 1999). Which does not appear to be the case here, as following a review of the specification examiner was not able to find a clear redefinition for “set” and accordingly the claimed limitation is indefinite.
Claim 3 inherits the deficiencies of the parent claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4-6, and 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Haddadin et al., US Pg-Pub 2020/0086480 in view of Walters et al., US Pg-Pub 2021/0264263.
Regarding Claim 1, Haddadin teaches:
A learning device ([0056] “The learning unit applies meta learning, which in particular means finding the right (optimal) parameters p*∈P.sub.l for solving a given task.”) comprising: a memory configured to store instructions; and a processor configured to execute the instructions ([0111] “a computer system with a data processing unit, wherein the data processing unit is designed and set up to carry out a method according to one of the preceding alternatives.”) to: select, ([0156] “A skill s is an element of the skill-space. It is defined as a tuple (S, O, C.sub.pre, C.sub.err, C.sub.suc, R, χ.sub.cmd, X, P, Q).”) from among search points ([0157] “Definition 2 (Space): Let S be the Cartesian product of I subspaces ζ.sub.i=R.sup.m.sup.i.sup.×n.sup.i relevant to the skill s, i.e.: S=ζ.sub.i=1×ζ.sub.i=2× . . . ×ζ.sub.i=I with i={1, 2, . . . , I} and I≥2, wherein the subspaces ζ.sub.i include a control variable and an external wrench including an external force and an external moment.”) indicating an operation (see fig. 1 [0152] e.g. “grasping”) of a control target, (see fig. 1, [0152] e.g. articulated robot 80) a search point (i.e. the tuple defining the skill in the control space) to be subjected to training data acquisition for learning of a control of the control target; ([0155] “The skill provides desired commands and trajectories to the adaptive controller 104 together with meta parameters and other relevant quantities for executing the task. In addition, a skill contains a quality metric and parameter domain to the learning algorithm of the learning unit 103, while receiving the learned set of parameters used in execution.”)
calculate information indicating an evaluation of whether or not an operation indicated by the selected search point is executable, ([0046] Definition 8 (Error Condition): C.sub.err denotes the chosen set for which the error condition c.sub.err(X(t)) holds, i.e., c.sub.err(X(t))=1. This follows from ∃x∈X: x(t)∈C.sub.err. If the error condition is fulfilled at time t, skill execution is interrupted.”) and an output value for the operation indicated by the selected search point to be output by a controller for controlling the control target; ([0204] “wherein the adaptive controller 104 receives skill commands χ.sub.cmd, wherein the skill commands χ.sub.cmd include the skill parameters P.sub.l, wherein based on the skill commands χ.sub.cmd the controller 104 controls the actuators of the robot 80, wherein the actual status X(t) of the robot 80 is sensed by respective sensors and/or estimated by respective estimators and fed back to the controller 104”)
acquire based on the selected search point, the information indicating the evaluation of whether or not the operation indicated by the selected search point is executable the output value for the operation indicated by the selected search point to be output by the controller, (ibid) training data for learning a control of the control target that is performed by the controller; ([0203] “wherein based on the actual status X(t), the second unit 102 determines the performance Q(t) of the skill carried out by the robot 80, and wherein the learning unit 103 receives P.sub.D, and Q(t) from the second unit 102, determines updated skill parameters P.sub.l(t) and provides P.sub.l(t) to the second unit 102 to replace hitherto existing skill parameters P.sub.l”)
Haddadin, differs from the claimed invention in that:
Haddadin does not appear to clearly articulate: determine, based on an evaluation of an acquisition status of the training data, whether or not to continue acquiring the training data.
However, Walters in the related field of generative intelligence models teaches determining based on a predictive evaluation of the state of training data ([0059] “Following completion of such a training mode, the one or more prediction models may then be operated in a prediction mode during which the one or more prediction models may be used to make… a prediction”) whether or not a further round of training is likely to result in further improvements (ibid. “a prediction of whether [new parameters] will likely be found through testing to improve the tuning… so as to come closer to achieving a threshold specified in the evaluation criteria such that it may be deemed efficacious to proceed with using the time, as well as processing and/or storage resources to perform such testing”)
Walters is analogous art because it is from the related field of endeavor of training generative models which is related to machine learning for control outputs because they rely on overlapping techniques, such as reinforcement learning and error descent gradients, etc.
One of ordinary skill in the art could have modified the teachings of Haddadin to include evaluating whether further training is likely to result in further improvements, and if not, to terminate the training.
One of ordinary skill in the art could have been motivated to make this modification in order to reduce instances in which resources are expended without resulting in further improvements, as suggested by Walters ([0059] “Such use of the one or more prediction models seeks to at least reduce the number of instances in which such resources are expended on testing sets of [parameters] that are deemed unlikely to lead to any improvement”)
Regarding Claims 6 and 8, these claims recite substantively the same subject matter as claim 1 above; except including a controller obtained by training on the training data; and embodied as a method, respectively; Mutatis mutandis, these claims are likewise obvious over Haddadin in view of Walters for the same reasons articulated with respect to claim 1.
Regarding Claim 4, Haddadin in view of Walters teaches all of the limitations of parent claim 1,
Haddadin further teaches:
wherein the search points include a parameter value of a skill in which an operation of the control target has been modularized. ([0019] “[0019] P: skill parameters, with P consisting of three subsets P.sub.t, P.sub.l, P.sub.D, with P.sub.t being the parameters resulting from a priori knowledge of the task, P.sub.l being the parameters not known initially which need to be learned and/or estimated during execution of the task, and P.sub.D being constraints of parameters P.sub.l”)
Regarding Claim 5, Haddadin in view of Walters teaches all of the limitations of parent claim 4,
Haddadin further teaches:
wherein the search points ([0037] “A skill s is an element of the skill-space. It is defined as a tuple (S, O, C.sub.pre, C.sub.err, R, χ.sub.cmd, X, P, Q). are configured by a combination of: an initial state of the control target ([0040] “O denotes the set of physical objects o∈O relevant to a skill s with n.sub.o=|O| and n.sub.o>0. Moreover, X(t) is defined as X(t)=(.sup.o.sup.nX(t), . . . , .sup.o.sup.nX(t)).sup.T. Note that in these considerations the set O is not changed during skill execution, i.e., n.sub.o=const.”) and an operation environment of the control target when a skill is started; ([0045] “C.sub.pre denotes the chosen set for which the precondition defined by c.sub.pre(X(t)) holds. The condition holds, i.e., c.sub.pre(X(t.sub.0))=1, iff ∀x∈X: x(t.sub.0)∈C.sub.pre. to denotes the time at start of the skill execution.” )
a parameter value of the skill; ([0042] “P denotes the set of all skill parameters consisting of three subsets P.sub.t, P.sub.l and P.sub.D. The set P.sub.t⊂P contains all parameters resulting from a priori task knowledge, experience and the intention under which the skill is executed.”)
and a target state of the control target and the operation environment of the control target when the skill is completed. ([0047] “C.sub.suc denotes the chosen set for which the success condition defined by c.sub.suc(X(t)) holds, i.e., c.sub.suc(X(t))=1 iff ∀x∈X: x(t)∈C.sub.suc. If the coordinates of all involved objects are within C.sub.suc the skill execution can terminate successfully. With this it is not stated that the skill has to terminate.” [0048] “The nominal result R∈S is the ideal endpoint of skill execution, i.e., the convergence point. Although the nominal result R is the ideal goal of the skill, its execution is nonetheless considered successful if the success conditions C.sub.suc hold. Nonetheless X(t) converges to this point.”
Allowable Subject Matter
The following is a statement of reasons for the indication of allowable subject matter: While Haddadin and Walters teach many of the limitations of the claimed invention as set forth above; and Okawa et al., US Pg-Pub 2021/0063974 teaches a reinforcement learning system for an articulated robot which parameterizes constraint conditions in a 2 dimensional search space (analogous to a level set function); none of the prior art references of record, alone or in reasonable combination, teach or fairly suggest all of the limitations of the claimed invention, particularly:
(Claim 2)
a prediction accuracy evaluation function that receives an input of a search point and outputs an evaluation value of an estimated accuracy of the level set function for the search point, and wherein the processor is configured to execute the instructions to determine whether or not to continue acquiring the training data, based on the prediction accuracy evaluation function.
(Excerpted)
… in combination with the remaining limitations and features of the claimed invention, the base claim, and any intervening claim(s).
Claim 3, being further dependent on dependent claim 2, is likewise persuasive over the prior art of record for at least the above noted reason(s).
Claims 2-3 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(a)/(b) or 35 U.S.C. 112 (pre-AIA ), 1st/2nd paragraphs, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Levine et al., US Pg-Pub 2019/0232488 which teaches a reinforcement learning method for an articulated robot manipulator which includes training on replay data from previous operational use and minimization of future error by predicting success based on the previous operational history.
Huh, Subin, and Insoon Yang. "Safe reinforcement learning for probabilistic reachability and safety specifications: A Lyapunov-based approach." arXiv preprint arXiv:2002.10126 (2020). – teaching the use of Lyapunov functions in reinforcement learning to predict reachability of a given control goal state from a provided initial state.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSHUA T SANDERS whose telephone number is (571)272-5591. The examiner can normally be reached Generally Monday through Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mohammad Ali can be reached at 571-272-4105. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.S./Examiner, Art Unit 2119
/MOHAMMAD ALI/Supervisory Patent Examiner, Art Unit 2119
Prosecution Insights