DETAILED ACTION
Claims 1-7 are pending; this office action supersedes the previous office action and reset the response time period.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for priority of U.S provisional application under 35 U.S.C. 119(e).
Information Disclosure Statement
The information disclosure statements provided complies with the provisions of MPEP § 609. It has been placed in the application file, and the information referred to therein has been considered as to the merits. A signed copy of the form is attached.
Specification
The objection to the abstract regarding the word “means”, is withdrawn.
The objection to the Content of Specification that is the title is withdrawn.
The rejection to claims 1-7 are rejected under 35 U.S.C 101 is withdrawn.
The rejection to claims 1-7 are rejected under 35 U.S.C. 103 as being unpatentable over Savarimuthu et al., (Teaching a Robot the Semantics of Assembly Tasks, IEEE) is withdrawn.
Applicant’s arguments with respect to claim(s) 1-7 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-7 are rejected under 35 U.S.C. 103 as being unpatentable over Savarimuthu et al., (Teaching a Robot the Semantics of Assembly Tasks, IEEE) in view of Funimoto et al. (US 20220056953).
As per claim 1, Savarimuthu et al., teaches essential feature of the invention substantially a planner apparatus (see abs., see page 670, second col. 2) comprising: at least one memory configured to store instructions (see item I., the programing has shown clear evidence of a memory for storing); and at least one processor configured to execute the instructions to acquire a state of a control target (see Figs. 1 and 3, page 373, section II, section A) at a first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time); and an action decision means (see section VI. first par.) configured to decide on an action at a second time that is a control timing subsequent to the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) such that a value calculated when the state has been input is trained such that a value related to a sum of rewards based on states of the control target (see Figs. 1 and 3, page 373, section II, section A) at control timings between the second time and a third time subsequent to the second time is calculated when a process of deciding on an action (see Fig. 16) between the second time and the third time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) from the state of the control target (see Figs. 1 and 3, page 373, section II, section A) at the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and the action at the second time has been iterated (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time). Savarimuthu et al. does not specifically teach a pre-trained value function is largest, wherein the value function.
Fujimoto et al. teaches a pre-trained value function is largest, wherein the value function (see par. [0171]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Savarimuthu et al. with the value function as taught by Fujimoto et al., this combination would have provided “values of the control-target device 50 and by using a technique of supervised learning or reinforcement learning” (see par. [0094]), thereby improving the planning programming recording as a whole.
As per claim 2, Savarimuthu et al., essential feature of the invention substantially at least one processor is configured to execute the instructions to decide on the action (see page 674, col. 2, item C.) on the basis of trajectory data including a time series (see Fig. 1B) of the state until the first time is reached (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) value is calculated from the trajectory data (see Fig. 16) and the action at the second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time). Savarimuthu et al. does not specifically teach value function, and wherein the value function is trained.
Fujimoto et al. teaches value function, and wherein the value function is trained (see par. [0071]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Savarimuthu et al. with the value function as taught by Fujimoto et al., this combination would have provided “values of the control-target device 50 and by using a technique of supervised learning or reinforcement learning” (see par. [0094]), thereby improving the planning programming recording as a whole.
As per claim 3, Savarimuthu et al., teaches wherein the trajectory data includes a time series of combinations of states and actions of the control target and rewards (see Figs. 1 and 3, page 373, section II, section A).
As per claim 4, Savarimuthu et al., teaches wherein, iteratively inputting an action to a prediction function of predicting (see Fig. 16) a state of the control target (see Figs. 1 and 3, page 373, section II, section A) and a reward at a subsequent control timing from a state of the control target (see Figs. 1 and 3, page 373, section II, section A) at a reference time and an action at the control timing subsequent to the reference time and obtaining the rewards between the second time and the third time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time). Savarimuthu et al. does not specifically teach a value function training process, the value is calculated.
Fujimoto et al. teaches a value function training process; the value is calculated (see par. [0071]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Savarimuthu et al. with the value function as taught by Fujimoto et al., this combination would have provided “values of the control-target device 50 and by using a technique of supervised learning or reinforcement learning” (see par. [0094]), thereby improving the planning programming recording as a whole.
As per claim 5, Savarimuthu et al., teaches wherein the prediction function is a trained model trained, by using previous state and a previous action of the control target (see Figs. 1 and 3, page 373, section II, section A) as a learning dataset, to output a state at the second time by inputting (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) the state of the control target (see Figs. 1 and 3, page 373, section II, section A) at the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and the action at the second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time).
As per claim 6, Savarimuthu et al., teaches a planning method comprising: acquiring a state of a control target (see Figs. 1 and 3, page 373, section II, section A) at a first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time); and deciding on an action at a second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) that is a control timing subsequent to the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) such that a value calculated when the state has been input based on states of the control target (see Figs. 1 and 3, page 373, section II, section A) at control timings between the second time and a third time subsequent to the second time is calculated when a process of deciding on an action between the second time and the third time (see Fig. 16 and page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) from the state of the control target (see Figs. 1 and 3, page 373, section II, section A) at the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and the action at the second time has been iterated (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time). Savarimuthu et al. does not specifically teach a pre-trained value function is largest, wherein the value function is trained such that a value related to a sum of rewards.
Fujimoto et al. teaches a pre-trained value function is largest, wherein the value function is trained such that a value related to a sum of rewards (see par. [0071]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Savarimuthu et al. with the value function as taught by Fujimoto et al., this combination would have provided “values of the control-target device 50 and by using a technique of supervised learning or reinforcement learning” (see par. [0094]), thereby improving the planning programming recording as a whole.
As per claim 7, Savarimuthu et al., teaches a non-trasitory computer-readable recording medium storing a planning program (see page 670, item I.) for allowing a computer to: acquire a state of a control target (see Figs. 1 and 3, page 373, section II, section A) at a first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time); and 25 decide on an action at a second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) that is a control timing subsequent to the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) such that a value calculated when the state has been input (see Figs. 1 and 3, page 373, section II, section A) at control timings between the second time and a third time subsequent to the second time is calculated when a process of deciding on an action between the second time and the third time (see Fig. 16 and page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) from the state of the control target (see Figs. 1 and 3, page 373, section II, section A) at the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and the action at the second time has been iterated (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time). Savarimuthu et al. does not specifically teach a pre-trained value function is largest (see Fig. 16), wherein the value function is trained such that a value related to a sum of rewards based on states of the control target.
Fujimoto et al. teaches a pre-trained value function is largest (see Fig. 16), wherein the value function is trained such that a value related to a sum of rewards based on states of the control target (see par. [0071]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Savarimuthu et al. with the value function as taught by Fujimoto et al., this combination would have provided “values of the control-target device 50 and by using a technique of supervised learning or reinforcement learning” (see par. [0094]), thereby improving the planning programming recording as a whole.
As per claim 8, Savarimuthu et al., teaches a learning device comprising: a prediction means configured to predict a state of a control target (see Figs. 1 and 3, page 373, section II, section A) at a second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) from a state of the control target (see Figs. 1 and 3, page 373, section II, section A) at a first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and an action at the second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) that is a control timing subsequent to the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time); a reward calculation means configured to calculate a sum of rewards (see Fig. 16) based on states of the control target (see Figs. 1 and 3, page 373, section II, section A) at control timings between the second time and a third time subsequent to the second time obtained by iteratively inputting an action after the second time to the prediction means as a value (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time); and an update means configured to update a parameter of (see Figs. 1 and 3, page 373, section II, section A) at the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and the action at the second time as inputs on the basis of the state, the action, and the value (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time). Savarimuthu et al. does not specifically teach a value function such that the value is output by designating the state of the control target.
Fujimoto et al. teaches a value function such that the value is output by designating the state of the control target (see par. [0071]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Savarimuthu et al. with the value function as taught by Fujimoto et al., this combination would have provided “values of the control-target device 50 and by using a technique of supervised learning or reinforcement learning” (see par. [0094]), thereby improving the planning programming recording as a whole.
As per claim 9, Savarimuthu et al., teaches a learning method comprising: calculating a sum of rewards (see Fig. 16) based on states of a control target (see Figs. 1 and 3, page 373, section II, section A) at control timings between a second time and a third time subsequent to the second time obtained by iteratively inputting an action after the second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) to a prediction function for predicting a state of the control target (see Figs. 1 and 3, page 373, section II, section A) at the second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) from a state of the control target (see Figs. 1 and 3, page 373, section II, section A) at a first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and an action at the second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) that is a control timing subsequent to the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) as a value; and updating a parameter (see Figs. 1 and 3, page 373, section II, section A) at the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and the action at the second time as inputs on the basis of the state, the action, and the value (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time). Savarimuthu et al. does not specifically teach a value function such that the value is output by designating the state of the control target.
Fujimoto et al. teaches a value function such that the value is output by designating the state of the control target (see par. [0071]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Savarimuthu et al. with the value function as taught by Fujimoto et al., this combination would have provided “values of the control-target device 50 and by using a technique of supervised learning or reinforcement learning” (see par. [0094]), thereby improving the planning programming recording as a whole.
As per claim 10, Savarimuthu et al., teaches a recording medium storing a learning program for allowing a computer to: calculate a sum of rewards (see Fig. 16) based on states of a control target (see Figs. 1 and 3, page 373, section II, section A) at control timings between a second time and a third time subsequent to the second time obtained by iteratively inputting an action after the second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) to a prediction function for predicting a state of the control target (see Figs. 1 and 3, page 373, section II, section A) at the second time from a state of the control target (see Figs. 1 and 3, page 373, section II, section A) at a first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and an action at the second time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) that is a control timing subsequent to the first time as a value (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time); and update a parameter (see Figs. 1 and 3, page 373, section II, section A) at the first time (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time) and the action at the second time as inputs on the basis of the state, the action, and the value (see page 673, second col. fourth par. “execution time the system ...sequencing of actions” meet first, second and Nth, time). Savarimuthu et al. does not specifically teach of a value function such that the value is output by designating the state of the control target.
Fujimoto et al. teaches a value function such that the value is output by designating the state of the control target (see par. [0071]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Savarimuthu et al. with the value function as taught by Fujimoto et al., this combination would have provided “values of the control-target device 50 and by using a technique of supervised learning or reinforcement learning” (see par. [0094]), thereby improving the planning programming recording as a whole.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MCDIEUNEL MARC whose telephone number is (571) 272-6964. The examiner can normally be reached on 9:00 AM - 5:00 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, WADE MILES can be reached on (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is (571)-273-3976.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
PNG
media_image1.png
275
275
media_image1.png
Greyscale
/McDieunel Marc/
Primary Examiner, Art Unit 3665
Prosecution Insights