DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . Claims 1-6 filed on 2/2/2024 have been reviewed and considered by this office action.
Information Disclosure Statement
The information disclosure statements filed on 2/2/2024 and 9/30/2024 have been reviewed and considered by this office action.
Drawings
The drawings filed on 2/2/2024 have been reviewed and are considered acceptable.
Specification
The specification filed on 2/2/2024 has been reviewed and is considered acceptable.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-6 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
In particular, claims 1 and 5-6, each include the limitation of, “performing learning on an operation rule of the controlled object using the second evaluation function, and performing learning on the operation rule of the controlled object using a learning result and the first evaluation function.”. Here, it is unclear what is meant by using a learning result refers to and how it’s utilized. For instance, is the learned result from the learning performed on the second evaluation function? Is it a learning result from the operation rule? Further, is the second learning clause combining the learned result and a first evaluation function result to provide a combined learned result? Or is it simply just performing learning on both the learned result and first evaluation function?
In order to further prosecution, any art the performs learning on multiple evaluation functions will be interpreted to read upon this claim until the claims are further amended and clarified.
Dependent claim 3 recites, “wherein the first evaluation function is set such that an evaluation relating to the operation of the controlled object is decreased when an evaluation relating to the operation of the controlled object is a lower evaluation than a threshold, and
the processor is configured to execute the instructions to generate the second evaluation function from the first evaluation function, such that the threshold is altered so that the evaluation relating to the operation of the controlled object easily becomes a high evaluation that is greater than or equal to the threshold.”
In this limitation, it’s not exactly clear if the threshold is being adjusted or if the second evaluation function is generated such that operation result will be greater than or equal to a threshold. Further, it’s not clear how it “easily” becomes a high evaluation, as there is no way to ascertain what defines “easily” making something achieve a result. In order to further prosecution, any generation of a result that is greater than or equal to a threshold/setpoint that is granted a positive reward and any function that is lower or farther from a threshold/setpoint that is given a negative reward will be interpreted to read upon this limitation until further clarification is provided.
Claim 4 is dependent upon rejected claim 1 and is thus rejected by virtue of dependency.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-6 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Tsuneki et al. (US PGPUB 2020015599).
Regarding Claims 1 and 5-6; Tsuneki teaches; An operation rule determination device comprising: (Tsuneki; at least paragraphs [0006]-[0007]; disclose an output device (i.e. operation rule determination device) for displaying a plurality of evaluation function results that have been applied to a machine learning unit)
a memory configured to store instructions; and (Tsuneki; at least paragraph [0007]; discloses storage unit (206))
a processor configured to execute the instructions to: (Tsuneki; at least paragraph [0007]; discloses control unit (205))
set a second evaluation function that has been altered from a first evaluation function in which a condition relating to operation of a controlled object is reflected, (Tsuneki; at least Fig. 3; paragraphs [0007], [0063], and [0066]; disclose a system and method for utilizing a plurality of evaluation functions based upon operation of a controlled servo device, wherein system can display a plurality of evaluation functions (i.e. weighting factors W1-W3) and wherein the evaluation functions each provide a different alteration to a component of the evaluation function, and further, the system allows for an operator to perform edits and set each different evaluation function)
such that a difference in an evaluation function between time steps of evaluation relating to the operation of the controlled object is reduced; and (Tsuneki; at least Fig. 3; paragraph [0077]; shows three evaluation functions superimposed in a same time step graph, and wherein based upon adjustments to the evaluation functions, and in particular, evaluation function W3, the shock is reduced compared to the evaluation functions W1 and W2)
perform learning on an operation rule of the controlled object using the second evaluation function, and perform learning on the operation rule of the controlled object using a learning result and the first evaluation function. (Tsuneki; at least Fig. 3; paragraphs [0066]-[0067] and [0072]-[0073]; disclose wherein the system and method allows a user to set various parameters to multiple evaluation functions (i.e. weighting factors W1 and W2 in the provided citations) and after providing desired parameters for the functions, sending them to a machine learning device which performs learning each function provided).
Regarding Claim 2; Tsuneki teaches; The operation rule determination device according to claim 1, wherein the first evaluation function is set such that the condition is reflected in a final time step among time steps of a series of operations of the controlled object, and the processor is configured to execute the instructions to generate the second evaluation function from the first evaluation function, such that alternation is performed in which a condition based on the condition of the final time step is reflected in a time step that is different from a final time step among time steps of a series of operations of the controlled object. (Tsuneki; at least Fig. 3; shows a time step chart in zone P3 which superimposes various evaluation functions W1-W3, wherein the responses relate to a servos ability to correct position error by reducing oscillations in positioning, wherein the first function W1 does not resolve the condition until the final time step whereas the second function, W3 in this case, is able to smooth out the position error by the third time step, thus the condition (i.e. position error oscillation suppression) is contained in a time step other than the last).
Regarding Claim 3; Tsuneki teaches; The operation rule determination device according to claim 1, wherein the first evaluation function is set such that an evaluation relating to the operation of the controlled object is decreased when an evaluation relating to the operation of the controlled object is a lower evaluation than a threshold, and (Tsuneki; at least paragraph [0140]; disclose wherein an evaluation function applied to a position step experiencing a position error in a current state, wherein the position error of the state represents the threshold, if the new evaluation function applied results in the position error getting worse (i.e. lower than the threshold), a negative reward value (i.e. decreased) is applied)
the processor is configured to execute the instructions to generate the second evaluation function from the first evaluation function, such that the threshold is altered so that the evaluation relating to the operation of the controlled object easily becomes a high evaluation that is greater than or equal to the threshold. (Tsuneki; at least paragraphs [0138]-[0139]; disclose wherein an evaluation function is generated after experiencing a position error in a servo, and wherein the position error state represents the threshold, and wherein the function is applied resulting in reducing the position error (i.e. greater than the threshold in a positive direction), a positive reward is applied (i.e. high evaluation) thus marking improvement in position for the learning process).
Regarding Claim 4; Tsuneki teaches; The operation rule determination device according to claim 1, wherein the processor is configured to execute the instructions to once again set an operation rule that has been previously set when an evaluation of an operation rule that has been set during learning of the operation rule is lower than a predetermined condition. (Tsuneki; at least paragraph [0126]; disclose wherein the system and method applies reward values to each evaluation function based upon it’s ability to reduce position error, wherein, if a function is applied which results in a decrease in reward, the system will revert to an action of a previous state).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bao (US PGPUB 20200342356): disclose a machine learning system and method for a numerical controller wherein various evaluation functions are applied to various thresholds, wherein machine learning is applied based upon the functions falling below the thresholds.
Ozeki (US PGPUB 20190384253): disclose a system and method for providing adjustments in machine tool positioning, wherein multiple evaluation functions are generated along with corresponding thresholds to be used to judge effectiveness of the respective function.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHRISTOPHER W CARTER whose telephone number is (469)295-9262. The examiner can normally be reached 9-6:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Robert Fennema can be reached at (571) 272-2748. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CHRISTOPHER W CARTER/Examiner, Art Unit 2117