Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. JP2022-099380, filed on 06/21/2022.
Response to Arguments
Applicant’s argument filed 03/25/2026 have been fully considered but they are not persuasive. The amendments have overcome the interpretation under 35 U.S.C. § 112(f) and 112(b) rejection. Thus, 112(f) and 112(b) have been removed.
Applicant’s Argument: On page 11-13 of Applicant’s response to rejections under 35 U.S.C. 101, applicant states that the amended claims does not recite an abstract idea of a mental process. The claims provide a particular solution for managing and correcting the behavior of cooperating machine learning models.
Examiner’s Response: Applicant’s argument is not persuasive. During examination, the examiner should analyze the "improvements" consideration by evaluating the specification and the claims to ensure that a technical explanation of the asserted improvement is present in the specification, and that the claim reflects the asserted improvement (see MPEP §2106.05(a)). The MPEP (§2106.05(a)(II)) also warns, “it is important to keep in mind that an improvement in the abstract idea itself (e.g. a recited fundamental economic concept) is not an improvement in technology.” Here, the alleged improvement in the form of “determining which model is responsible when abnormal behavior occurs and selectively retraining that model” is an improvement to the abstract idea of a mental process that can be performed in the human mind.
An important consideration in determining whether a claim improves technology is the extent to which the claim covers a particular solution to a problem or a particular way to achieve a desired outcome, as opposed to merely claiming the idea of a solution or outcome (see MPEP 2106.05(a)). The amended claims do not provide sufficient details to describe any technological improvement. If the specifications explicitly set forth an improvement but in a conclusory manner (see MPEP 2106.04(d)(1): a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology.
Applicant’s Argument: On page 13-14 of Applicant’s response to rejections under 35 U.S.C. 103, applicant states that the recited references do not disclose estimating which of the evaluation model or operation model is a main cause of abnormality. Lee discloses selecting among multiple prediction models and is silence to describing two distinct types of models; an evaluation model and an operation model. Further, The Q-values taught in Lee does not teach abnormality.
Examiner’s Response: Applicant’s argument is not persuasive. Claim 1 recites “an operation model is used to control a control target provided in the facility” and the Specification (par. 24) discloses that the term control may be broadly interpreted to include an indirect control of the operation terminal. The term “control target” is not explicitly defined in claim 1. Under the broadest reasonable interpretation, the operation model could be a model that provides a status of a mechanical system and the status is a form of indirect control because a user can provide inputs to the mechanical system based on the provided status.
Lee (par. 8) discloses a plurality of prediction models that generates a prediction based on one or more features that is associated with a degradation status of the mechanical system. Lee (par. 68) further discloses generating a confidence value of the prediction and the confidence value provides valuable information for maintenance practitioners to take an action on the mechanical system based on the degradation status. Thus, the prediction model constitutes as an operation model because it provides indirect control to the physical system. Each of the prediction model also constitutes as an evaluation model because the prediction model in Lee (par. 68) calculates a predicted performance index based on the input features. Lee (par. 19) discloses the input features are measurement data from the sensors of the mechanical system. Therefore, under the broadest reasonable interpretation, each of the plurality of prediction models constitutes as both the evaluation and operation model.
Claim 1 does not explicitly disclose what constitutes as abnormality. Under the broadest reasonable interpretation, abnormality is defined as a condition that deviates from an expected standard. Therefore, a model with low accuracy constitutes as abnormality because the performance of the model is significantly different from the expected performance.
The claims as recited discloses an evaluation model and an operation model that processes information associated with a facility. The abnormality index indicates the performance of the models and based on the abnormality index, one of the two models is selected for finetuning.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-18, and 20 rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claims contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventors, at the time the application was filed, had possession of the claimed invention. The following claims recite subject matter which is not described in the specification:
“abnormality index acquisition unit that uses the at least one processor” in claims 1 and 20
“estimation unit that uses the at least one processor” in claims 1, 2, 4, and 20
“output unit that uses the at least one processor” in claims 1, 3, 5, 6, and 20
“state data acquisition unit that uses the at least one processor” in claims 7, 8, 9, and 10
“abnormality detection unit that uses the at least one processor” in claim 11
“control unit that uses the at least one processor” in claim 12
“operation model generation unit that uses the at least one processor” in claims 13, 14, and 15
“evaluation model generation unit that uses the at least one processor” in claims 16, 17, and 18
The Specification fails to disclose any of the recited units uses a processor to perform the function. The Specification (par. 116 and 119) discloses that certain steps and sections may be implemented by providing a processor with the computer-readable instructions. The Specification does not explicitly provide support for any of the recited units uses the at least one processor to perform the function in the claims. Figure 10 shows the plurality of units and there is no depiction of any processor to support the recited claim limitations.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Subject Matter Eligibility Analysis Step 1:
Claim 1 recites “An estimation apparatus comprising” and is thus a machine, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind, i.e. judgement)
Claim 1 therefore recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
“at least one processor” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
"at least one processorin a facility, among pieces of the state data indicating a state of the facility ” (This step is directed to data gathering, which is understood to be insignificant extra solution activity - see MPEP 2106.05(g))
“an abnormality index acquisition unit that uses the at least one processor ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“an estimation unit that uses the at least one processor to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“an output unit that uses the at least one processor to execute an output in accordance with a result of the estimate” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is directed to the abstract idea.
Subject Matter Eligibility Analysis Step 2B:
"” (This step is directed to transmitting or receiving information, which is understood to be insignificant extra solution activity and well understood, routine and conventional activity of transmitting and receiving data and/or gathering and analyzing information using conventional techniques and displaying the result as identified by the court - see MPEP 2106.05(d))
“an abnormality index acquisition unit that uses the at least one processor ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“an estimation unit that uses the at least one processor to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
“an output unit that uses the at least one processor to execute an output in accordance with a result of the estimate” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
The additional elements as disclosed above alone or in combination do not recite significantly more than the abstract idea itself as they are mere insignificant extra solution activity in combination of generic computer functions being implemented with generic computer elements in a high level of generality to perform the disclosed abstract idea above. Therefore, Claim 1 is subject-matter ineligible.
Regarding Claim 2:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the estimation unit uses the at least one processor to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 3:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“” (This step is directed to data gathering, which is understood to be insignificant extra solution activity (2106.05(g) in step 2A prong 2) and well understood, routine and conventional activity of transmitting and receiving data as identified by the court (2106.05(d) in step 2B))
“wherein the output unit uses the at least one processor to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 4:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“wherein the estimation unit uses the at least one processor to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“” (This step is directed to data gathering, which is understood to be insignificant extra solution activity (2106.05(g) in step 2A prong 2) and well understood, routine and conventional activity of transmitting and receiving data as identified by the court (2106.05(d) in step 2B))
“wherein the output unit uses the at least one processor to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“” (This step is directed to data gathering, which is understood to be insignificant extra solution activity (2106.05(g) in step 2A prong 2) and well understood, routine and conventional activity of transmitting and receiving data as identified by the court (2106.05(d) in step 2B))
“wherein the output unit uses the at least one processor to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claims 7, 8, 9, and 10:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“” (This step is directed to data gathering, which is understood to be insignificant extra solution activity (2106.05(g) in step 2A prong 2) and well understood, routine and conventional activity of transmitting and receiving data as identified by the court (2106.05(d) in step 2B))
“a state data acquisition unit that uses the at least one processor to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 11:
Subject Matter Eligibility Analysis Step 2A Prong 1:
“” (a mental process that can be performed in the human mind, i.e. judgement)
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“an abnormality detection unit that uses the at least one processor to ” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 12:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“a control unit that uses the at least one processor to control the control target by using the operation model” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 13, 14, and 15:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“an operation model generation unit that uses the at least one processor to generate the operation model by reinforcement learning” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 16, 17, and 18:
Subject Matter Eligibility Analysis Step 2A Prong 1: None
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“an evaluation model generation unit that uses the at least one processor to generate the evaluation model by machine learning” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Regarding Claim 19:
The claim recites a method that performs the process as described in claim 1. Therefore, claim 19 is rejected for the same reasons as disclosed for claim 1.
Regarding Claim 20:
The claim recites an article of manufacture that performs the method as described in claim 1. Therefore, claim 20 is rejected for the same reasons as disclosed for claim 1. The limitations for additional elements of claim 20 are analyzed below.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Please see Step 2A Prong 1 analysis of claim 1
Subject Matter Eligibility Analysis Step 2A Prong 2 & 2B:
“A non-transitory computer-readable medium having recorded thereon an estimation program that is executed by a computer having at least one processor and that causes the computer to function as” (mere instructions to apply the exception using a generic computer component - see MPEP 2106.05(f))
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 4, 7-8, 10-14, 16-17, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Fukuda (US20070255482A1) in view of Lee (US20100023307A1).
Regarding claim 1, Fukuda teaches:
“An estimation apparatus comprising: at least one processor” (abstract, 0034, An evaluation device outputs a manipulated variable of a predetermined controlled equipment. The evaluation device includes a calculation device that is provided as a computer unit having a microprocessor.)
“an abnormality index acquisition unit that uses the at least one processor to acquire, as an abnormality index, an evaluation index which is output by an evaluation model according to state data being input when an abnormality occurs in a facility, among pieces of the state data indicating a state of the facility when an operation model is used to control a control target provided in the facility, the operation model being generated ” ([0008, 0034-0035, 0037-0038, 0040-0041, 0043, 0050, 0057, Figure 1], The analyzer (evaluation model) monitors the bed temperature presumption values output from the state quantity presumption model and evaluate the risk of the bed temperature peak value. The analyzer may be a FMEA model. The degree of risk (evaluation index) is sent to the simulation controller (abnormality index acquisition unit) and stored in an internal memory of the calculation device. The state quantity presumption model outputs state quantities to the simulation controller. The state quantity presumption model presumes the state quantity reflecting the error (abnormality). The evaluation of control systems may include plant facilities. The ECU-model (operation model) calculates manipulated variable for controlling the engine. The engine control unit operates various equipment so the engine is controlled to be in a target operation. The ECU-model may output changes (action) for the control algorithm for determining the additive amount based on differences in bed temperature. The simulation controller is executed by the processor of the calculation device.)
“an estimation unit that uses the at least one processor to estimate ” ([0034, 0050-0054], A Risk Priority Number (RPN) is calculated based on the degree of detection, which is evaluated for the bed temperature presumption error. A high RPN may indicate low robustness of the control system and in some embodiment, requires to improve the accuracy of the algorithm related to the control system. The analyzer is executed by the processor of the calculation device.)
“an output unit that uses the at least one processor to execute an output in accordance with a result of the estimate” ([0011, 0053], The simulation controller outputs the degree of risk and other results to a monitor. The analyzing results may be displayed for a user and the output may be highlighted to indicate a problem with the system. The monitor is an output device and it is inherent that a monitor uses a graphics processing unit to display the results on the monitor.)
Fukuda does not explicitly disclose an implementation of “the operation model being generated by reinforcement learning in which an output of the evaluation model trained by machine learning to output the evaluation index in accordance with the state of the facility is set as at least a part of a reward”, and “an estimation unit that uses the at least one processor to estimate which of the evaluation model or the operation model is a main cause of the abnormality, based on the abnormality index”. However, Lee discloses in the same field of endeavor:
“... the operation model being generated by reinforcement learning in which an output of the evaluation model trained by machine learning to output the evaluation index in accordance with the state of the facility is set as at least a part of a reward ...” ([0019, 0033, 0052-0053, 0068, Figure 4], A plurality of prediction models may be used for prognosing mechanical systems such as ARMA and RNN models. A reinforcement learning framework with a positive and negative reward is used to train the prediction models. The action of selecting a particular model is determined based on a certain state. Q-values are determined for each state/action pair as shown in Figure 4. A high Q-value indicates the most appropriate model to select for a certain state and a low Q-value may indicate a model that may not perform well.)
“an estimation unit that uses the at least one processor to estimate which of the evaluation model or the operation model is a main cause of the abnormality, based on the abnormality index” ([0052-0053, 0060, 0110], A reinforcement learning model rewards the prediction model based on the prediction accuracy of the prediction model. A high Q-value indicates the most accurate model to select for a certain state and a low Q-value may indicate a model having a large error in prediction accuracy. The estimation process can be implemented in one or more microprocessor based systems.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “the operation model being generated by reinforcement learning in which an output of the evaluation model trained by machine learning to output the evaluation index in accordance with the state of the facility is set as at least a part of a reward”, and “an estimation unit that uses the at least one processor to estimate which of the evaluation model or the operation model is a main cause of the abnormality, based on the abnormality index” from Lee into the teaching of Fukuda. Doing so can improve the process of evaluating when a failure in a mechanical system occurs by implementing a reinforcement learning model to determine the most accuracy prediction model (Lee, abstract).
Regarding claim 2, Fukuda in view of Lee teaches:
“wherein the estimation unit uses the at least one processor to estimate that the main cause of the abnormality is the operation model, when the abnormality index does not satisfy a predetermined criterion” ([Lee, 0048-0056, 0060, 0110], The reinforcement learning framework continues to train the models for multiple iterations. Models having low accuracy are trained to maximize the reward. The estimation process can be implemented in one or more microprocessor based systems.)
Regarding claim 4, Fukuda in view of Lee teaches:
“wherein the estimation unit uses the at least one processor to estimate that the main cause of the abnormality is the evaluation model, when the abnormality index does not satisfy a predetermined criterion” ([Lee, 0048-0056, 0060, 0099-0100, 0110], The reinforcement learning framework continues to train the models for multiple iterations. Models having low accuracy are trained to maximize the reward. The estimation process can be implemented in one or more microprocessor based systems.)
Regarding claims 7, 8, and 10, Fukuda in view of Lee teaches:
“a state data acquisition unit that uses the at least one processor to acquire the state data” ([Fukuda, 0033], The catalyst temperature may be detected by a catalyst temperature sensor. There may also be other input equipment of the controlled system. The engine control unit is implemented as a computer unit. It is inherent that the computer unit consist of at least one processor.)
Regarding claim 11, Fukuda in view of Lee teaches:
“an abnormality detection unit that uses the at least one processor to detect that an abnormality has occurred in the facility, based on the state data” ([Fukuda, 0034, 0037-0039], The state quantity presumption model predicts determines the state of the controlled system based on the input and the error. The state quantity presumption model is executed by the processor of the calculation device.)
Regarding claim 12, Fukuda in view of Lee teaches:
“a control unit that uses the at least one processor to control the control target by using the operation model” ([Fukuda, 0034-0035], The ECU-equivalent model is used to provide operating parameters to the ECU. The ECU-equivalent model is executed by the processor of the calculation device.)
Regarding claims 13, and 14, Fukuda in view of Lee teaches:
“an operation model generation unit that uses the at least one processor to generate the operation model by reinforcement learning” ([Lee, 0052, 0110], A reinforcement learning framework is used to generate a plurality of models for predicting the state of a system. The framework is implemented in one or more microprocessor based system.)
Regarding claims 16, and 17, Fukuda in view of Lee teaches:
“an evaluation model generation unit that uses the at least one processor to generate the evaluation model by machine learning” ([Lee, 0052, 0110], A reinforcement learning framework is used to generate a plurality of models for predicting the state of a system. The framework is implemented in one or more microprocessor-based system.)
Regarding Claim 19:
The claim recites a method that performs the process as described in claim 1. Therefore, claim 19 is rejected for the same reasons as disclosed for claim 1.
Regarding Claim 20:
Claim 20 recites an article of manufacture that performs the same process as described in Claim 1. Therefore claim 20 is rejected under the same reasons mention for claim 1. The additional elements of claim 20 is addressed below by Lee:
“A non-transitory computer-readable medium having recorded thereon an estimation program that is executed by a computer having at least one processor and that causes the computer to function as” ([0110], A conventional computer readable medium is used to store instructions to perform the steps of the invention.)
Claims 3, 5-6, 9, 15, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Fukuda (US20070255482A1) in view of Lee (US20100023307A1) and Maiorana (US10904381B1).
Regarding claim 3, Fukuda in view of Lee teaches:
“wherein the output unit uses the at least one processor ” ([Lee, 0051-0056, 0060, 0101-0110], The goal of the reinforcement learning framework is to train the plurality of models to maximize the reward. In some embodiment, when a model receives a negative reward, the model will be retrained to optimize its prediction accuracy. The training results of Q-values are displayed and it would be obvious that the models having low Q-values needs to be retrained. Experiments were conducted to generate a health map that represents the failure mode of bearings. The trivial features were determined using 2 different methods that impairs the performance and the model was re-trained using the new set of features.)
Fukuda in view of Lee does not explicitly disclose an implementation of “output a message of an instruction”. However, Maiorana discloses in the same field of endeavor:
“wherein the output unit uses the at least one processor to output a message of an instruction to train the operation model by relearning ...” ([col. 11, lines 5-20, col. 17, lines 52-57], A performance monitoring unit may output an instruction to re-train the deep learning model.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “output a message of an instruction” from Maiorana into the teaching of Fukuda in view of Lee. Doing so can improve the machine learning process of performing a particular predictive task by re-training to improve the model’s accuracy (Maiorana, abstract, col. 10, par. 52-57).
Regarding claim 5, Fukuda in view of Lee teaches:
“wherein the output unit uses the at least one processor to ” ([Lee, 0051-0056, 0060, 0101-0110], The goal of the reinforcement learning framework is to train the plurality of models to maximize the reward. In some embodiment, when a model receives a negative reward, the model will be retrained to optimize its prediction accuracy. The training results of Q-values are displayed and it would be obvious that the models having low Q-values needs to be retrained. Experiments were conducted to generate a health map that represents the failure mode of bearings. The trivial features were determined using 2 different methods that impairs the performance and the model was re-trained using the new set of features.)
Fukuda in view of Lee does not explicitly disclose an implementation of “output a message of an instruction”. However, Maiorana discloses in the same field of endeavor:
“wherein the output unit uses the at least one processor to output a message of an instruction to train the operation model by relearning ...” ([col. 11, par. 5-20, col. 17, lines 52-57], A performance monitoring unit may output an instruction to re-train the deep learning model.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “output a message of an instruction” from Maiorana into the teaching of Fukuda in view of Lee. Doing so can improve the machine learning process of performing a particular predictive task by re-training to improve the model’s accuracy (Maiorana, abstract, col. 10, par. 52-57).
Regarding claim 6, Fukuda in view of Lee teaches:
“wherein the output unit uses the at least one processor to output, in a case ” ([Lee, 0051-0056, 0060, 0101-0110], The training results of Q-values are displayed and it would be obvious that the models having low Q-values needs to be retrained. The Q-values represent the sum of the reinforcements received when performing an action following a given policy. The re-training of these models would determine a new reward for each model.)
Fukuda in view of Lee does not explicitly disclose an implementation of “a message of an instruction”. However, Maiorana discloses in the same field of endeavor:
“wherein the output unit uses the at least one processor to output, in a case where the message of the instruction to train the evaluation model by the relearning is output, a message of an instruction to train the operation model by relearning ...” ([col. 11, par. 5-20, col. 17, lines 52-57, Figure 2], A performance monitoring unit may output an instruction to re-train the deep learning model. The system consists of a deep learning model and a fraud detection unit.)
It would be obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of “a message of an instruction” from Maiorana into the teaching of Fukuda in view of Lee. Doing so can improve the machine learning process of performing a particular predictive task by re-training to improve the model’s accuracy (Maiorana, abstract, col. 10, par. 52-57).
Regarding claim 9, Fukuda in view of Lee teaches:
“a state data acquisition unit that uses the at least one processor to acquire the state data” ([Fukuda, 0033], The catalyst temperature may be detected by a catalyst temperature sensor. There may also be other input equipment of the controlled system. The engine control unit is implemented as a computer unit. It is inherent that the computer unit consist of at least one processor.)
Regarding claim 15, Fukuda in view of Lee teaches:
“an operation model generation unit that uses the at least one processor to generate the operation model by reinforcement learning” ([Lee, 0052, 0110], A reinforcement learning framework is used to generate a plurality of models for predicting the state of a system. The framework is implemented in one or more microprocessor based system.)
Regarding claim 18, Fukuda in view of Lee teaches:
“an evaluation model generation unit that uses the at least one processor to generate the evaluation model by machine learning” ([Lee, 0052, 0110], A reinforcement learning framework is used to generate a plurality of models for predicting the state of a system. The framework is implemented in one or more microprocessor based system.)
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GARY MAC whose telephone number is (703)756-1517. The examiner can normally be reached Monday - Friday 8:00 AM - 5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GARY MAC/Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights