DETAILED ACTION
DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
Applicant is reminded of the proper content of an abstract of the disclosure.
A patent abstract is a concise statement of the technical disclosure of the patent and should include that which is new in the art to which the invention pertains. The abstract should not refer to purported merits or speculative applications of the invention and should not compare the invention with the prior art.
If the patent is of a basic nature, the entire technical disclosure may be new in the art, and the abstract should be directed to the entire disclosure. If the patent is in the nature of an improvement in an old apparatus, process, product, or composition, the abstract should include the technical disclosure of the improvement. The abstract should also mention by way of example any preferred modifications or alternatives.
Where applicable, the abstract should include the following: (1) if a machine or apparatus, its organization and operation; (2) if an article, its method of making; (3) if a chemical compound, its identity and use; (4) if a mixture, its ingredients; (5) if a process, the steps.
Extensive mechanical and design details of an apparatus should not be included in the abstract. The abstract should be in narrative form and generally limited to a single paragraph within the range of 50 to 150 words in length. In particular, the abstract is not a single paragraph.
See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. MPEP 2106.03.
The claim is to a computer-implemented method, i.e. one of the statutory categories.
Step 2A prong one: This part of the eligibility analysis evaluates whether the claim recites a judicial exception. As explained in MPEP 2106.04(11) and the October 2019 Update, a claim "recites" a judicial exception when the judicial exception is "set forth" or "described" in the claim.
The claim recites:
“providing the cost function for the purpose of determining system parameters of the technical system,
determining a function space with a definition space, wherein the function space corresponds to a set of functions in which the cost function lies, and
determining random system parameters that lie in the definition space, and
applying the random system parameters to the technical system and determining the output values corresponding to the random system parameters, and
modeling the technical system by way of a statistical analysis method, and training the statistical analysis method using the system parameters as input values and the output values as target values, and
generating a plurality of rules on which the technical system is based and which are based on the different system parameters and the corresponding output values,
generating a plurality of probability functions using one or more of the plurality of rules, wherein each of the probability functions indicates a probability with which the rule is satisfied by any cost function from the function space,
combining all probability functions in order to determine the cost function by increasing the overall probability of all rules,”
These limitations recite concepts that can be practically performed in the human mind but for the recitation of generic computer components. Thus, the limitations fall into the “Mental Processes” grouping of abstract ideas. (Step 2A prong one: YES).
Step 2A prong two: This part of the eligibility analysis evaluates whether the claim as a whole integrates the recited judicial exception into a practical application of the exception. This evaluation is performed by (a) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (b) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. 2019 PEG Section lll{A){2), 84 Fed. Reg. at 54-55.
This judicial exception is not integrated into a practical application because: Besides the abstract idea, the claim recites the additional limitations of:
“A computer-implemented method for determining system parameters of a technical system using a cost function, the method comprising:
wherein the technical system has different components that are adjustable by the system parameters, and
wherein, when the system parameters are set, the technical system generates different output values for the different components,
updating the system parameters given the cost function, and
outputting the updated system parameters in order to adjust the components of the technical system.”
The computer-implemented method and a technical system are a recited at a high level of generality and are recited as performing generic computer functions routinely used in computer applications. Thus, these limitations represent no more than mere instructions to apply the judicial exceptions on a computer.
The limitations “wherein the technical system has different components that are adjustable by the system parameters, and wherein, when the system parameters are set, the technical system generates different output values for the different components, updating the system parameters given the cost function, and outputting the updated system parameters in order to adjust the components of the technical system” does not integrate the invention into a practical application because it’s just “applying” the abstract idea. It can also be viewed as generally linking the use of the judicial exception to a technological environment.
It should be noted that because the courts have made it clear that mere physicality or tangibility of an additional element or elements is not a relevant consideration in the eligibility analysis, computer-implemented method and a technical system do not affect this analysis. See MPEP 2106.05(1) for more information on this point, including explanations from judicial decisions including Alice Corp. Pty. Ltd. v. CLS Bank lnt'I, 573 U.S. 208, 224-26 (2014).
Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception (Step 2A prong two: NO).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim. MPEP 2106.05
Regarding the additional elements:
The computer-implemented method and a technical system are recited at a high level of generality and are recited as performing generic computer functions routinely used in computer applications. Thus, these limitations represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP 2106.05(f) Implementing an abstract idea on generic electronic components as a tool to perform an abstract idea does not amount to significantly more. See Elec. Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1355 (Fed. Cir. 2016) (“Nothing in the claims, understood in light of the specification, requires anything other than off-the-shelf, conventional computer, network, and display technology for gathering, sending, and presenting the desired information.”)
The limitations “wherein the technical system has different components that are adjustable by the system parameters, and wherein, when the system parameters are set, the technical system generates different output values for the different components, updating the system parameters given the cost function, and outputting the updated system parameters in order to adjust the components of the technical system” merely add insignificant extra-solution activity to the judicial exception because they claim mere data outputting. Chen (US10909450B2) discloses update the model parameters based on the gradient of the cost function, e.g., according to a gradient-descent algorithm, and return the value of the cost function with the new parameters. Markovic (US10679642B2) discloses μ denotes a step size parameter and the update W is typically based on the gradient (with respect to the coefficient matrix W) of a certain cost function. Further, McConaghy (US8006220B2) discloses an update to the initial sample vectors and their scalar cost is made.
In view of the foregoing, in accord with MPEP 2106.05(d), simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception does not qualify the claim as reciting “significantly more”. Even when considered in combination, these additional elements represent mere instructions to apply an exception and insignificant extra-solution activity, which do not provide an inventive concept (Step 2B: NO). The claim is not patent eligible.
Regarding claims 2-7, under their broadest reasonable interpretation, the limitations of claims 2-7 further defines the method, which have been established to include abstract ideas. There are no additional limitations in the claims to apply, rely on, or use the judicial exception in a manner that would impose a meaningful limit on the judicial exception. Accordingly, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. Thus, claims 2-7 are not patent eligible.
Regarding claim 8, the claims have similar limitations as claim 1; moreover, claim 8 recites a computer system, which is generic computer components and does not practically integrate the invention nor amount to significantly more. The claim 8 is not patent eligible.
Dependent claims 9-11 are the claims have similar limitations as claims 2, 4, and 7. Therefore, the rejections applied to claims 2, 4, and 7 above also apply to claims 9-11, and as such, they are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2 and 4-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Klenske et al. (US20220197229A1 -hereinafter Klenske) in view of Brochu et al. (NPL: "A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning." -hereinafter Brochu)
Regarding Claim 1, Klenske teaches a computer-implemented method for determining system parameters of a technical system using a cost function (see [0013]; Klenske: “The quality function (cost function) corresponds to a mathematical model that assesses the quality of the behavior of the control process of the technical system depending on the model parameters of the underlying control strategy.”), the method comprising:
providing the cost function for the purpose of determining system parameters of the technical system (see [0029]; Klenske: “the expansion distance with respect to the relevant dimensions can be indicated depending on a gradient of a Gaussian process posterior mean of the Gaussian cost function for the respectively current model parameters, the expansion distance assuming a higher relative or absolute value, the greater the gradient.”), wherein the technical system has different components that are adjustable by the system parameters (see [0089]-[0090]; Klenske: “The model parameters of the control strategy can be adapted while technical system 2 is being controlled, continuously or at regular or predetermined times, so that the control process of technical system 2 perceptibly improves as operation proceeds.” See [0045]: “Operation of technical system 2 occurs depending on input variables u(t) with the aid of one or several actuators 22 of technical system 2. For example, a motion of a robot or vehicle can be controlled, or a drive unit or a driver assistance system of a vehicle can be controlled.”), and wherein, when the system parameters are set, the technical system generates different output values for the different components, (see [0045]; Klenske: “Control is applied to actuator 22 in accordance with the one or several input variables u, and it performs a corresponding action. See [0046]: “In an exemplifying embodiment of the present invention, control unit 3 is used to control an internal combustion engine constituting a technical system.”. See [0047]: “In an exemplifying embodiment, control unit 3 is used to control an at least semiautonomous robot, in particular an at least semiautonomous motor vehicle, constituting technical system 2.”)
-determining a function space with a definition space (see [0028]; Klenske: “Adapting the value ranges for model parameters makes it possible, proceeding from a limited value range, to adapt that range dynamically during optimization only for those dimensions of the model parameter vector for which optimization relates to a boundary region of the model parameter domain.”), and
-determining random system parameters that lie in the definition space, (see [0081]; Klenske: “Step S8 checks whether one of the model parameters of the test model parameter set lies at a boundary of the model parameter domain.”)
-applying the random system parameters to the technical system and determining the output values corresponding to the random system parameters, and (see [0043]; Klenske: “A control unit 3 applies control to technical system 2 with a sequence of input variables u constituting manipulated variables, which result in specific operating points (system states) of technical system 2.”)
-modeling the technical system by way of a statistical analysis method, and training the statistical analysis method using the system parameters as input values and the output values as target values, and (see [0057]; Klenske: “A quality function is modeled with the aid of a Gaussian process regression or another trainable regression method with which the performance capability of the control process of technical system 2 is defined as a function of the model parameters.”)
-updating the system parameters given the cost function, and (see [0083]; Klenske: “An estimated value of the optimum of the model parameter values, i.e. the minimum of the approximated quality function in the current model parameter domain Θi, exists during execution of the Bayesian optimization.”)
-outputting the updated system parameters in order to adjust the components of the technical system. (see [0089]-[0090]; Klenske: “The model parameters of the control strategy can be adapted while technical system 2 is being controlled, continuously or at regular or predetermined times, so that the control process of technical system 2 perceptibly improves as operation proceeds.”)
However, Klenske does not explicitly teach:
wherein the function space corresponds to a set of functions in which the cost function lies,
-generating a plurality of rules on which the technical system is based and which are based on the different system parameters and the corresponding output values,
-generating a plurality of probability functions using one or more of the plurality of rules, wherein each of the probability functions indicates a probability with which the rule is satisfied by any cost function from the function space,
-combining all probability functions in order to determine the cost function by increasing the overall probability of all rules,
Brochu from the same or similar field of endeavor teaches:
wherein the function space corresponds to a set of functions in which the cost function lies, (see page 5, last paragraph; Brochu: “Bayesian optimization uses the prior and evidence to define a posterior distribution over the space of functions. The Bayesian model allows for an elegant means by which informative priors can describe attributes of the objective function, such as smoothness or the most likely locations of the maximum, even when the function itself is not known.”)
-generating a plurality of rules on which the technical system is based and which are based on the different system parameters and the corresponding output values, (see page 37, first paragraph; Brochu: “Figure 11: Task Hierarchies. Each composite task is a separate SMDP whose policy is optimal given the optimal policies of its subtasks (recursive optimality).”)
-generating a plurality of probability functions using one or more of the plurality of rules (see page 32, paragraph 4; Brochu: “Each task in an HRL hierarchy is a semi-Markov Decision Process [Sutton et al., 1999], that models repeated decision making in a stochastic environment, where the actions can take more than one time step. Formally, an SMDP is defined as a tuple: {S, A, P(s’, N|s,a), R(s,a)} where S is the set of state variables, A is a set of actions, P(s’, N|s,a) is the transition probability of arriving to state s’ in N time steps after taking action a in s, and R(s,a)is the reward received. The solution of this process is a policy π* ϵ A, that selects the action with the highest expected discounted reward in each state.” See page 33, last paragraph: “Recursive optimality, satisfied by MAXQ and RAR, means that each subtask is locally optimal, given the optimal policies of the descendants.”) [The policies read on ‘rules’], wherein each of the probability functions indicates a probability with which the rule is satisfied by any cost function from the function space, (see page 32, paragraph 4; Brochu: “The function V*(s) is the value of state s when following the optimal policy. Equivalently, the Q* (s,a) function stores the value of taking action a in state s and following the optimal policy thereafter.”)
-combining all probability functions in order to determine the cost function by increasing the overall probability of all rules, (see page 11, second paragraph; Brochu: “The early work of Kushner [1964] suggested maximizing the probability of improvement over the incumbent f(x+) … This function is also sometimes called MPI (for maximum probability of improvement") or \the P-algorithm" (since the utility is the probability of improvement).”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the teaching of Klenske to include Brochu’s features of wherein the function space corresponds to a set of functions in which the cost function lies, generating a plurality of rules on which the technical system is based and which are based on the different system parameters and the corresponding output values, generating a plurality of probability functions using one or more of the plurality of rules, wherein each of the probability functions indicates a probability with which the rule is satisfied by any cost function from the function space, and combining all probability functions in order to determine the cost function by increasing the overall probability of all rules. Doing so would significantly speed up the learning process. (Brochu, page 31, third paragraph)
Regarding Claim 2, the combination of Klenske and Brochu teaches all the limitations of claim 1 above, Klenske further teaches further comprising:
inputting the updated system parameters obtained by the determined cost function into the trained statistical analysis method (see [0019]; Klenske: “After evaluation of the behavior of the technical system (for instance by measurement at the evaluation point), the trainable regression function is updated or retrained using the new data, and a next evaluation point is selected using the method described above.”) and determining the new output values by way of the trained statistical analysis method. (see [0019]-[0020]; Klenske: “Typically, this process is repeated until a termination criterion is met, i.e., for instance the length of time that is to be spent optimizing the unknown function f (i.e. the behavior of the technical system). Once the optimization process is complete, the function value x is recommended as the location of the minimum of the quality function, i.e., the value that minimizes the expected value of the Gaussian process.”)
Regarding Claim 4, the combination of Klenske and Brochu teaches all the limitations of claim 1 above, Klenske further teaches wherein the trained statistical analysis method has an uncertainty and the cost function has an uncertainty (see [0015]; Klenske: “This is reflected in a low uncertainty in the quality function.”), and wherein the computer-implemented method further comprises:
reducing the cost function with regard to the uncertainties in order to obtain the updated system parameters (see [0017]; Klenske: “a new evaluation point for evaluation or measurement of the system behavior is selected in such a way that on the one hand the informative value of the quality function is improved, so that the uncertainty of the estimated expected value of the quality function is reduced.”), and inputting the system parameters updated thereby into the trained statistical analysis method (see [0019]; Klenske: “After evaluation of the behavior of the technical system (for instance by measurement at the evaluation point), the trainable regression function is updated or retrained using the new data, and a next evaluation point is selected using the method described above.”), and determining new output values by way of the trained statistical analysis method. (see [0019]-[0020]; Klenske: “Typically, this process is repeated until a termination criterion is met, i.e., for instance the length of time that is to be spent optimizing the unknown function f (i.e. the behavior of the technical system). Once the optimization process is complete, the function value x is recommended as the location of the minimum of the quality function, i.e., the value that minimizes the expected value of the Gaussian process.”)
Regarding Claim 5, the combination of Klenske and Brochu teaches all the limitations of claim 4 above, Brochu further teaches wherein methods from a field of active learning are used to reduce the uncertainty of the cost function and the uncertainty of the trained statistical analysis method. (see page 3, last paragraph; Brochu: “The decision represents an automatic trade-off between exploration (where the objective function is very uncertain) and exploitation (trying values of x where the objective function is expected to be high). This optimization technique has the nice property that it aims to minimize the number of objective function evaluations.”)
The same motivation to combine Klenske and Brochu a set forth for Claim 1 equally applies to Claim 5.
Regarding Claim 6, the combination of Klenske and Brochu teaches all the limitations of claim 1 above, Klenske further teaches wherein a regression analysis is used as the trained statistical analysis method. (see [0012]; Klenske: “Bayesian optimization methods for ascertaining a control model iteratively apply various control strategies using a technical system, and efficiently optimize the control process. The quality function can be modeled with the aid of a trainable regression function, in particular with the aid of a Gaussian process regression, in order to model the performance capability of the system model as a function of the model parameters of the control model, the Gaussian process regression being created based on measured (and therefore noisy) state variables.”) [That is, Gaussian process regression reads on ‘the trained statistical analysis method’]
The same motivation to combine Klenske and Brochu a set forth for Claim 1 equally applies to Claim 6.
Regarding Claim 7, the combination of Klenske and Brochu teaches all the limitations of claim 6 above, Klenske further teaches wherein a Gaussian process regression is used as the regression analysis. (see [0012]; Klenske: “The quality function can be modeled with the aid of a trainable regression function, in particular with the aid of a Gaussian process regression.”)
Regarding Claim 8, the limitations in this claim is taught by the combination of Klenske and Brochu as discussed connection with claim 1.
Regarding Claim 9, the combination of Klenske and Brochu teaches all the limitations of claim 8 above, Klenske further teaches wherein the processor is further configured to optimize update the cost function with respect to the costs in order to obtain optimized the updated system parameters (see [0023]; Klenske: “The quality function can, if applicable, be constantly updated, depending on the respective model parameters, in accordance with the performance capability of the control system. Updating of the quality function requires an assessment of the respective model parameters, which necessitates operation of the control system in the real environment using the respective model parameters.”) and to input the updated system parameters into the trained statistical analysis method (see [0019]; Klenske: “After evaluation of the behavior of the technical system (for instance by measurement at the evaluation point), the trainable regression function is updated or retrained using the new data, and a next evaluation point is selected using the method described above.”) and to determine new output values by way of the statistical analysis method. (see [0019]-[0020]; Klenske: “Typically, this process is repeated until a termination criterion is met, i.e., for instance the length of time that is to be spent optimizing the unknown function f (i.e. the behavior of the technical system). Once the optimization process is complete, the function value x is recommended as the location of the minimum of the quality function, i.e., the value that minimizes the expected value of the Gaussian process.”)
Regarding Claim 10, the limitations in this claim is taught by the combination of Klenske and Brochu as discussed connection with claim 4.
Regarding Claim 11, the limitations in this claim is taught by the combination of Klenske and Brochu as discussed connection with claim 7.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Klenske et al. (US20220197229A1 -hereinafter Klenske) in view of Brochu in view of Maier et al. (US20180139451A1 -hereinafter Maier).
Regarding Claim 3, the combination of Klenske and Brochu teaches all the limitations of claim 2 above, Brochu further teaches further comprising:
aborting the computer-implemented method in a case of an abort criterion (see page 31, last paragraph; Brochu: “Manually coding hierarchical policies is the mainstay of video game AI development. The requirements for automated HRL to be a viable solution are it must be easy to customize task-specific implementations, state abstractions, reward models, termination criteria and it must support continuous state and action spaces.”) [Customizing termination criteria reads on ‘an abort criterion’],
The same motivation to combine Klenske and Brochu a set forth for Claim 1 equally applies to Claim 3.
However, it does not explicitly teach wherein the abort criterion depends on predefined costs.
Maier from the same or similar field of endeavor teaches wherein the abort criterion depends on predefined costs. (see [0034]; Maier: “The sequential testing of refiner 22 is aborted according to a predetermined abort criterion. Several alternatives exist for defining the predetermined abort criterion, which leads to an aborting of the sequential testing so that, relating to all possible circumstances including, for example, the coding costs associated with set 26, an aborting of the sequential testing prior to having tested all higher-pel resolution candidate vectors of set 30 is possible. More advantageously, the predetermined abort criterion is such that an abort of the sequential testing takes place for more than half, or even all of, the manifold combinations of coding costs of the set 26 of lower-pel resolution vectors.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify the teaching of Klenske and Brochu to include Maier’s features of the abort criterion depends on predefined costs. Doing so would target an optimal trade-off between computational complexity and coding efficiency. (Maier, [0007])
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Hewing (NPL: “Cautious Model Predictive Control using Gaussian Process Regression”) discloses Gaussian process regression as a probability distribution over functions, such that every finite sample of function values is jointly Gaussian distributed.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VI N TRAN whose telephone number is (571)272-1108. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ROBERT FENNEMA can be reached at (571) 272-2748. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.N.T./Examiner, Art Unit 2117
/ROBERT E FENNEMA/Supervisory Patent Examiner, Art Unit 2117