DETAILED ACTION
This action is in response to the amendment filed 07/09/2026. Claims 1 and 4-14 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 1 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Examiner notes that lines 16-22 of claim 1 recite “wherein said selecting includes modifying, on said basis of statistics, the value of the at least one hyperparameters used in the instances of the ML algorithm not selected for continued use in controlling the technical system to be more similar to the at least one hyperparameters of the at least one instance selected for continued use in controlling the technical system, repeating or continuing a training of the instances of the ML algorithm not selected for continued use in the technical system using the modified at least one hyperparameters value, and repeating the mapping and evaluating steps.” Examiner notes that the specification does not reasonably convey to one skilled in the relevant art modifying hyperparameters of an instance not selected for continued use. Paragraph 0035 of the instant specification describes remaining instances operating in hot-standby mode and mimicking a primary instance. Paragraph 0035 does not, however describe modifying hyperparameters or selecting an instance. Examiner respectfully notes that paragraph 0034 describes selecting an instance to replace the primary instance but does not disclose information regarding the continued training of an instance that is not selected.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, and 4-14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the basis of statistics" in line 14. There is insufficient antecedent basis for this limitation in the claim. For purposes of examination, Examiner has interpreted this basis of statistics to be the first instance of basis of statistics.
Regarding claims 4-14, claims 4-14 are rejected for at least the same reasons as claim 1 since claims 4-14 depend on claim 1.
Claim 10 recites the limitation “at least one hyperparameters” in line 2. It is unclear as to whether these at least one hyperparameters are the same hyperparameters recited in claim 1. For purposes of examination, Examiner has interpreted these hyperparameters to be the same as recited in claim 1.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-2 and 4-14 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Subject Matter Eligibility Analysis Step 1:
Claim 1 recites a method and is thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 1 recites
mapping each input state to a plurality of outputs … wherein the outputs correspond to control signals of the technical system (This limitation is a mental process as it encompasses a human mentally mapping each input to outputs.)
evaluating a predefined quality metric for the outputs, wherein the quality metric relates to a quantity for the technical system (This limitation is a mental process as it encompasses a human mentally evaluating a quality metric.)
on the basis of statistics of the quality metric for the outputs, selecting at least one instance of the ML algorithm for continued use in controlling the technical system. (This limitation is a mental process as it encompasses a human mentally selecting an instance of the algorithm.)
wherein said selecting includes modifying, on said basis of statistics, the value of the at least one hyperparameters used in the instances of the ML algorithm not selected for continued use in controlling the technical system to be more similar to the at least one hyperparameters of the at least one instance selected for continued use in controlling the technical system (This limitation is a mental process as it encompasses a human mentally changing a value of a hyperparameter to be more similar to another hyperparameter.)
repeating the mapping and evaluating steps (This limitation is a mental process as it encompasses a human mentally repeating the mental steps of mapping and evaluating)
Therefore, claim 1 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 1 further recites additional elements of
a computer-implemented method of managing a machine-learning, ML, algorithm for controlling a technical system, wherein training of the ML algorithm is dependent on at least hyperparameters (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
providing, in parallel, a plurality of instances of the ML algorithm trained using different values of the hyperparameters (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
wherein the hyperparameters include one or more of: learning rate, discount factor, parameters affecting convergence rate, probability of fallback to taking a random action. (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
obtaining a plurality of input states related to the technical system (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
by providing each input state as input to the instances of the ML algorithm (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
repeating or continuing a training of the instances of ML algorithm not selected for continued use in the technical system using the modified at least one parameter (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of iteratively training.)
Therefore, claim 1 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 1 do not provide significantly more than the abstract idea itself, taken alone and in combination because
a computer-implemented method of managing a machine-learning, ML, algorithm for controlling a technical system, wherein training of the ML algorithm is dependent on at least one hyperparameters specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Providing, in parallel, a plurality of instances of the ML algorithm trained using different values of the at least one hyperparameters is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
wherein the hyperparameters include one or more of: learning rate, discount factor, parameters affecting convergence rate, probability of fallback to taking a random action specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
obtaining a plurality of input states related to the technical system is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
by providing each input state as input to the instances of the ML algorithm is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
repeating or continuing a training of the instances of ML algorithm not selected for continued use in the technical system using the modified at least one parameter is the well understood, routine, and conventional activity of iteratively training (Metzler et al. (US 2021/0125108 A1), page 11, paragraph 0051, “A classifier is trained using a conventional iterative machine learning training process that determines trained weights for each result list position. Based on initial weights assigned to each result list position, the iterative process attempts to find optimal weights.”)
Therefore, claim 1 is subject-matter ineligible.
Regarding Claim 4:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 4 recites
wherein said modifying includes enforcing a minimum spread of the values of the at least one hyperparameters used in the instances of the ML algorithm (This limitation is a mental process as it encompasses a human mentally modifying the hyperparameters.)
Therefore, claim 4 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 4 does not further recite any additional elements. Therefore, claim 4 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 4 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 4 is subject-matter ineligible.
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 5 recites the same abstract ideas as claim 1. Therefore, claim 5 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 5 further recites additional elements of
wherein the quality metric is productivity of the technical system (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 5 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 5 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the quality metric is productivity of the technical system specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 5 is subject-matter ineligible.
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 6 recites
wherein said selecting is based on comparing averages of the quality metric across the instances of the ML algorithms (This limitation is a mental process as it encompasses a human selecting the instance of the ML algorithm based on comparing averages.)
Therefore, claim 6 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 6 further does not recite any additional elements. Therefore, claim 6 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements to provide significantly more than the abstract idea itself, taken alone and in combination, claim 6 is subject-matter ineligible.
Regarding Claim 7:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 7 recites
detecting an execution failure of the at least one instance selected to control the technical system, (This limitation is a mental process as it encompasses a human mentally identifying an execution failure.)
in response thereto, replacing the failed at least one instance with the at least one instance trained using the modified values of the at least one hyperparameters. (This limitation is a mental process as it encompasses a human mentally replacing one instance with another.
Therefore, claim 7 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 7 does not further recite any additional elements. Therefore, claim 7 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 7 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 7 is subject-matter ineligible.
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 8 recites
wherein the input state indicates the planning nodes occupied by the vehicles and the output corresponds to a sequence of motion control commands to be applied to the vehicles. (This limitation is a mental process as it further describes the abstract idea of mapping each input state to output states.)
Therefore, claim 8 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 8 further recites additional elements of
using the selected instance of the ML algorithm in a traffic planning method for controlling a plurality of vehicles (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
wherein each vehicle occupies one node in a shared set of planning nodes and is movable to other nodes along predefined edges between pairs of the nodes in accordance with a finite set of motion commands (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 8 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim # do not provide significantly more than the abstract idea itself, taken alone and in combination because
using the selected instance of the ML algorithm in a traffic planning method for controlling a plurality of vehicles specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
wherein each vehicle occupies one node in a shared set of planning nodes and is movable to other nodes along predefined edges between pairs of the nodes in accordance with a finite set of motion commands specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 8 is subject-matter ineligible.
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 9 recites the same abstract ideas as claim 1. Therefore, claim 9 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 9 further recites additional elements of
wherein the vehicles are autonomous vehicles. (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 9 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 9 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the vehicles are autonomous vehicles specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 9 is subject-matter ineligible.
Regarding Claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 10 recites the same abstract ideas as claim 1. Therefore, claim 10 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 10 further recites additional elements of
A device configured to manage a machine-learning, ML, algorithm which is dependent on at least one hyperparameters, the device comprising processing circuitry configured to execute the method of claim 1. (This element does not integrate the abstract idea into a practical application because amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 10 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 10 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A device configured to manage a machine-learning, ML, algorithm which is dependent on one or more hyperparameters, the device comprising processing circuitry configured to execute the method of claim 1 uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 10 is subject-matter ineligible.
Regarding Claim 11:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 11 recites the same abstract ideas as claim 1. Therefore, claim 11 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 11 further recites additional elements of
an interface configured to control a technical system. (This element does not integrate the abstract idea into a practical application because amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 11 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 11 do not provide significantly more than the abstract idea itself, taken alone and in combination because
an interface configured to control a technical system uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 11 is subject-matter ineligible.
Regarding Claim 12:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 12 recites the same abstract ideas as claim 1. Therefore, claim 12 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 12 further recites additional elements of
A non-transitory computer readable medium storing a computer program comprising instructions which, when executed, cause a processor to execute the method of claim 1. (This element does not integrate the abstract idea into a practical application because amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 12 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 12 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A non-transitory computer readable medium storing a computer program comprising instructions which, when executed, cause a processor to execute the method of claim 1 uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 12 is subject-matter ineligible.
Regarding Claim 13:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 13 recites the same abstract ideas as claim 1. Therefore, claim 13 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 13 further recites additional elements of
wherein the quality metric is uptime of the technical system (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 13 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 13 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the quality metric is uptime of the technical system specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 13 is subject-matter ineligible.
Regarding Claim 14:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 14 recites the same abstract ideas as claim 1. Therefore, claim 14 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 14 further recites additional elements of
wherein the technical system is a vehicle fleet and the quality metric is a driving economy indicator (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 13 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 13 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the technical system is a vehicle fleet and the quality metric is a driving economy indicator specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 13 is subject-matter ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-5 and 10-13 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Cao et al. (US 2022/0292303) (hereafter referred to as Cao).
Regarding claim 1, Cao teaches
A computer-implemented method of managing a machine-learning, ML, algorithm for controlling a technical system, wherein training of the ML algorithm is dependent on at least one hyperparameters, (Cao, page 10, paragraph 0022, “The hyperparameter tuning involves selection of the right set of hyperparameters so that an ML model can be trained with efficiency and results in an accurate production ML model” where “hyperparameter or deep learning parameter optimization/tuning involves choosing a set of optimal hyperparameter values for an ML algorithm (i.e., a parameter whose value is used to control the learning process)” (Cao, page 11, paragraph 0030). ) the method comprising:
Providing, in parallel, a plurality of instances of the ML algorithm trained using different values of the at least one hyperparameters (Cao, page 10-11, paragraph 0023, “Hyperparameter tuning typically involves a large number of exploratory experiments to test different combinations of possible values of various hyperparameters. The hyperparameter tuning can involve training an ML model with a small amount of data (e.g., small training jobs) to determine how well the model will work” where “jobs can be parallelized in the distributed configuration to potentially reduce the time needed to accomplish model exploration” (Cao, page 11, paragraph 0024). Examiner notes that the jobs are the instances of the ML algorithm);
Wherein the at least one hyperparameter is one of learning rate, discount factor, parameter affecting convergence rate, and probability of fallback to taking a random action (Cao, page 10, paragraph 0015-0020, “Some exemplary hyperparameters include: … Neural network: learning rate, number of epochs, size of mini-batch, etc.” Examiner notes that the hyperparameters include learning rate.)
obtaining a plurality of input states related to the technical system (Cao, page 14, paragraph 0049, “The resource configuration generator 302 can take as input, various parameters that specify how to generate the DT [distributed training]-configurations.” Examiner notes that the parameters are the input states.);
mapping each input state to a plurality of outputs, by providing each input state as input to the instances of the ML algorithm, wherein the outputs from the instances of the ML algorithm correspond to control signals of the technical system (Cao, page 15, paragraph 0051, “The Bayesian optimization can be performed by optimizer 306 of FIG. 3 to determine, based on prior executions of jobs using possible DT-configurations, what remaining possible DT-configurations may correspond to an optimal resource configuration. Although Bayesian optimization is used, other optimization techniques are possible. Using Bayesian optimization, the confidence gained through additional iterations (in this case executing training jobs in accordance with possible resource configurations) results in being able to better narrow down possible resource configurations. Generally, Bayesian optimization analyzes possible parameter values (in this case, DT-configuration candidates), and gradually outputs specific parameter values (again, in this case, specific DT-configuration candidates determined as being optimal) for achieving shortest possible job completion time to try/test/evaluate. That is, a job completion time can be determined based on a particular DT-configuration, which can be fed back into the Bayesian optimization process that will assess its effectiveness based on historical information, and a next DT-configuration candidate to try/test/evaluate can be output” where “hyperparameter or deep learning parameter optimization/tuning involves choosing a set of optimal hyperparameter values for an ML algorithm (i.e., a parameter whose value is used to control the learning process)” (Cao, page 11, paragraph 0030). Examiner notes analyzing parameter values and outputting specific parameter values is the act of mapping where the specific parameter values for achieving shortest possible job completion time are the outputs and the possible parameter values is the input state. Examiner notes that the DT-configuration that has the possible parameters is fed or input into the Bayesian optimization process which is a job or instance. Additionally, the specific parameter values are control signals since they control the learning process. Examiner further notes that the job is the instance of the ML algorithm as the job is used to narrow down possible resource configurations.);
evaluating a predefined quality metric for the outputs, wherein the quality metric relates to a quantity for the technical system (Cao, page 15, paragraph 0056, “A current completion time associated with job performance using a current resource configuration candidate can be compared to a previous completion time associated with a previously tested resource configuration candidate. If the current completion time does not improve over the previous completion time by at least the threshold level of time difference, that current resource configuration candidate can be deemed "good enough," such that subsequent resource configuration candidate testing can stop.” Examiner notes that the threshold level of time difference is the predefined quality metric for the outputs. Examiner further notes that the time difference is a number and thus a quantity for the technical system.);
and on the basis of statistics of the quality metric for the outputs, selecting at least one instance of the ML algorithm for continued use in controlling the technical system (Cao, page 16, paragraph 0057, “The resource configuration candidate producing the best completion time in those 70 trials may be selected as the optimum resource configuration to use for the remaining jobs” where “different resource configurations may be used to execute different subsets of training jobs” (Cao, page 16, paragraph 0060). Examiner notes that the multiple jobs or instances are selected based on the best time as the selected resource configuration executes different jobs.);
wherein said selecting includes modifying, on said basis of statistics, the value of the at least one hyperparameters used in the instances of the ML algorithm not selected for continued use in controlling the technical system to be more similar to the at least one hyperparameters of the at least one instance selected for continued use in controlling the technical system (Cao, page 16, paragraphs 0063-0064, “A resource configuration that results in the best performance metrics (e.g., shortest training time) compared to previous resource configurations can be selected as an optimal resource configuration. Alternatively, a resource configuration that satisfy a termination condition (e.g., completing within a predefined training time) can be selected as an optimal resource configuration. If the optimal resource configuration is found, remaining training jobs can be executed using the optimal resource configuration at 504. [0064] At 524, model quality can be determined for hyperparameters used to execute the training job. If the model quality does not satisfy a termination condition, a new hyperparameter can be selected. Hyperparameter tuning can continue with the new hyperparameter and the process can iterate until the hyperparameters that provide the best model quality are determined” and Cao, page 6, Figure 5,
PNG
media_image1.png
693
936
media_image1.png
Greyscale
Examiner notes that the remaining training jobs are the instances not selected. Examiner further notes that the remaining training jobs continue training at box 504. Box 504 executes the training jobs after hyperparameters are tuned or modified in box 524. Examiner further notes that by changing the hyperparameters to satisfy a termination condition, the values of the hyperparameters are more similar to the hyperparameters of the ML instance selected.)
repeating or continuing a training of the instances of the ML algorithm not selected for continued use in the technical system using the modified at least one hyperparameters value (Cao, page 16, paragraphs 0063-0064, “A resource configuration that results in the best performance metrics (e.g., shortest training time) compared to previous resource configurations can be selected as an optimal resource configuration. Alternatively, a resource configuration that satisfy a termination condition (e.g., completing within a predefined training time) can be selected as an optimal resource configuration. If the optimal resource configuration is found, remaining training jobs can be executed using the optimal resource configuration at 504. [0064] At 524, model quality can be determined for hyperparameters used to execute the training job. If the model quality does not satisfy a termination condition, a new hyperparameter can be selected. Hyperparameter tuning can continue with the new hyperparameter and the process can iterate until the hyperparameters that provide the best model quality are determined” and Cao, page 6, Figure 5,
PNG
media_image1.png
693
936
media_image1.png
Greyscale
Examiner notes that the remaining training jobs are the instances not selected. Examiner further notes that the remaining training jobs continue training at box 504. Box 504 executes the training jobs after hyperparameters are tuned or modified in box 524. Examiner further notes that by changing the hyperparameters to satisfy a termination condition, the values of the hyperparameters are more similar to the hyperparameters of the ML instance selected.) ,
and repeating the mapping and evaluating steps (Cao, page 15, paragraph 0051, “The Bayesian optimization can be performed by optimizer 306 of FIG. 3 to determine, based on prior executions of jobs using possible DT-configurations, what remaining possible DT-configurations may correspond to an optimal resource configuration. Although Bayesian optimization is used, other optimization techniques are possible. Using Bayesian optimization, the confidence gained through additional iterations (in this case executing training jobs in accordance with possible resource configurations) results in being able to better narrow down possible resource configurations. Generally, Bayesian optimization analyzes possible parameter values (in this case, DT-configuration candidates), and gradually outputs specific parameter values (again, in this case, specific DT-configuration candidates determined as being optimal) for achieving shortest possible job completion time to try/test/evaluate. That is, a job completion time can be determined based on a particular DT-configuration, which can be fed back into the Bayesian optimization process that will assess its effectiveness based on historical information, and a next DT-configuration candidate to try/test/evaluate can be output” and “at 426, the DT-loop 404 receives reports of ML job performance 424 and updates the current performance context. Each time an ML job completes, the job performance 424 is reported to the DT-loop404. At 428, the DT-loop can determine whether a stopping criterion (e.g., a termination condition) has been reached. [0056] The stopping criterion can be an improvement in the completion time between the best DT-configuration and the second best DT-configuration is at a smaller than a threshold level of time difference. That is, a current completion time associated with job performance using a current resource configuration candidate can be compared to a previous completion time associated with a previously tested resource configuration candidate. If the current completion time does not improve over the previous completion time by at least the threshold level of time difference, that current resource configuration candidate can be deemed "good enough," such that subsequent resource configuration candidate testing can stop” (Cao, page 15, paragraph 0055-0056) where “if the DT-loop 404 determines that at least one of the above conditions is reached, then the DT-loop 404 can exit with the best DT-configuration at 430. If, however, further reduction in completion time is required/desired, the search for an optimal DT-configuration can continue. The DT-loop 404 can again select/generate a DT-configuration at 408 and repeat the process” (Cao, page 16, paragraph 0058). Examiner notes that trying/testing/evaluating over multiple iterations is the act of repeating the mapping step. Examiner further notes that the DT-loop repeating the process selecting/generating a DT-configuration is the act of repeating the evaluating step.
Regarding claim 4, Cao teaches
The method of claim 3, wherein said modifying includes enforcing a minimum spread of the values of the at least one hyperparameters used in the instances of the ML algorithm (Cao, page 11, paragraph 0031, “In order to optimize the model, the hyperparameters can be tuned. Tuning of the hyperparameters allows one or more values to be selected for use by/in the model.” Examiner notes that the minimum spread of modified hyperparameters is tuning the hyperparameters to optimize the model.).
Regarding claim 5, Cao teaches
The method of claim 1, wherein the quality metric is productivity of the technical system (Cao, page 15, paragraph 0056, “A current completion time associated with job performance using a current resource configuration candidate can be compared to a previous completion time associated with a previously tested resource configuration candidate. If the current completion time does not improve over the previous completion time by at least the threshold level of time difference, that current resource configuration candidate can be deemed "good enough," such that subsequent resource configuration candidate testing can stop.” Examiner notes that the threshold level of time difference is the predefined quality metric for the outputs. Examiner further notes that the completion time is the productivity.).
Regarding claim 10, Cao teaches
A device configured to manage a machine-learning, ML, algorithm which is dependent on at least one hyperparameters, the device comprising processing circuitry configured to execute the method of claim 1 (Cao, page 18, paragraph 0084, “Such instructions may be read into main memory 806 from another storage medium, such as storage device 810. Execution of the sequences of instructions contained in main memory 806 causes processor(s) 804 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions” where “the environment 200 provides a network environment for implementing machine learning models (Cao, page 12, paragraph 0034) and where “The hyperparameter tuning involves selection of the right set of hyperparameters so that an ML model can be trained with efficiency and results in an accurate production ML model” (Cao, page 10, paragraph 0022).).
Regarding claim 11, Cao teaches
The system of claim 10, further comprising an interface configured to control a technical system (Cao, page 12, paragraph 0035, “Each deployment cluster can have an associated application programming interface (API) server configured for dependent distribution to allocate large-scale processing clusters in the environment 200.”).
Regarding claim 12, Cao teaches
A non-transitory computer readable medium storing a computer program comprising instructions which, when executed, cause a processor to execute the method of claim 1 (Cao, page 17, paragraph 0070-0071, “In some embodiments, machine-readable storage medium 704 may be a non-transitory storage medium, where the term “non-transitory” does not encompass transitory propagating signals. As described in detail below, machine-readable storage medium 704 may be encoded with executable instructions, for example, instructions 706-714. [0071] Hardware processor 702 may execute instruction 706 to determine a plurality of computing resource configurations used to perform machine learning model training jobs.”).
Regarding claim 13, Cao teaches
The method of claim 1, wherein the quality metric is uptime of the technical system (Cao, page 15, paragraph 0056, “A current completion time associated with job performance using a current resource configuration candidate can be compared to a previous completion time associated with a previously tested resource configuration candidate. If the current completion time does not improve over the previous completion time by at least the threshold level of time difference, that current resource configuration candidate can be deemed "good enough," such that subsequent resource configuration candidate testing can stop.” Examiner notes that the threshold level of time difference is the predefined quality metric for the outputs. Examiner further notes that the completion time is the uptime.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 6-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao in view of Marinakis et al. (US 2023/0378753 A1) (hereafter referred to as Marinakis).
Regarding claim 6, Cao teaches the method of claim 1. Cao does not teach, but Marinakis does teach
wherein said selecting is based on comparing averages of the quality metric across the instances of the ML algorithms (Marinakis, page 23, paragraph 0091-0092, “The architecture supervisor may select a machine learning model architecture that is identical to a machine learning model architecture that has previously been trained, and the method may comprise performing system simulations and evaluating a performance based on scenarios that have meanwhile been determined to cause other machine learning model architectures to underperform. [0092] The performance may be computed in accordance with a performance metric or several performance metrics” where “The performance metric(s) may be any one or any combination of: operation cost, energy not served, electricity price, system stability, available power transfer capacity, selectivity and dependability, controller stability, without being limited thereto” (Marinakis, page 23, paragraph 0100) and “The performance metric may be based on computed deviations between actions taken by the decision logic in the system simulations and actions known to be correct. [0104] The deviations can be computed in accordance with a norm. The actions known to be correct may be defined by an expert input (e.g., in supervised learning) or may be derived from historical data” (Marinakis, page 23, paragraph 0103-0104). Examiner notes that comparing averages of the quality metric is evaluating a performance that is based on computed deviations in accordance with a norm.).
Cao and Marinakis are analogous to the claimed invention because they teach updating hyperparameters in machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to compare averages like in Marinakis. Doing so “allow[s] the decision logic generation system to generate and alter scenarios according to the performance assessment” (Marinakis, page 54, paragraph 0890).
Regarding claim 7, Cao teaches the method of claim 1. Cao does not teach, but Marinakis does teach
The method of claim 1, further comprising detecting an execution failure of the at least one instance selected to control the technical system and, in response thereto, replacing the failed at least one instance with the at least one instance trained using the modified values of the at least one hyperparameters (Marinakis, page 22, paragraph 0060-0061, “The two or more different machine learning model architectures may comprise artificial neural network architectures different from each other in a number of nodes and/or a number of layers, without being limited thereto. [0061] Generating the decision logic candidates may comprise selecting a machine learning model architecture; training a decision logic candidate having the machine learning model architecture until a first termination criterion is fulfilled, and storing performance information for the trained decision logic candidate; if a second termination criterion is not fulfilled, repeating the training and storing steps for a different decision logic candidate having a different machine learning model architecture” where “methods and system are provided that allow machine learning to be harnessed for generating a decision logic….[0014] Embodiments allow the development of power system operational planning, control and protection to be performed automatically” (Marinakis, page 20-21, paragraph 0013-0014) and “The termination control module 103 may trigger the decision logic generator 60 to select another decision logic architecture, e.g., by changing hyperparameters” (Marinakis, page 57, paragraph 0966). Examiner notes that the plurality of instances of the ML algorithm are the decision logic candidates, the primary instance is the a decision logic candidate and the technical system is the automatic power system operational planning, control, and protection.),
Cao and Marinakis are analogous to the claimed invention because they teach updating hyperparameters in machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to replace a failed instance like in Marinakis. Doing so “allow[s] the decision logic generation system to generate and alter scenarios according to the performance assessment” (Marinakis, page 54, paragraph 0890).
Claim(s) 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao in view of Moreira-Matias et al. (US 2019/0318248 A1) (hereafter referred to as Matias) in further view of Petroff (US 11,049,208 B2) (hereafter referred to as Petroff)
Regarding claim 8, Cao teaches the method of claim 1. Cao does not teach, but Matias does teach
using the selected instance of the ML algorithm in a traffic planning method for controlling a plurality of vehicles (Matias, page 8, paragraph 0025, “Finally, the generalization error of each of these 3 models is compared with the one obtained in step 108, and the best one is selected” where “Embodiments automate the real-time vehicle dispatching on transit systems (with autonomous vehicles)” (Matias, page 10, paragraph 0038).)
Cao and Matias are analogous to the claimed invention because they teach tuning hyperparameters in a machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to use the instance in a traffic planning method like in Matias. Doing so “optimize[s] operational key performance indicators (KPIs) such as Excess Waiting Time (EWT) and On-Time Adherence (EWT)” (Matias, page 10, paragraph 0037).
Cao in view of Matias does not teach, but Petroff does teach
wherein each vehicle occupies one node in a shared set of planning nodes and is movable to other nodes along predefined edges between pairs of the nodes in accordance with a finite set of motion commands (Petroff, page 4, Fig. 2,
PNG
media_image2.png
942
842
media_image2.png
Greyscale
And “Thus, in the embodiment shown in FIGS. 1 and 2, the work area 8 comprises a plurality of source locations S1, S2 and a plurality of destination locations D1, D2 along paths, routes or legs of a round trip. The vehicles 6 travel to and from the source locations S1, S2 and the destination locations D1, D2” (Petroff, page 8, column 4, lines 10-15) where “The linear programming of the vehicle dispatch system 1 or method 30 generates the schedule, which may be a simple, optimal abstract schedule. An example of the schedule that defines the number of trips to be traveled along each path between source locations S1, S2 and destination locations D1, D2 is given in table 1 [see below]
PNG
media_image3.png
252
542
media_image3.png
Greyscale
(Petroff, page 10, column 7, lines 5-22). Examiner notes that the source locations and the destination locations are the planning nodes, the routes along are the predefined edges, and the schedule is the finite set of motion commands.) ,
wherein the input state indicates the planning nodes occupied by the vehicles and the output corresponds to a sequence of motion control commands to be applied to the vehicles (Petroff, page 1, abstract, “a method for dispatching a plurality of vehicles operating in a work area among a plurality of destination locations and a plurality of source locations includes … utilizing a reinforcement learning algorithm that takes in the schedule as input and cycles through possible environmental states that could occur within the schedule by choosing one possible action for each possible environmental state and by observing the reward obtained by taking the action at each possible environmental state, developing a policy for each possible environmental state, and providing instructions to follow an action associated with the policy.” Examiner notes that the input state is the schedule and the instructions are the motion control command to be applied to the vehicles. ).
Cao in view of Matias and Petroff are analogous to the claimed invention because they teach fleet dispatching using machine learning techniques. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Cao in view of Matias to use planning nodes and edges with a finite set of motions. Doing so “advantageously provides vehicle dispatch method 30 to communicate with many vehicles 6 operating among and between the source locations S1, S2 and destination locations D1, D2 to achieve one or more goals or objectives, e.g., maximizing the amount of material hauled, minimizing delivery time, etc.” (Petroff, page 8, column 4, lines 61-66).
Regarding claim 9, Cao in view of Matias and Petroff teach the method of claim 8. Matias further teaches
wherein the vehicles are autonomous vehicles (Matias, page 10, paragraph 0037, “FIG. 4 illustrates a real-time vehicle dispatching transit system using autonomous vehicles.”)
Cao and Matias are analogous to the claimed invention because they teach tuning hyperparameters in a machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to use autonomous vehicles in a traffic planning method like in Matias. Doing so “optimize[s] operational key performance indicators (KPIs) such as Excess Waiting Time (EWT) and On-Time Adherence (EWT)” (Matias, page 10, paragraph 0037).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao in view of Petroff.
Regarding claim 14, Cao teaches the method of claim 1. Cao does not teach, but Petroff does teach
wherein the technical system is a vehicle fleet and the quality metric is a driving economy indicator (Petroff, page 12, column 12, lines 19-21, “Tests were also performed to evaluate disturbances in the reinforcement learning’s simulation represented as an entropy value. The larger the entropy value, the more disturbances occur” where “Vehicle dispatching encounters many disturbances. For example, vehicles 6 break down, roads go out of operation, source and destination locations S1, S2, D1, D2 go out of operation due to breakdown or changes in location, etc.” (Petroff, page 11, column 10, lines 26-29) and “the invention may be adapted to use in many diverse applications such as…dispatching and fleet management of police and emergency vehicles,… commercial or government vehicle fleets” (Petroff, page 13, lines 4-14). Examiner notes that the driving economy indicator is the entropy value. ).
Cao and Petroff are analogous to the claimed invention because they teach optimizing parameters using machine learning techniques. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Cao to have the quality metric as a driving economy indicator. Doing so “is most advantageous in that it addresses these real-world problems during simulation. Thus the vehicle dispatching system 1 or method 30 may develop the best policy π to follow for a given environmental state of the work area 8 and is well prepared for real-world events when they occur in the work area 8.” (Petroff, page 8, column 4, lines 61-66).
Response to Arguments
The claim objections have been overcome in light of the instant amendments.
The previous 112(b) rejection on claim 1 has been overcome in light of the instant amendments. Examiner notes that new 112(b) rejections have been made in light of the instant amendments.
On pages 6-7, Applicant argues:
At least paragraph [0037] of the specification provides the basis for the "repeated or continued training of the ML instances not selected for continued use in controlling the technical system. Paragraph [0037] explicitly states that:
"[a]n optional further functionality of the selection module 404 is to generate updated values of the hyperparameter values in view of the statistics of the quality metric. The updated values of the hyperparameter values may be used to provide a new set of instances of the ML algorithm, and training may be performed in accordance with these. It is noted that the option of taking the hyperparameter value hio from the best-performing instance of the ML algorithm P and conceptually assigning it to the primary instance is normally not available. Indeed, a hyperparameter controls the learning process of an instance the ML algorithm and might have no influence on that instance's decision-making unless training is re-run or resumed."
From this paragraph alone, it is clear that the specification envisages that the ML instances, in particularly those for whom the hyperparameters are updated to more resemble those of the "primary instance" (i.e. the instance selected for continued use in controlling the technical system" are to be re-run (i.e. repeated) or resumed (i.e. continued).
Regarding the Applicant’s argument that paragraph 0037 supports the limitation of “repeated or continued training of the ML instances not selected for continued use in controlling the technical system”, Examiner respectfully disagrees. Specifically, Examiner notes that paragraph 0037 has no recitation of instances that are not selected. Further, because paragraph 0037 has no recitation of instance that are not selected, paragraph 0037 does not recite continuing training for instances not selected.
On pages 8-9, Applicant argues:
As amended, claim 1 requires that a plurality of instances of the ML algorithm be provided and trained, in parallel, using different values of at least one hyperparameter selected from an enumerated set of technical parameters, namely learning rate, discount factor, a parameter affecting convergence rate, or a probability of fallback to taking a random action, and further requires that, for at least one instance not selected for continued use in controlling the technical system, training of that instance be repeated or continued using a modified hyperparameter value, after which the mapping and evaluating steps are themselves repeated. Training and retraining of a machine-learning algorithm instance necessarily involves iterative numerical computation whereby the internal, learned parameters of the model are adjusted based on input data and the modified hyperparameter value, which cannot practically be performed in the human mind or with pen and paper, particularly where a plurality of such instances must be trained concurrently, in parallel, while another, already-trained instance is simultaneously and continuously generating real-time control signals for a technical system. Claim 1, considered as a whole, is therefore not directed to a mental process, and Step 2A, Prong One is not satisfied.
Regarding the Applicant’s argument that claim 1 is not directed to a mental process, Examiner respectfully disagrees. Specifically, Examiner respectfully notes that “mapping each input state to a plurality of outputs … wherein the outputs correspond to control signals of the technical system,” “evaluating a predefined quality metric for the outputs, wherein the quality metric relates to a quantity for the technical system,” “on the basis of statistics of the quality metric for the outputs, selecting at least one instance of the ML algorithm for continued use in controlling the technical system,” “wherein said selecting includes modifying, on said basis of statistics, the value of the at least one hyperparameters used in the instances of the ML algorithm not selected for continued use in controlling the technical system to be more similar to the at least one hyperparameters of the at least one instance selected for continued use in controlling the technical system” and “repeating the mapping and evaluating steps” are mental processes as described above in the 101 rejection.
Examiner further notes that “providing, in parallel, a plurality of instances of the ML algorithm trained using different values of the at least one hyperparameters” is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)) and “repeating or continuing a training of the instances of ML algorithm not selected for continued use in the technical system using the modified at least one parameter” is the well understood, routine, and conventional activity of iteratively training (Metzler et al. (US 2021/0125108 A1), page 11, paragraph 0051, “A classifier is trained using a conventional iterative machine learning training process that determines trained weights for each result list position. Based on initial weights assigned to each result list position, the iterative process attempts to find optimal weights.”) which cannot provide significantly more than the abstract idea.
On pages 9-10, Applicant argues:
MPEP § 2106.05(f) addresses claims that merely recite implementing an abstract idea on a generic computer, or using a computer as a tool to perform the abstract idea. That is not the case here. The steps identified as reciting an abstract idea are mapping each input state to a plurality of outputs, evaluating a predefined quality metric for the outputs, selecting at least one instance of the ML algorithm for continued use, and modifying the hyperparameter values of the non-selected instances. The additional element of repeating or continuing training of the non- selected instances does not correspond to performing any of these mapping, evaluating, selecting, or modifying steps using a computer; it is a separate and additional technical operation, namely the training or retraining of one or more ML algorithm instances using their respective modified hyperparameter values. Because claim 1 is not structured such that the additional element merely performs the alleged abstract-idea steps via a computer, the "apply it on a computer" rationale of MPEP § 2106.05(f) does not apply, and the same reasoning applies with equal force to the corresponding rationale advanced under Step 2B, below.
Regarding the Applicant’s argument that the additional element of repeating or continuing training is a technical operation that integrates the abstract idea into a practical application, Examiner respectfully disagrees. Specifically, Examiner notes that this limitation of “repeating or continuing a training of the instances of ML algorithm not selected for continued use in the technical system using the modified at least one parameter” does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of iteratively training. Further this limitation is the well understood, routine, and conventional activity of iteratively training (Metzler et al. (US 2021/0125108 A1), page 11, paragraph 0051, “A classifier is trained using a conventional iterative machine learning training process that determines trained weights for each result list position. Based on initial weights assigned to each result list position, the iterative process attempts to find optimal weights.”).
On pages 10-11, Applicant argues:
The claimed retraining is, moreover, integrated into a specific technical process for controlling a technical system. While at least one instance of the ML algorithm continues, uninterrupted, to generate control signals for the technical system, the remaining, non-selected instances are concurrently retrained, using hyperparameter values modified to be more similar to those of the currently-selected instance, and continue to receive the same real-time input states describing the technical system as the currently-selected instance. As explained in paragraph [0035] of the specification, subjecting all instances, selected and non-selected alike, to the same input signals allows the non-selected instances to base their decision-making on the same, current view of the state of the technical system as the instance currently in control, such that a subsequent hand-off of control to a previously non-selected instance proceeds more smoothly and with reduced risk of abrupt changes to how the technical system is controlled. Paragraph [0015] of the specification further explains that training multiple, differently-parameterized instances of the ML algorithm in this manner allows for more efficient exploration of the hyperparameter domain than single-sample optimization techniques, such as gradient descent, which risk becoming trapped in local minima or maxima. These are concrete, technical improvements to the reliability and efficiency of ML-based control of a technical system, integrated directly into the steps of claim 1.
For the same reasons, the characterization of paragraphs [0014 ], [0015], and [003 5] of the specification as merely asserting an improvement in a conclusory manner is respectfully traversed. Paragraph [0015] explains specifically why executing multiple instances of the ML algorithm, each configured with different hyperparameter values, provides a more efficient search of the hyperparameter domain than conventional single-sample gradient-descent approaches. Paragraph [0035] explains specifically how supplying the same input states to all instances, selected and non-selected, enables a smoother transition of control among instances. This is precisely the kind of specific, technical explanation of how and why an improvement is achieved that MPEP § 2106.04( d)(l) requires, and it is not a bare, unsupported assertion. Because claim 1, as amended, now expressly requires that the plurality of ML algorithm instances be provided and trained in parallel, the technical benefit described in the specification, namely more efficient, concurrent exploration of the hyperparameter domain coupled with continuous readiness of non-selected instances to assume control, is achieved directly by the claimed combination of steps.
Regarding the Applicant’s argument that these elements provide an improvement, Examiner respectfully disagrees. Specifically, Examiner notes that the Applicant provides a bare assertion of an improvement without the detail necessary to be apparent to one of ordinary skill in the art and, thus, cannot provide an improvement (MPEP 2106.04(d)(1)). More specifically, Examiner notes that not enough detail is provided within paragraph 0014 and 0015 to provide an improvement. Examiner further notes that the hot standby mode recited in paragraph 0035 is not recited in the claims and thus cannot provide an improvement because the claims do not reflect the improvement set forth in the specification (MPEP 2106.04(d)(1)).
On pages 11-12, Applicant argues:
For the reasons set forth above, the additional element of repeating or continuing training of the non-selected ML algorithm instances is not a well-understood, routine, or conventional computer function. The Office Action identifies no evidence that it was well-understood, routine, or conventional, as of the effective filing date, to concurrently maintain a plurality of ML algorithm instances in which one instance generates real-time control signals for a technical system while the remaining instances are continuously retrained, using hyperparameter values modified to track those of the selected instance, based on the same real-time input data as the selected instance. To the contrary, and as discussed above, the specification explains that this specific, ordered combination of elements provides a concrete improvement over prior gradient descent- based tuning approaches. Considered individually and as an ordered combination, the additional elements of claim I therefore amount to significantly more than the abstract idea itself, and the rejection is not sustainable under Step 2B.
Regarding the Applicant’s argument that the additional element of repeating or continuing training of the non-selected ML algorithm instances is not well-understood, routine, or conventional, Examiner respectfully disagrees. Specifically, “repeating or continuing a training of the instances of ML algorithm not selected for continued use in the technical system using the modified at least one parameter” is the well understood, routine, and conventional activity of iteratively training (Metzler et al. (US 2021/0125108 A1), page 11, paragraph 0051, “A classifier is trained using a conventional iterative machine learning training process that determines trained weights for each result list position. Based on initial weights assigned to each result list position, the iterative process attempts to find optimal weights.”). Examiner further notes that MPEP 2106.05(d) also recites that performing repetitive calculations are well-understood, routine, conventional as well.
On page 13, Applicant argues:
As amended, claim I recites that the at least one hyperparameter is one of learning rate, discount factor, a parameter affecting convergence rate, or a probability of fall back to taking a random action, each of which is a parameter that controls the training or learning process of the ML algorithm itself Cao's disclosure relied upon by the Examiner, by contrast, is directed to "parameters" of a distributed-training configuration, such as a number of parameter servers, a number of worker nodes, and a resource budget, which define the computational resources allocated to execute a training job rather than any aspect of the ML algorithm's learning process. In light of the amendments to claim 1, the Office Action's interpretation that Cao's distributed training- configuration "parameters" correspond to the claimed "hyperparameters" is not well founded.
Regarding the Applicant’s argument that Cao’s parameters do not teach the claim hyperparameters, Examiner respectfully disagrees. Specifically, as cited above, “hyperparameter or deep learning parameter optimization/tuning involves choosing a set of optimal hyperparameter values for an ML algorithm (i.e., a parameter whose value is used to control the learning process)” (Cao, page 11, paragraph 0030).
Additionally, Examiner notes that according paragraph 0010 of the specification, “a ‘hyperparameter’ is a parameter controlling the ML algorithm’s learning process, including deep learning or reinforcement learning.” As such, Cao’s hyperparameter and/or parameters teach the hyperparameters according to the specification.
On pages 13-14, Applicant argues:
In addition, the Office Action's interpretation that Cao's Bayesian optimizer and training jobs together form a single "instance of an ML algorithm" remains unsupported, as Cao does not disclose parallel execution of multiple instances of the Bayesian optimizer or of any ML algorithm.
Regarding the Applicant’s argument that Cao does not disclose parallel execution, Examiner respectfully disagrees. Specifically, Cao supports this in paragraph 0024 with “jobs can be parallelized in the distributed configuration to potentially reduce the time needed to accomplish model exploration” (Cao, page 11, paragraph 0024).
On page 14, Applicant argues:
The Office Action further mischaracterizes claim l's requirement that the outputs from the instances of the ML algorithm correspond to control signals of the technical system. The Office Action maps this limitation to Cao's "specific parameter values for achieving shortest possible job completion time," i.e., the optimal resource configuration output by Cao's Bayesian optimizer. Cao discloses, however, that this resource-configuration optimization, directed to minimizing job completion time, is an entirely separate process from Cao's hyperparameter tuning loop, in which hyperparameters are iteratively adjusted based on model quality alone, independent of any resource-configuration output. As illustrated in Cao's Figure 5, a hyperparameter is tried and a job is executed using a given resource configuration; model quality is then determined for the hyperparameters used; and a new hyperparameter is selected only if that quality does not satisfy a termination condition, all without reference to, or use of, the resource-configuration output. Cao's optimal-resource-configuration output therefore is not, and cannot reasonably be construed to be, an output "from the instances of the ML algorithm" that "correspond[s] to control signals of the technical system," nor is it used to select an ML algorithm instance or to modify its hyperparameters, as claim 1 requires.
Regarding the Applicant’s argument that Cao does not teach “wherein the outputs from the instances of the ML algorithm correspond to control signals of the technical system”, Examiner respectfully disagrees. Specifically, Examiner notes that under the broadest reasonable interpretation, this limitation encompasses outputs that are derived in some way from the instances of the ML algorithm because of the broadness of the word “from.” Thus, Cao teaches this via the specific parameter values for achieving shortest possible job completion time being the outputs and the possible parameter values being the input state. Additionally, the specific parameter values are control signals since they control the learning process.
On page 15, Applicant argues:
Claim 1 as amended also forecloses the Office Action's prior treatment of related arguments as moot. the immediately preceding Office Action declined to read the requirement that hyperparameters be modified "on the basis of statistics of the quality metric for the outputs" together with the separately recited step of repeating or continuing training of non-selected instances, reasoning that the two clauses were not required to be read together. Claim 1 as now amended eliminates any such ambiguity: the claim expressly recites that said selecting includes modifying, on said basis of statistics, the value of the at least one hyperparameter used in the instances not selected to be more similar to that of the instance selected, repeating or continuing training of the non-selected instances using the modified value, and repeating the mapping and evaluating steps. Cao discloses no such linkage. As discussed above, Cao's hyperparameter adjustments are made in dependence on model quality alone, and are wholly independent of, and unconnected to, any statistics relating to Cao's resource-configuration outputs, i.e., the very "outputs" the Office Action relies upon for the evaluating and selecting limitations of claim 1. Accordingly, even accepting the Office Action's own mapping of Cao's disclosure, Cao fails to disclose modifying hyperparameters on the basis of statistics of the quality metric for the outputs, as that requirement must now be read in claim 1 as amended.
Regarding the Applicant’s argument that the Examiner did not consider the limitation “on the basis of statistics…”, Examiner respectfully disagrees. Specifically, Examiner notes that the basis of statistics as shown above is selecting the best times of the configurations which are reliant on hyperparameters. As such, Cao discusses in paragraphs 0063-0064, a configuration and its inherent hyperparameters is selected, trained, and modified in regards to which configuration has the shortest, or best, training time.
On pages 15-16, Applicant argues:
Claim 1 as amended further recites that the plurality of instances of the ML algorithm are provided, and trained, in parallel. In responding to Applicant's prior remarks, the Office Action observed that claim 1 did not include language claiming parallel processing. That is no longer the case. Cao's disclosure relied upon is, however, expressly sequential: Cao's Figure 5 depicts a single, iterative hyperparameter-search loop in which a hyperparameter is tried, a job is executed, model quality is determined, and, if a termination condition is not satisfied, a new hyperparameter is selected and the loop repeats. Cao does not disclose that a plurality of instances of the ML algorithm, or of Cao's Bayesian optimizer, are provided or executed in parallel, and this silence confirms that Cao does not disclose this additional limitation of claim 1 as amended.
Regarding the Applicant’s argument that Cao does not disclose parallel execution, Examiner respectfully disagrees. Specifically, Cao supports this in paragraph 0024 with “jobs can be parallelized in the distributed configuration to potentially reduce the time needed to accomplish model exploration” (Cao, page 11, paragraph 0024).
On page 16, Applicant argues:
The Office Action's mapping of Cao's disclosure to the limitations of claim 1 is, in any event, technically untenable. Cao's Bayesian optimizer receives, as its input, candidate resource configurations, such as combinations of a number of parameter servers, a number of worker nodes, and memory, CPU, or disk allocations, and outputs a subset of such candidates considered most likely to minimize job completion time. Substituting a hyperparameter, as recited in claim 1 as amended, for the resource-configuration candidates that Cao's Bayesian optimizer is designed to evaluate would render Cao's disclosed solution unable to perform its stated function of identifying an optimal resource configuration for executing training jobs. This confirms that Cao's resource-configuration parameters and Cao's hyperparameters are, and must remain, distinct in Cao's own disclosure, further undermining the Office Action's contrary mapping.
Regarding the Applicant’s argument that Cao’s parameters do not teach the claim hyperparameters, Examiner respectfully disagrees. Specifically, as cited above, “hyperparameter or deep learning parameter optimization/tuning involves choosing a set of optimal hyperparameter values for an ML algorithm (i.e., a parameter whose value is used to control the learning process)” (Cao, page 11, paragraph 0030).
Additionally, Examiner notes that according paragraph 0010 of the specification, “a ‘hyperparameter’ is a parameter controlling the ML algorithm’s learning process, including deep learning or reinforcement learning.” As such, Cao’s hyperparameter and/or parameters teach the hyperparameters according to the specification.
On pages 16-17, Applicant argues:
More broadly, the rejection does not interpret the claims in a manner consistent with the technical disclosure of either Cao or the present application. Cao is directed to identifying an optimal allocation of computational resources for executing hyperparameter-tuning jobs, so as to minimize the completion time for each such job. Cao is not directed to improving the hyperparameter-tuning process itself, i.e., the process of executing an instance of an ML algorithm using given hyperparameters, evaluating the resulting quality or accuracy of the ML algorithm, determining how to modify the hyperparameters to obtain a better result, executing a new instance of the ML algorithm with the modified hyperparameters to confirm whether the quality improved, and repeating that process until a satisfactory result is achieved.
Embodiments of the invention, by contrast, are directed to performing hyperparameter tuning of multiple instances of the same ML algorithm simultaneously, by feeding the same input states to each instance, evaluating the resulting performance of each instance, selecting one instance to control the technical system, and adjusting the hyperparameters of the other instances to be more similar to those of the selected instance while continuing or repeating training of those non-selected instances using their modified hyperparameters. As explained in the specification, this technique allows the search space of possible hyperparameters to be explored more efficiently, in a manner analogous to a genetic algorithm, and ensures that each instance maintains a similar view of the state of the technical system, such that a subsequent hand-off to a non-selected instance, in the event of failure of the currently-selected instance, proceeds more smoothly. Neither this technique, nor the problem it addresses, is disclosed or suggested in Cao.
Regarding the Applicant’s argument that Cao does not teach the limitations of claim 1, Examiner respectfully disagrees. Specifically, Examiner notes that the claim does not recite “improving the hyperparameter-tuning process itself.” Thus under BRI, Cao, which discloses analyzing parameter values and outputting specific parameter values for achieving shortest possible job completion and executing the training jobs after hyperparameters are tuned or modified in box 524 to meet a threshold reads on claim 1 as shown in the above 102 rejection.
Examiner further notes that claim 1 does not recite “feeding the same input states to each instance.” Cao does teach however “evaluating a predefined quality metric for the outputs, wherein the quality metric relates to a quantity for the technical system, and on the basis of statistics of the quality metric for the outputs, selecting at least one instance of the ML algorithm for continued use in controlling the technical system wherein said selecting includes modifying, on said basis of statistics, the value of the at least one hyperparameters used in the instances of the ML algorithm not selected for continued use in controlling the technical system to be more similar to the at least one hyperparameters of the at least one instance selected for continued use in controlling the technical system, repeating or continuing a training of the instances of the ML algorithm not selected for continued use in the technical system using the modified at least one hyperparameters value” as can be seen above in the 102 rejection section.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Yi et al. (“An Automated Hyperparameter Search-Based Deep Learning Model for Highway Traffic Prediction”) also describes tuning hyperparameters in a traffic planning environment.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN R LAU whose telephone number is (571)272-1429. The examiner can normally be reached Monday - Thursday: 8:00 am - 6:00 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.L./ Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148
Prosecution Insights