DETAILED ACTION
This action is in response to the amendment filed 09/25/2025. Claims 1-14 are pending and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 1 is objected to because of the following informalities:
Regarding claim 1, in lines 2-3, “wherein the ML algorithm which is dependent on one or more hyperparameters” should read “wherein the ML algorithm is dependent on one or more hyperparameters”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 1 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Examiner notes that lines 13-17 of claim 1 recite “and for at least one instance of the ML algorithm not selected for continued use in controlling the technical system, repeating or continuing training of the instance after modifying its values of the hyperparameters to be more similar to those of the at least one instance of the ML algorithm selected for continued use in controlling the technical system”. Examiner notes that the specification does not reasonably convey to one skilled in the relevant art continuing training of an instance not selected for continued use. Paragraph 0035 of the instant specification describes remaining instances operating in hot-standby mode and mimicking a primary instance. Paragraph 0035 does not, however describe continued use or selecting an instance. Examiner respectfully notes that paragraph 0034 describes selecting an instance to replace the primary instance but does not disclose information regarding the continued training of an instance that is not selected.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1-14 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation "the instance" in the third to last line. There is insufficient antecedent basis for this limitation in the claim. It is unclear as to whether this instance is the instance not selected for continued use or if this instance is the instance selected for continued use. For purposes of examination, Examiner has interpreted the instance to be the instance not selected for continued use.
Regarding claims 2-14, claims 2-14 are rejected for at least the same reasons as claim 1 since claims 2-14 depend on claim 1.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-2 and 4-14 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Subject Matter Eligibility Analysis Step 1:
Claim 1 recites a method and is thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 1 recites
mapping each input state to a plurality of outputs … wherein the outputs correspond to control signals of the technical system (This limitation is a mental process as it encompasses a human mentally mapping each input to outputs.)
evaluating a predefined quality metric for the outputs, wherein the quality metric relates to a quantity for the technical system (This limitation is a mental process as it encompasses a human mentally evaluating a quality metric.)
on the basis of statistics of the quality metric for the outputs, selecting at least one instance of the ML algorithm for continued use in controlling the technical system. (This limitation is a mental process as it encompasses a human mentally selecting an instance of the algorithm.)
modifying its values of the hyperparameters to be more similar to those of the at least one instance of the ML algorithm selected for continued use in controlling the technical system (This limitation is a mental process as it encompasses a human mentally modifying hyperparameters.)
Therefore, claim 1 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 1 further recites additional elements of
a computer-implemented method of managing a machine-learning, ML, algorithm for controlling a technical system, wherein training of the ML algorithm is dependent on one or more hyperparameters (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
providing a plurality of instances of the ML algorithm trained using different values of the hyperparameters (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
obtaining a plurality of input states related to the technical system (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
by providing each input state as input to the instances of the ML algorithm (This element does not integrate the abstract idea into a practical application because it recites insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)).)
for at least one instance of the ML algorithm not selected for continued use in controlling the technical system, repeating or continuing training of the instance after modifying its values of the hyperparameters … (This element does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 1 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 1 do not provide significantly more than the abstract idea itself, taken alone and in combination because
a computer-implemented method of managing a machine-learning, ML, algorithm for controlling a technical system, wherein training of the ML algorithm is dependent on one or more hyperparameters specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
providing a plurality of instances of the ML algorithm trained using different values of the hyperparameters is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
obtaining a plurality of input states related to the technical system is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
by providing each input state as input to the instances of the ML algorithm is the well understood, routine, and conventional activity of “transmitting or receiving data over a network” (see MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network)).
for at least one instance of the ML algorithm not selected for continued use in controlling the technical system, repeating or continuing training of the instance after modifying its values of the hyperparameters… uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 1 is subject-matter ineligible.
Regarding Claim 2:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 2 recites the same abstract ideas as claim 1. Therefore, claim 2 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 2 further recites additional elements of
wherein the hyperparameters include one or more of: learning rate, discount factor, parameters affecting convergence rate, probability of fallback to taking a random action. (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 2 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 2 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the hyperparameters include one or more of: learning rate, discount factor, parameters affecting convergence rate, probability of fallback to taking a random action specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 2 is subject-matter ineligible.
Regarding Claim 4:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 4 recites
wherein said modifying includes enforcing a minimum spread of the values of the hyperparameters used in the instances of the ML algorithm (This limitation is a mental process as it encompasses a human mentally modifying the hyperparameters.)
Therefore, claim 4 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 4 does not further recite any additional elements. Therefore, claim 4 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 4 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 4 is subject-matter ineligible.
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 5 recites the same abstract ideas as claim 1. Therefore, claim 5 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 5 further recites additional elements of
wherein the quality metric is productivity of the technical system (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 5 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 5 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the quality metric is productivity of the technical system specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 5 is subject-matter ineligible.
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 6 recites
wherein said selecting is based on comparing averages of the quality metric across the instances of the ML algorithms (This limitation is a mental process as it encompasses a human selecting the instance of the ML algorithm based on comparing averages.)
Therefore, claim 6 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 6 further does not recite any additional elements. Therefore, claim 6 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements to provide significantly more than the abstract idea itself, taken alone and in combination, claim 6 is subject-matter ineligible.
Regarding Claim 7:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 7 recites
detecting an execution failure of the at least one instance selected to control the technical system, (This limitation is a mental process as it encompasses a human mentally identifying an execution failure.)
in response thereto, replacing the failed at least one instance with the at least one instance trained using the modified values of the hyperparameters. (This limitation is a mental process as it encompasses a human mentally replacing one instance with another.
Therefore, claim 7 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 7 does not further recite any additional elements. Therefore, claim 7 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
Since there are no additional elements, claim 7 does not provide significantly more than the abstract idea itself, taken alone and in combination. Therefore, claim 7 is subject-matter ineligible.
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 8 recites
wherein the input state indicates the planning nodes occupied by the vehicles and the output corresponds to a sequence of motion control commands to be applied to the vehicles. (This limitation is a mental process as it further describes the abstract idea of mapping each input state to output states.)
Therefore, claim 8 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 8 further recites additional elements of
using the selected instance of the ML algorithm in a traffic planning method for controlling a plurality of vehicles (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
wherein each vehicle occupies one node in a shared set of planning nodes and is movable to other nodes along predefined edges between pairs of the nodes in accordance with a finite set of motion commands (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 8 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim # do not provide significantly more than the abstract idea itself, taken alone and in combination because
using the selected instance of the ML algorithm in a traffic planning method for controlling a plurality of vehicles specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
wherein each vehicle occupies one node in a shared set of planning nodes and is movable to other nodes along predefined edges between pairs of the nodes in accordance with a finite set of motion commands specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 8 is subject-matter ineligible.
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 9 recites the same abstract ideas as claim 1. Therefore, claim 9 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 9 further recites additional elements of
wherein the vehicles are autonomous vehicles. (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 9 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 9 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the vehicles are autonomous vehicles specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 9 is subject-matter ineligible.
Regarding Claim 10:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 10 recites the same abstract ideas as claim 1. Therefore, claim 10 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 10 further recites additional elements of
A device configured to manage a machine-learning, ML, algorithm which is dependent on one or more hyperparameters, the device comprising processing circuitry configured to execute the method of claim 1. (This element does not integrate the abstract idea into a practical application because amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 10 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 10 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A device configured to manage a machine-learning, ML, algorithm which is dependent on one or more hyperparameters, the device comprising processing circuitry configured to execute the method of claim 1 uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 10 is subject-matter ineligible.
Regarding Claim 11:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 11 recites the same abstract ideas as claim 1. Therefore, claim 11 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 11 further recites additional elements of
an interface configured to control a technical system. (This element does not integrate the abstract idea into a practical application because amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 11 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 11 do not provide significantly more than the abstract idea itself, taken alone and in combination because
an interface configured to control a technical system uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 11 is subject-matter ineligible.
Regarding Claim 12:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 12 recites the same abstract ideas as claim 1. Therefore, claim 12 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 12 further recites additional elements of
A non-transitory computer readable medium storing a computer program comprising instructions which, when executed, cause a processor to execute the method of claim 1. (This element does not integrate the abstract idea into a practical application because amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
Therefore, claim 12 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 12 do not provide significantly more than the abstract idea itself, taken alone and in combination because
A non-transitory computer readable medium storing a computer program comprising instructions which, when executed, cause a processor to execute the method of claim 1 uses a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)).
Therefore, claim 12 is subject-matter ineligible.
Regarding Claim 13:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 13 recites the same abstract ideas as claim 1. Therefore, claim 13 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 13 further recites additional elements of
wherein the quality metric is uptime of the technical system (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 13 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 13 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the quality metric is uptime of the technical system specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 13 is subject-matter ineligible.
Regarding Claim 14:
Subject Matter Eligibility Analysis Step 2A Prong 1:
Claim 14 recites the same abstract ideas as claim 1. Therefore, claim 14 recites an abstract idea.
Subject Matter Eligibility Analysis Step 2A Prong 2:
Claim 14 further recites additional elements of
wherein the technical system is a vehicle fleet and the quality metric is a driving economy indicator (This element does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)).)
Therefore, claim 13 is not integrated into a practical application.
Subject Matter Eligibility Analysis Step 2B:
The additional elements of claim 13 do not provide significantly more than the abstract idea itself, taken alone and in combination because
wherein the technical system is a vehicle fleet and the quality metric is a driving economy indicator specifies a particular technological environment to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(h)).
Therefore, claim 13 is subject-matter ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4-5 and 10-13 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Cao et al. (US 2022/0292303) (hereafter referred to as Cao).
Regarding claim 1, Cao teaches
A computer-implemented method of managing a machine-learning, ML, algorithm for controlling a technical system, wherein training of the ML algorithm is dependent on one or more hyperparameters, (Cao, page 10, paragraph 0022, “The hyperparameter tuning involves selection of the right set of hyperparameters so that an ML model can be trained with efficiency and results in an accurate production ML model” where “hyperparameter or deep learning parameter optimization/tuning involves choosing a set of optimal hyperparameter values for an ML algorithm (i.e., a parameter whose value is used to control the learning process)” (Cao, page 11, paragraph 0030). ) the method comprising:
providing a plurality of instances of the ML algorithm trained using different values of the hyperparameters (Cao, page 10-11, paragraph 0023, “Hyperparameter tuning typically involves a large number of exploratory experiments to test different combinations of possible values of various hyperparameters. The hyperparameter tuning can involve training an ML model with a small amount of data (e.g., small training jobs) to determine how well the model will work.” Examiner notes that the jobs are the instances of the ML algorithm);
obtaining a plurality of input states related to the technical system (Cao, page 14, paragraph 0049, “The resource configuration generator 302 can take as input, various parameters that specify how to generate the DT [distributed training]-configurations.” Examiner notes that the parameters are the input states.);
mapping each input state to a plurality of outputs, by providing each input state as input to the instances of the ML algorithm, wherein the outputs from the instances of the ML algorithm correspond to control signals of the technical system (Cao, page 15, paragraph 0051, “The Bayesian optimization can be performed by optimizer 306 of FIG. 3 to determine, based on prior executions of jobs using possible DT-configurations, what remaining possible DT-configurations may correspond to an optimal resource configuration. Although Bayesian optimization is used, other optimization techniques are possible. Using Bayesian optimization, the confidence gained through additional iterations (in this case executing training jobs in accordance with possible resource configurations) results in being able to better narrow down possible resource configurations. Generally, Bayesian optimization analyzes possible parameter values (in this case, DT-configuration candidates), and gradually outputs specific parameter values (again, in this case, specific DT-configuration candidates determined as being optimal) for achieving shortest possible job completion time to try/test/evaluate. That is, a job completion time can be determined based on a particular DT-configuration, which can be fed back into the Bayesian optimization process that will assess its effectiveness based on historical information, and a next DT-configuration candidate to try/test/evaluate can be output” where “hyperparameter or deep learning parameter optimization/tuning involves choosing a set of optimal hyperparameter values for an ML algorithm (i.e., a parameter whose value is used to control the learning process)” (Cao, page 11, paragraph 0030). Examiner notes analyzing parameter values and outputting specific parameter values is the act of mapping where the specific parameter values for achieving shortest possible job completion time are the outputs and the possible parameter values is the input state. Examiner notes that the DT-configuration that has the possible parameters is fed or input into the Bayesian optimization process which is a job or instance. Additionally, the specific parameter values are control signals since they control the learning process. Examiner further notes that the job is the instance of the ML algorithm as the job is used to narrow down possible resource configurations.);
evaluating a predefined quality metric for the outputs, wherein the quality metric relates to a quantity for the technical system (Cao, page 15, paragraph 0056, “A current completion time associated with job performance using a current resource configuration candidate can be compared to a previous completion time associated with a previously tested resource configuration candidate. If the current completion time does not improve over the previous completion time by at least the threshold level of time difference, that current resource configuration candidate can be deemed "good enough," such that subsequent resource configuration candidate testing can stop.” Examiner notes that the threshold level of time difference is the predefined quality metric for the outputs. Examiner further notes that the time difference is a number and thus a quantity for the technical system.);
and on the basis of statistics of the quality metric for the outputs, selecting at least one instance of the ML algorithm for continued use in controlling the technical system (Cao, page 16, paragraph 0057, “The resource configuration candidate producing the best completion time in those 70 trials may be selected as the optimum resource configuration to use for the remaining jobs” where “different resource configurations may be used to execute different subsets of training jobs” (Cao, page 16, paragraph 0060). Examiner notes that the multiple jobs or instances are selected based on the best time as the selected resource configuration executes different jobs.);
and for at least one instance of the ML algorithm not selected for continued use in controlling the technical system, repeating or continuing training of the instance after modifying its values of the hyperparameters to be more similar to those of the at least one instance of the ML algorithm selected for continued use in controlling the technical system (Cao, page 16, paragraphs 0063-0064, “A resource configuration that results in the best performance metrics (e.g., shortest training time) compared to previous resource configurations can be selected as an optimal resource configuration. Alternatively, a resource configuration that satisfy a termination condition (e.g., completing within a predefined training time) can be selected as an optimal resource configuration. If the optimal resource configuration is found, remaining training jobs can be executed using the optimal resource configuration at 504. [0064] At 524, model quality can be determined for hyperparameters used to execute the training job. If the model quality does not satisfy a termination condition, a new hyperparameter can be selected. Hyperparameter tuning can continue with the new hyperparameter and the process can iterate until the hyperparameters that provide the best model quality are determined” and Cao, page 6, Figure 5,
PNG
media_image1.png
693
936
media_image1.png
Greyscale
Examiner notes that the remaining training jobs are the instances not selected. Examiner further notes that the remaining training jobs continue training at box 504. Box 504 executes the training jobs after hyperparameters are tuned or modified in box 524. Examiner further notes that by changing the hyperparameters to satisfy a termination condition, the values of the hyperparameters are more similar to the hyperparameters of the ML instance selected.)
Regarding claim 2, Cao teaches
The method of claim 1, wherein the hyperparameters include one or more of: learning rate, discount factor, parameters affecting convergence rate, probability of fallback to taking a random action (Cao, page 10, paragraph 0015-0020, “Some exemplary hyperparameters include: … Neural network: learning rate, number of epochs, size of mini-batch, etc.” Examiner notes that the hyperparameters include learning rate.).
Regarding claim 4, Cao teaches
The method of claim 3, wherein said modifying includes enforcing a minimum spread of the values of the hyperparameters used in the instances of the ML algorithm (Cao, page 11, paragraph 0031, “In order to optimize the model, the hyperparameters can be tuned. Tuning of the hyperparameters allows one or more values to be selected for use by/in the model.” Examiner notes that the minimum spread of modified hyperparameters is tuning the hyperparameters to optimize the model.).
Regarding claim 5, Cao teaches
The method of claim 1, wherein the quality metric is productivity of the technical system (Cao, page 15, paragraph 0056, “A current completion time associated with job performance using a current resource configuration candidate can be compared to a previous completion time associated with a previously tested resource configuration candidate. If the current completion time does not improve over the previous completion time by at least the threshold level of time difference, that current resource configuration candidate can be deemed "good enough," such that subsequent resource configuration candidate testing can stop.” Examiner notes that the threshold level of time difference is the predefined quality metric for the outputs. Examiner further notes that the completion time is the productivity.).
Regarding claim 10, Cao teaches
A device configured to manage a machine-learning, ML, algorithm which is dependent on one or more hyperparameters, the device comprising processing circuitry configured to execute the method of claim 1 (Cao, page 18, paragraph 0084, “Such instructions may be read into main memory 806 from another storage medium, such as storage device 810. Execution of the sequences of instructions contained in main memory 806 causes processor(s) 804 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions” where “the environment 200 provides a network environment for implementing machine learning models (Cao, page 12, paragraph 0034) and where “The hyperparameter tuning involves selection of the right set of hyperparameters so that an ML model can be trained with efficiency and results in an accurate production ML model” (Cao, page 10, paragraph 0022).).
Regarding claim 11, Cao teaches
The system of claim 10, further comprising an interface configured to control a technical system (Cao, page 12, paragraph 0035, “Each deployment cluster can have an associated application programming interface (API) server configured for dependent distribution to allocate large-scale processing clusters in the environment 200.”).
Regarding claim 12, Cao teaches
A non-transitory computer readable medium storing a computer program comprising instructions which, when executed, cause a processor to execute the method of claim 1 (Cao, page 17, paragraph 0070-0071, “In some embodiments, machine-readable storage medium 704 may be a non-transitory storage medium, where the term “non-transitory” does not encompass transitory propagating signals. As described in detail below, machine-readable storage medium 704 may be encoded with executable instructions, for example, instructions 706-714. [0071] Hardware processor 702 may execute instruction 706 to determine a plurality of computing resource configurations used to perform machine learning model training jobs.”).
Regarding claim 13, Cao teaches
The method of claim 1, wherein the quality metric is uptime of the technical system (Cao, page 15, paragraph 0056, “A current completion time associated with job performance using a current resource configuration candidate can be compared to a previous completion time associated with a previously tested resource configuration candidate. If the current completion time does not improve over the previous completion time by at least the threshold level of time difference, that current resource configuration candidate can be deemed "good enough," such that subsequent resource configuration candidate testing can stop.” Examiner notes that the threshold level of time difference is the predefined quality metric for the outputs. Examiner further notes that the completion time is the uptime.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 6-7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao in view of Marinakis et al. (US 2023/0378753 A1) (hereafter referred to as Marinakis).
Regarding claim 6, Cao teaches the method of claim 1. Cao does not teach, but Marinakis does teach
wherein said selecting is based on comparing averages of the quality metric across the instances of the ML algorithms (Marinakis, page 23, paragraph 0091-0092, “The architecture supervisor may select a machine learning model architecture that is identical to a machine learning model architecture that has previously been trained, and the method may comprise performing system simulations and evaluating a performance based on scenarios that have meanwhile been determined to cause other machine learning model architectures to underperform. [0092] The performance may be computed in accordance with a performance metric or several performance metrics” where “The performance metric(s) may be any one or any combination of: operation cost, energy not served, electricity price, system stability, available power transfer capacity, selectivity and dependability, controller stability, without being limited thereto” (Marinakis, page 23, paragraph 0100) and “The performance metric may be based on computed deviations between actions taken by the decision logic in the system simulations and actions known to be correct. [0104] The deviations can be computed in accordance with a norm. The actions known to be correct may be defined by an expert input (e.g., in supervised learning) or may be derived from historical data” (Marinakis, page 23, paragraph 0103-0104). Examiner notes that comparing averages of the quality metric is evaluating a performance that is based on computed deviations in accordance with a norm.).
Cao and Marinakis are analogous to the claimed invention because they teach updating hyperparameters in machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to compare averages like in Marinakis. Doing so “allow[s] the decision logic generation system to generate and alter scenarios according to the performance assessment” (Marinakis, page 54, paragraph 0890).
Regarding claim 7, Cao teaches the method of claim 1. Cao does not teach, but Marinakis does teach
The method of claim 1, further comprising detecting an execution failure of the at least one instance selected to control the technical system and, in response thereto, replacing the failed at least one instance with the at least one instance trained using the modified values of the hyperparameters (Marinakis, page 22, paragraph 0060-0061, “The two or more different machine learning model architectures may comprise artificial neural network architectures different from each other in a number of nodes and/or a number of layers, without being limited thereto. [0061] Generating the decision logic candidates may comprise selecting a machine learning model architecture; training a decision logic candidate having the machine learning model architecture until a first termination criterion is fulfilled, and storing performance information for the trained decision logic candidate; if a second termination criterion is not fulfilled, repeating the training and storing steps for a different decision logic candidate having a different machine learning model architecture” where “methods and system are provided that allow machine learning to be harnessed for generating a decision logic….[0014] Embodiments allow the development of power system operational planning, control and protection to be performed automatically” (Marinakis, page 20-21, paragraph 0013-0014) and “The termination control module 103 may trigger the decision logic generator 60 to select another decision logic architecture, e.g., by changing hyperparameters” (Marinakis, page 57, paragraph 0966). Examiner notes that the plurality of instances of the ML algorithm are the decision logic candidates, the primary instance is the a decision logic candidate and the technical system is the automatic power system operational planning, control, and protection.),
Cao and Marinakis are analogous to the claimed invention because they teach updating hyperparameters in machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to replace a failed instance like in Marinakis. Doing so “allow[s] the decision logic generation system to generate and alter scenarios according to the performance assessment” (Marinakis, page 54, paragraph 0890).
Claim(s) 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao in view of Moreira-Matias et al. (US 2019/0318248 A1) (hereafter referred to as Matias) in further view of Petroff (US 11,049,208 B2) (hereafter referred to as Petroff)
Regarding claim 8, Cao teaches the method of claim 1. Cao does not teach, but Matias does teach
using the selected instance of the ML algorithm in a traffic planning method for controlling a plurality of vehicles (Matias, page 8, paragraph 0025, “Finally, the generalization error of each of these 3 models is compared with the one obtained in step 108, and the best one is selected” where “Embodiments automate the real-time vehicle dispatching on transit systems (with autonomous vehicles)” (Matias, page 10, paragraph 0038).)
Cao and Matias are analogous to the claimed invention because they teach tuning hyperparameters in a machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to use the instance in a traffic planning method like in Matias. Doing so “optimize[s] operational key performance indicators (KPIs) such as Excess Waiting Time (EWT) and On-Time Adherence (EWT)” (Matias, page 10, paragraph 0037).
Cao in view of Matias does not teach, but Petroff does teach
wherein each vehicle occupies one node in a shared set of planning nodes and is movable to other nodes along predefined edges between pairs of the nodes in accordance with a finite set of motion commands (Petroff, page 4, Fig. 2,
PNG
media_image2.png
942
842
media_image2.png
Greyscale
And “Thus, in the embodiment shown in FIGS. 1 and 2, the work area 8 comprises a plurality of source locations S1, S2 and a plurality of destination locations D1, D2 along paths, routes or legs of a round trip. The vehicles 6 travel to and from the source locations S1, S2 and the destination locations D1, D2” (Petroff, page 8, column 4, lines 10-15) where “The linear programming of the vehicle dispatch system 1 or method 30 generates the schedule, which may be a simple, optimal abstract schedule. An example of the schedule that defines the number of trips to be traveled along each path between source locations S1, S2 and destination locations D1, D2 is given in table 1 [see below]
PNG
media_image3.png
252
542
media_image3.png
Greyscale
(Petroff, page 10, column 7, lines 5-22). Examiner notes that the source locations and the destination locations are the planning nodes, the routes along are the predefined edges, and the schedule is the finite set of motion commands.) ,
wherein the input state indicates the planning nodes occupied by the vehicles and the output corresponds to a sequence of motion control commands to be applied to the vehicles (Petroff, page 1, abstract, “a method for dispatching a plurality of vehicles operating in a work area among a plurality of destination locations and a plurality of source locations includes … utilizing a reinforcement learning algorithm that takes in the schedule as input and cycles through possible environmental states that could occur within the schedule by choosing one possible action for each possible environmental state and by observing the reward obtained by taking the action at each possible environmental state, developing a policy for each possible environmental state, and providing instructions to follow an action associated with the policy.” Examiner notes that the input state is the schedule and the instructions are the motion control command to be applied to the vehicles. ).
Cao in view of Matias and Petroff are analogous to the claimed invention because they teach fleet dispatching using machine learning techniques. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Cao in view of Matias to use planning nodes and edges with a finite set of motions. Doing so “advantageously provides vehicle dispatch method 30 to communicate with many vehicles 6 operating among and between the source locations S1, S2 and destination locations D1, D2 to achieve one or more goals or objectives, e.g., maximizing the amount of material hauled, minimizing delivery time, etc.” (Petroff, page 8, column 4, lines 61-66).
Regarding claim 9, Cao in view of Matias and Petroff teach the method of claim 8. Matias further teaches
wherein the vehicles are autonomous vehicles (Matias, page 10, paragraph 0037, “FIG. 4 illustrates a real-time vehicle dispatching transit system using autonomous vehicles.”)
Cao and Matias are analogous to the claimed invention because they teach tuning hyperparameters in a machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to use autonomous vehicles in a traffic planning method like in Matias. Doing so “optimize[s] operational key performance indicators (KPIs) such as Excess Waiting Time (EWT) and On-Time Adherence (EWT)” (Matias, page 10, paragraph 0037).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao in view of Petroff.
Regarding claim 14, Cao teaches the method of claim 1. Cao does not teach, but Petroff does teach
wherein the technical system is a vehicle fleet and the quality metric is a driving economy indicator (Petroff, page 12, column 12, lines 19-21, “Tests were also performed to evaluate disturbances in the reinforcement learning’s simulation represented as an entropy value. The larger the entropy value, the more disturbances occur” where “Vehicle dispatching encounters many disturbances. For example, vehicles 6 break down, roads go out of operation, source and destination locations S1, S2, D1, D2 go out of operation due to breakdown or changes in location, etc.” (Petroff, page 11, column 10, lines 26-29) and “the invention may be adapted to use in many diverse applications such as…dispatching and fleet management of police and emergency vehicles,… commercial or government vehicle fleets” (Petroff, page 13, lines 4-14). Examiner notes that the driving economy indicator is the entropy value. ).
Cao and Petroff are analogous to the claimed invention because they teach optimizing parameters using machine learning techniques. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Cao to have the quality metric as a driving economy indicator. Doing so “is most advantageous in that it addresses these real-world problems during simulation. Thus the vehicle dispatching system 1 or method 30 may develop the best policy π to follow for a given environmental state of the work area 8 and is well prepared for real-world events when they occur in the work area 8.” (Petroff, page 8, column 4, lines 61-66).
Response to Arguments
The claim objections have been overcome in light of the instant amendments.
The previous 112(b) rejection on claim 4 has been overcome in light of the instant amendments. Examiner notes that new 112(b) rejections have been made in light of the instant amendments.
On page 7-9, Applicant argues:
The amendments to claim 1 introduce the further element of repeating or continuing training of at least one ML algorithm instance that is not selected for continued use in controlling the technical system, after modifying its hyperparameter values to be more similar to those of the at least one instance of the ML algorithm instance that is selected for such continued use.
It must be noted that this further element does not explicitly recite performing mathematical calculations or similar. It must also be noted that training of an ML algorithm instance cannot practically be performed in the human mind, and the further element does therefore not fall within either of the "mathematical concepts" or "mental processes" groupings of abstract ideas. Consequently, the added element does not form part of any judicial exception and corresponds exactly to an "additional element" as mentioned in Step 2AProng Two a).
To evaluate whether such an additional element serves to integrate the judicial exception into a practical application, it is evaluated whether the claim as a whole improves the functioning of a computer or improves another technology or technical field. For this task, the specification is consulted for technical explanations of the asserted improvement and evaluate whether the claim reflects that improvement.
Paragraph [0014] of the specification describes how the claimed solution makes no assumption about the exact form or functioning of the ML algorithm, as long as it is configured to map inputs to outputs. This allows the claimed method to apply to a broad range of ML algorithms, and without requiring the ML algorithm to be parameterizable with respect to the hyperparameters. Further, as explained in [0015], providing multiple instances of the ML algorithm allows for a more efficient exploration of the hyperparameter domain, compared to optimization algorithms (such as gradient descent) relying on single sample-probing that risk getting stuck in local maxima or minima.
Even more importantly, as explained in [0035], feeding of the state information about the technical system (i.e., the plurality of input states) to all instances of the ML algorithm allows for the instances not currently selected to control the technical system to perform decision-making on a same basis as the instance currently controlling the technical system (rather than, say, on a state which would result if the non-selected/used instance's output was actually applied to the technical system). By retraining ( or continuing to train) such non-selected instances with hyperparameter values that are more similar to those of the selected instance than before, the internal states of the non-selected instances will be similar, or sometimes identical, to that of the instance selected to control the system. This has the benefit that these non-selected instances are more ready to take over control of the technical system should the currently selected instance fail. This in a more seamless manner and with reduced risk of causing abrupt changes to how the technical system is controlled, due to the increased similarly of expected operation resulting from how the instances are trained, while still being allowed to explore the hyperparameter value space.
It is clear that the claimed solution provides a safer and more efficient way of controlling a technical system using an ML algorithm, and that such an improvement of a technological field would be recognized by one of ordinary skill in the art after studying the description.
Regarding the Applicant’s argument that the further element of repeating or continuing training provides an improvement, Examiner respectfully disagrees. Examiner firstly notes that “modifying its values of the hyperparameters to be more similar to those of the at least one instance of the ML algorithm selected for continued use in controlling the technical system” recites a mental process as it encompasses a human mentally modifying values of hyperparameters. Examiner further notes that “for at least on instance of the ML algorithm not selected for continued use in controlling the technical system, repeating or continuing training of the instance after modifying its values…” cannot integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” and using a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)). Examiner further notes that paragraphs [0014], [0015], and [0035] merely asserts an improvement in a conclusory manner and therefore does not disclose an improvement to the technology and does not integrate the abstract idea into a practical application under 2106.04(d)(1).
On pages 9-10, Applicant argues:
Under Step 2AProng Two, even if the claim (as amended) may recite one or more elements forming part of a judicial exception, the claim is not directed to the judicial exception but integrates the judicial exception into a practical application. Claim 1 (as well as any claim depending thereon) is therefore (subject-matter) eligible.
Regarding the Applicant’s argument that the claim integrates the judicial exception into a practical application, Examiner respectfully disagrees. Specifically, Examiner notes that the integration into a practical application must come from the additional elements. However, the additional elements do not provide an integration into a practical application. For example,
“a computer-implemented method of managing a machine-learning, ML, algorithm for controlling a technical system, wherein training of the ML algorithm is dependent on one or more hyperparameters” does not integrate the abstract idea into a practical application because it recites a technological environment in which to apply a judicial exception (see MPEP 2106.05(h)). “Providing a plurality of instances of the ML algorithm trained using different values of the hyperparameters”, “obtaining a plurality of input states related to the technical system”, and “by providing each input state as input to the instances of the ML algorithm” do not integrate the abstract idea into a practical application because they recite insignificant extra-solution activity of data gathering (see MPEP 2106.05(g)). “For at least one instance of the ML algorithm not selected for continued use in controlling the technical system, repeating or continuing training of the instance after modifying its values of the hyperparameters …” does not integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” (see MPEP 2106.05(f)).)
On page 10, Applicant argues:
Further claim 1 as amended does not only disclose a generic way of mapping inputs to outputs using ML algorithm instances, or to just "pick the best performing" such instance for controlling the technical system. To the contrary, claim 1 as amended recites specific limitations for what to do with the other, non-selected instances, how these are to be (re)trained using modified hyperparameter values, and how these hyperparameter values are to be modified. These limitations are not just mere/insignificant extra-solution activity, but is what provides the above mentioned improvement of a technological field.
Regarding Applicant’s argument that the amended limitation of “for at least one instance of the ML algorithm not selected for continued use in controlling the technical system, repeating or continuing training of the instance after modifying its values” provides an improvement, Examiner respectfully disagrees. Specifically, Examiner notes that cannot integrate the abstract idea into a practical application because it amounts to mere “apply it on a computer” and using a computer as a tool to perform the abstract idea and cannot provide significantly more (see MPEP 2106.05(f)). Examiner further notes that paragraphs [0014], [0015], and [0035] merely asserts an improvement in a conclusory manner and therefore does not disclose an improvement to the technology and does not integrate the abstract idea into a practical application under 2106.04(d)(1).
On pages 11-12, Applicant argues:
The Final Rejection (FR) alleges that page 14, paragraph [0049] of Cao discloses the claimed element of "obtaining a plurality of input states related to the technical system". This passage of Cao discloses that "[t]he resource configuration generator 302 can take, as input, various parameters that specify how to generate the DT [distributed training]-configurations". The Office Action further notes that "the parameters are the input states [as recited in claim I as amended].
Cao explains in [0049] that a DT-configuration is a tuple of one or more parameters, such as of a number nps of parameter servers and a number nwn of worker nodes, together with a resource budget introducing limitations on what values of such parameters that the resource configuration generator 302 is allowed to use.
It has thus been established that what is in Cao considered as corresponding to the claimed "input states related to the technical system" are instructions received by the resource configuration generator 302 regarding how to generate the DT-configurations that define a DT configuration search space of tuples, or, in other words, what the DT-configuration search space that is to be explored is (i.e. how many dimensions the space has, and what limits on each dimension and/or joint dimensions that are imposed). Phrased differently, the corresponding "input states" as found in Cao are what defines the boundaries of the DT-configuration search space from which the configuration generator 302 selects possible DT-configuration candidates.
If assuming that these "various parameters" corresponds to some form of "input state" of the system in which the solution of Cao is implemented, it is obvious that Cao does not disclose anywhere to provide the "various parameters" as input to any ML algorithm. This is clear i.e. from Fig. 4 of Cao, wherein the only input to the resource allocation loop 412 (in which the actual "ML jobs" are executed) is a particular resource configuration to evaluate and not the "various parameters" received by the resource configuration generator 302. Phrased differently, in Cao, how the resource configuration that is to be evaluated when running the ML job is defined depends on the "various parameters" provided to the resource configuration generator 302, but the "various parameters" are only used to define limits, number of parameters, etc. of each resource configuration.
Thus, Cao obviously fails to disclose providing "each input state as input to the instances of the ML algorithm" as recited in claim 1 as amended.
Regarding the Applicant’s argument that Cao does not teach the limitation “by providing each input state as input to the instances of the ML algorithm”, Examiner respectfully disagrees. Specifically, Examiner notes that the DT-configuration that has the possible parameters is fed or input into the Bayesian optimization process which is a job or instance (Cao, page 15, paragraph 0051; Cao, page 11, paragraph 0030). Under the broadest reasonable interpretation, by feeding or inputting the configuration that contains the possible parameters into the algorithm, each input state is provided as input to the instances of the ML algorithm.
On page 13-14, Applicant argues:
The Office Action further alleges that in Cao, the "specific parameter values for achieving shortest possible job completion time" correspond to our claimed "outputs corresponding to control signals of the technical system". As claim 1 as amended clarifies, the outputs of our claim are outputs from the ML algorithm instances, which is certainly thus not the case in Cao. To the contrary, in Cao, the outputs of the "ML JOBS" 414 are used to assess the quality of the ML algorithm/model itself and have nothing to do with the resource configuration (that is, number of CPU cores, memory amount, etc.) used to perform the actual execution of the ML JOBS. See Figure 5 of Cao.
Thus, if interpreting the "specific parameter values for achieving shortest possible job completion time" (that is, the optimal resource configuration candidate), it is clear that Cao does not disclose obtaining these as output from the ML JOBS, and it is certainly clear that Cao does not disclose adjusting any hyperparameter values whatsoever based on such "outputs" (i.e., based on such optimal resource configuration candidates). As made obvious from Figure 5 of Cao, the hyperparameters are selected solely based on how well the ML JOB (box 504, evaluated by box 524) performs, and in no way based on the resource configurations. Phrased differently, the hyperparameters control the quality (i.e., accuracy) of the ML model, while the resource configuration controls how much processing power that is allotted for executing the ML model, which are two separate things. Optimizing the resource configurations is made in order to optimize execution time but has nothing with accuracy or quality of the model to do, as the latter is instead decided by the selected hyperparameter values.
From the above, it is clear that in Cao, "resource configurations" is not the same as "hyperparameters", and the optimization of each of the two is not based on or related to that of the other. Cao thus fails also to disclose the cited limitations of obtaining outputs from the instances of the ML algorithm, and to base any decision about which ML algorithm instance to use to control the system based on statistics of such outputs.
Regarding the Applicant’s argument that Cao does not disclose the limitation of “outputs from the instances of the ML algorithm correspond to control signals of the technical system”, Examiner respectfully disagrees. Specifically, Examiner notes that the specific parameter values for achieving shortest possible job completion time are the outputs. Examiner further notes that the specific parameter values correspond to control signals since they control the learning process. Examiner further notes that the job is the instance of the ML algorithm as the job is used to narrow down possible resource configurations (Cao, page 15, paragraph 0051; Cao, page 11, paragraph 0030). Additionally, Examiner notes that the specific parameter values are parameters. According to paragraph 0030 of Cao, parameters can be hyperparameters which control the technical system.
On page 14, Applicant argues:
Further, Cao does not mention that several different instances of an ML job can be executed in parallel (or sequentially) and that e.g. one of the ML jobs is used for controlling a technical system, while one or more other ML jobs are not used for such control, and that hyperparameter values of the non-used ML jobs are to be adjusted such that they become more similar to those of the ML job used to control the technical system. In fact, Cao does not disclose selection of any particular ML job for continued use in controlling the technical system. To the contrary, what is selected in Cao is which resource configuration to use to for executing of remaining ML jobs, which is certainly not even remotely similar.
Regarding the applicant’s argument that Cao does not disclose an ML can be executed in parallel, Examiner respectfully notes that Claim 1 does not include language claiming parallel processing.
Regarding the applicant’s argument that Cao does not disclose the limitation of “for at least one instance of the ML algorithm not selected for continued use in controlling the technical system, repeating or continuing training of the instance after modifying its values of the hyperparameters to be more similar to those of the at least one instance of the ML algorithm selected for continued use in controlling the technical system”, Examiner respectfully disagrees. Specifically, paragraphs 0063-0064 and Figure 5 teach this limitation. Examiner notes that the remaining training jobs described in paragraph 0063 and 0064 are the instances not selected. Examiner further notes that the remaining training jobs continue training at box 504 in Figure 5. Box 504 executes the training jobs after hyperparameters are tuned or modified in box 524 of Figure 5. Examiner further notes that by changing the hyperparameters to satisfy a termination condition, the values of the hyperparameters are more similar to the hyperparameters of the ML instance selected (Cao, page 16, paragraphs 0063-0064; Figure 5).
On pages 14-15, Applicant argues:
When rejecting previous claim 3, the subject-matter of which is now included as part of amended claim 1, the Office Action alleges that paragraph [0064] of Cao discloses "wherein said selecting includes modifying the values of the hyperparameters ... and repeating the mapping and evaluating steps". However, this phrase has to be read together with what was already in previous claim 1, i.e. that the selecting is performed on the basis of statistics of the quality metric for the outputs. As the Office Action has concluded, the "outputs" in Cao is the optimal resource configuration candidate, and there is surely nothing whatsoever in Cao that discloses to modify the hyperparameters based on such an output (as made obvious from studying of Figures 4 and 5 of Cao). Thus, Cao fails to disclose the subject matter of previous claim 3, and thereby also the corresponding part of claim 1 as amended.
Regarding the Applicant’s argument that the prior art fails to disclose the subject matter of previous claim 3 and thus also the corresponding part of claim 1, Examiner respectfully disagrees. Specifically, Examiner notes that cancelled claim 3 was not incorporated into claim 1 word for word. Thus claim 1 cannot be analyzed on what was recited previously in cancelled claim 3.
Regardless of what was recited in cancelled claim 3, the prior art teaches on the amended limitation “for at least one instance of the ML algorithm not selected for continued use in controlling the technical system, repeating or continuing training of the instance after modifying its values of the hyperparameters to be more similar to those of the at least one instance of the ML algorithm selected for continued use in controlling the technical system”. Examiner further notes that under broadest reasonable interpretation, this newly amended limitation does not need to be read together with “on the basis of statistics..” since “on the basis of statistics of the quality metric for the outputs ” is being interpreted with “selecting at least one instance of the ML algorithm for continued use in controlling the technical system” which is all read before a semi colon indicating the end of a limitation.
Examiner further respectfully notes that the limitation of “wherein said selecting includes modifying the values of the hyperparameters…and repeating the mapping and evaluating steps” is not recited in claim 1 and thus is a moot point.
On page 15-16, Applicant argues:
Although each of these documents appear to mention hyperparameter tuning, it would not have been obvious for any person having ordinary skill in the art to e.g. modify the solution of Cao to arrive at the claimed invention. This is because Cao is really not about hyperparameter tuning, but about finding an optimal resource configuration to use as part of performing such hyperparameter tuning (in Cao, hyperparameter tuning is performed by repeating training of an ML model but with different hyperparameters, until some quality criteria is met). For example, it has not at all been shown why or how e.g. replacing the "various parameters" of Cao with something related to the "input states" as claimed would not break the functioning of the solution of Cao completely. Neither has it been shown why e.g. replacing the alleged "output" of Cao, that is, the optimal resource configuration, with the output from one of the ML jobs would not fully break the functioning of the solution of Cao.
In response to applicant’s argument that there is no teaching, suggestion, or motivation to combine the references, the examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, Cao and Marinakis are analogous to the claimed invention because they teach updating hyperparameters in machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to replace a failed instance like in Marinakis. Doing so “allow[s] the decision logic generation system to generate and alter scenarios according to the performance assessment” (Marinakis, page 54, paragraph 0890).
Cao and Matias are analogous to the claimed invention because they teach tuning hyperparameters in a machine learning architecture. It would have been obvious to one having ordinary skill in the art before the effective filing date to have modified Cao to use the instance in a traffic planning method like in Matias. Doing so “optimize[s] operational key performance indicators (KPIs) such as Excess Waiting Time (EWT) and On-Time Adherence (EWT)” (Matias, page 10, paragraph 0037).
Cao in view of Matias and Petroff are analogous to the claimed invention because they teach fleet dispatching using machine learning techniques. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Cao in view of Matias to use planning nodes and edges with a finite set of motions. Doing so “advantageously provides vehicle dispatch method 30 to communicate with many vehicles 6 operating among and between the source locations S1, S2 and destination locations D1, D2 to achieve one or more goals or objectives, e.g., maximizing the amount of material hauled, minimizing delivery time, etc.” (Petroff, page 8, column 4, lines 61-66).
Cao and Petroff are analogous to the claimed invention because they teach optimizing parameters using machine learning techniques. It would have been obvious to one having ordinary skill in the art prior to the effective filing date to have modified Cao to have the quality metric as a driving economy indicator. Doing so “is most advantageous in that it addresses these real-world problems during simulation. Thus the vehicle dispatching system 1 or method 30 may develop the best policy π to follow for a given environmental state of the work area 8 and is well prepared for real-world events when they occur in the work area 8.” (Petroff, page 8, column 4, lines 61-66).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Yi et al. (“An Automated Hyperparameter Search-Based Deep Learning Model for Highway Traffic Prediction”) also describes tuning hyperparameters in a traffic planning environment.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KAITLYN R HAEFNER whose telephone number is (571)272-1429. The examiner can normally be reached Monday - Thursday: 7:15 am - 5:15 pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michelle Bechtold can be reached at (571) 431-0762. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.R.H./ Examiner, Art Unit 2148 /MICHELLE T BECHTOLD/Supervisory Patent Examiner, Art Unit 2148