DETAILED ACTION
This action is responsive to the application filed on 04/15/2024. Claims 1-20 are pending in the case. Claims 1, 11, and 20 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/28/2024 is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 4, 12, and 14 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 2, the claim recites: “first set of performance metrics” in line 7. The parent claim recites: “a first set of performance metrics” in line 7. It is unclear if applicant is attempting to recite a new claim element or if applicant is attempting to refer to a previously recited claim element. For examination purposes, the limitation has been interpreted as “the first set of performance metrics”, referring to the previously recited claim element.
Regarding claim 4, the claim recites: “the hyperparameter” in line 8. The claim also recites: “a hyperparameter of the first set of hyperparameters” in line 3 and “a hyperparameter” in line 5, line 6, and line 7. It is unclear if applicant is attempting to recite a new claim element or if applicant is attempting to refer to a previously recited claim element. For examination purposes, the claim is being interpreted as “the hyperparameter” in line 5, line 6, line 7, and line 8, referring to the previously recited claim element in line 3.
Regarding claim 12, the claim recites: “first set of performance metrics” in line 6. The parent claim recites: “a first set of performance metrics” in line 5. It is unclear if applicant is attempting to recite a new claim element or if applicant is attempting to refer to a previously recited claim element. For examination purposes, the limitation has been interpreted as “the first set of performance metrics”, referring to the previously recited claim element.
Regarding claim 14, the claim recites: “the hyperparameter” in line 7. The claim also recites: “a hyperparameter of the first set of hyperparameters” in lines 2-3 and “a hyperparameter” in line 4, line 5, and line 6. It is unclear if applicant is attempting to recite a new claim element or if applicant is attempting to refer to a previously recited claim element. For examination purposes, the claim is being interpreted as “the hyperparameter” in line 4, line 5, line 6, and line 7, referring to the previously recited claim element in lines 2-3.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Step 1 Statutory Category: Claim 1 is directed to a machine, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 1 recites, in part, “obtaining a first set of performance metrics corresponding to the first output”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case a judgment. See MPEP § 2106.04(a)(2)(III). Further, the claim recites: “based at least on the first set of performance metrics, computing a first adjustment for the first set of values for the first set of hyperparameters, the first adjustment comprising modifying at least one value of the first set of values”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case an evaluation. See MPEP § 2106.04(a)(2)(III). Additionally, in light of applicant’s specification paragraphs 0126 and 0165, the limitation covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, the claim recites: “applying the first adjustment to the first set of values for the first set of hyperparameters to generate a second set of values for the first set of hyperparameters”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case an evaluation. See MPEP § 2106.04(a)(2)(III). Additionally, in light of applicant’s specification paragraphs 0166 and 0172, the limitation covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “One or more non-transitory computer readable media comprising instructions which, when executed by one or more hardware processors, cause performance of operations”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “applying a first machine learning model, with a first set of values for a first set of hyperparameters, to a first set of input to generate a first output”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g). Further, the claim recites: “applying the first machine learning model, with the second set of values for the first set of hyperparameters, to a second set of input”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element: “One or more non-transitory computer readable media comprising instructions which, when executed by one or more hardware processors, cause performance of operations” amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the additional elements: “applying a first machine learning model, with a first set of values for a first set of hyperparameters, to a first set of input to generate a first output” and “applying the first machine learning model, with the second set of values for the first set of hyperparameters, to a second set of input” amount to adding insignificant extra-solution activity to the judicial exception, and further, are well‐understood, routine, and conventional as taught by activity is supported under Berkheimer Option 2, Gervais et al., U.S. Patent No. 12632736, Col 7, Lines 59-62, “The neural network inference engine 112 provides functionalities of a neural network, allowing to infer output(s) based on inputs using the predictive model (generated by the training server 300), as is well known in the art”. The claim is not patent eligible.
Regarding claim 2, the rejection of claim 1 is incorporated, and further, the claim recites: “obtaining a second set of performance metrics corresponding to a second output generated by the application of the first machine learning model to the second set of input”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Further, the claim recites: “determining a performance effect of the first adjustment based at least in part on first set of performance metrics and the second set of performance metrics”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Further, the claim recites: “computing a second adjustment for the second set of values for the first set of hyperparameters based at least in part on the performance effect of the first adjustment”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0126 and 0165, recites mathematical concepts in addition to those identified in the rejection of the parent claim. Further, the claim recites: “applying the second adjustment to the second set of values for the first set of hyperparameters to generate a third set of values for the first set of hyperparameters”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0166 and 0172, recites mathematical concepts in addition to those identified in the rejection of the parent claim.
Further, the claim recites: “configuring the first machine learning model with the second set of values for the first set of hyperparameters”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the claim recites: “applying the first machine learning model, with the third set of hyperparameters, to a third set of input to generate a third output”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception, see MPEP §2106.05(g), and further is well‐understood, routine, and conventional as taught by activity is supported under Berkheimer Option 2, Gervais et al., U.S. Patent No. 12632736, Col 7, Lines 59-62, “The neural network inference engine 112 provides functionalities of a neural network, allowing to infer output(s) based on inputs using the predictive model (generated by the training server 300), as is well known in the art”. The claim is not patent eligible.
Regarding claim 3, the rejection of claim 1 is incorporated, and further, the claim recites: “obtaining a second set of performance metrics corresponding to the second output”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Further, the claim recites: “based at least on the second set of performance metrics, computing a second adjustment for the third set of values for the second set of hyperparameters, the second adjustment comprising an increase or decrease to at least one value of the third set of values”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0126 and 0165, recites mathematical concepts in addition to those identified in the rejection of the parent claim. Further, the claim recites: “applying the second adjustment to the third set of values for the second set of hyperparameters to generate a fourth set of values for the second set of hyperparameters”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0166 and 0172, recites mathematical concepts in addition to those identified in the rejection of the parent claim.
Further, the claim recites: “applying the first machine learning model, with a third set of values for a second set of hyperparameters, to a third set of input to generate a second output” and “applying the first machine learning model, with the fourth set of values for the second set of hyperparameters, to a fourth set of input”. These limitations are additional elements that amount to adding insignificant extra-solution activity to the judicial exception, see MPEP §2106.05(g), and further is well‐understood, routine, and conventional as taught by activity is supported under Berkheimer Option 2, Gervais et al., U.S. Patent No. 12632736, Col 7, Lines 59-62, “The neural network inference engine 112 provides functionalities of a neural network, allowing to infer output(s) based on inputs using the predictive model (generated by the training server 300), as is well known in the art”. The claim is not patent eligible.
Regarding claim 4, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein applying the first adjustment to the first set of values for the first set of hyperparameters comprises removing an effect of a hyperparameter of the first set of hyperparameters at least by one of: a) setting the value of a hyperparameter to a default value; b) setting the value of a hyperparameter to a null value; c) adjusting the weight of a hyperparameter; or d) masking the hyperparameter”. This limitation is a continuation of the “applying the first adjustment to the first set of values for the first set of hyperparameters to generate a second set of values for the first set of hyperparameters” limitation identified as an abstract idea in the rejection of the parent claim. Thus, the claim recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 5, the rejection of claim 1 is incorporated, and further, the claim recites: “in response at least in part to receiving a user instruction to adjust a value of a hyperparameter, adjusting a value of the second set of values for a hyperparameter of the first set of hyperparameters”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0166 and 0172, recites mathematical concepts in addition to those identified in the rejection of the parent claim.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 6, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein computing the first adjustment for the first set of values for the first set of hyperparameters comprises: in response at least in part to interpreting configuration data to determine that a first hyperparameter of the first set of hyperparameters is configured to be adjustable, computing an adjustment for the value for the first hyperparameter” .This limitation is a continuation of the “based at least on the first set of performance metrics, computing a first adjustment for the first set of values for the first set of hyperparameters, the first adjustment comprising modifying at least one value of the first set of values” limitation identified as an abstract idea in the rejection of the parent claim. Thus, the claim recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 7, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein computing the first adjustment for the first set of values for the first set of hyperparameters comprises: in response at least in part to interpreting a configuration to determine that a first value for a first hyperparameter of the first set of hyperparameters is configured to be non-adjustable, retaining the first value for the first hyperparameter; and in response at least in part to interpreting a configuration to determine that a second value for a second hyperparameter of the first set of hyperparameters is configured to be adjustable, computing an adjustment for the second value for the second hyperparameter”. This limitation is a continuation of the “based at least on the first set of performance metrics, computing a first adjustment for the first set of values for the first set of hyperparameters, the first adjustment comprising modifying at least one value of the first set of values” limitation identified as an abstract idea in the rejection of the parent claim. Thus, the claim recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 8, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein computing the first adjustment for the first set of values for the first set of hyperparameters comprises: in response at least in part to interpreting a configuration to determine that a first hyperparameter of the first set of hyperparameters is configured to satisfy a value-restricting condition: computing a value that does not satisfy the value-restricting condition for the first hyperparameter of the first set of hyperparameters”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0126 and 0165, recites mathematical concepts in addition to those identified in the rejection of the parent claim. Further, the claim recites: “adjusting the first value to generate a value that does satisfy the value-restricting condition, wherein the second value is in the first set of values”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0126 and 0165, recites mathematical concepts in addition to those identified in the rejection of the parent claim. Further, the claim recites: “in response at least in part to interpreting a configuration to determine that a third value for a second hyperparameter of the first set of hyperparameters is adjustable, computing an adjustment for the second value for the second hyperparameter; including the second value and the third value in the first adjustment”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0126 and 0165, recites mathematical concepts in addition to those identified in the rejection of the parent claim.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 9, the rejection of claim 8 is incorporated, and further, the claim recites: “wherein computing the first adjustment for the first set of values for the first set of hyperparameters comprises: in response at least in part to interpreting a configuration to determine that a fourth value for a third hyperparameter of the first set of hyperparameters is configured to be non-adjustable, retaining the fourth value for the third hyperparameter”. This limitation is a continuation of the “based at least on the first set of performance metrics, computing a first adjustment for the first set of values for the first set of hyperparameters, the first adjustment comprising modifying at least one value of the first set of values” limitation identified as an abstract idea in the rejection of the parent claim. Thus, the claim recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 10, the rejection of claim 1 is incorporated, and further, the claim recites: “obtaining a second set of performance metrics corresponding to the second output”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Further, the claim recites: “obtaining a third set of performance metrics corresponding to the third output”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim. Further, the claim recites: “generating a model value score based at least on: a) the second set of performance metrics; b) the third set of performance metrics; c) a resource usage metric associated with the first machine learning model; and d) a resource usage metric associated with the second machine learning model”. This limitation recites mathematical concepts in addition to those identified in the rejection of the parent claim. Thus, the claim recites a judicial exception.
Further, the claim recites: “based at least in part on the application of the first machine learning model to the second set of input, generating a second output” and “applying a second machine learning model, with the second set of values for the first set of hyperparameters, to the second set of input to generate a third output”. These limitations are additional elements that amount to adding insignificant extra-solution activity to the judicial exception, see MPEP §2106.05(g), and further is well‐understood, routine, and conventional as taught by activity is supported under Berkheimer Option 2, Gervais et al., U.S. Patent No. 12632736, Col 7, Lines 59-62, “The neural network inference engine 112 provides functionalities of a neural network, allowing to infer output(s) based on inputs using the predictive model (generated by the training server 300), as is well known in the art”. The claim is not patent eligible.
Regarding claim 11:
Step 1 Statutory Category: Claim 11 is directed to a method, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 11 recites, in part, “obtaining a first set of performance metrics corresponding to the first output”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case a judgment. See MPEP § 2106.04(a)(2)(III). Further, the claim recites: “based at least on the first set of performance metrics, computing a first adjustment for the first set of values for the first set of hyperparameters, the first adjustment comprising modifying at least one value of the first set of values”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case an evaluation. See MPEP § 2106.04(a)(2)(III). Additionally, in light of applicant’s specification paragraphs 0126 and 0165, the limitation covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, the claim recites: “applying the first adjustment to the first set of values for the first set of hyperparameters to generate a second set of values for the first set of hyperparameters”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case an evaluation. See MPEP § 2106.04(a)(2)(III). Additionally, in light of applicant’s specification paragraphs 0166 and 0172, the limitation covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “wherein the method is performed by at least one device including a hardware processor”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “applying a first machine learning model, with a first set of values for a first set of hyperparameters, to a first set of input to generate a first output”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g). Further, the claim recites: “applying the first machine learning model, with the second set of values for the first set of hyperparameters, to a second set of input”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element: “wherein the method is performed by at least one device including a hardware processor” amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the additional elements: “applying a first machine learning model, with a first set of values for a first set of hyperparameters, to a first set of input to generate a first output” and “applying the first machine learning model, with the second set of values for the first set of hyperparameters, to a second set of input” amount to adding insignificant extra-solution activity to the judicial exception, and further, are well‐understood, routine, and conventional as taught by activity is supported under Berkheimer Option 2, Gervais et al., U.S. Patent No. 12632736, Col 7, Lines 59-62, “The neural network inference engine 112 provides functionalities of a neural network, allowing to infer output(s) based on inputs using the predictive model (generated by the training server 300), as is well known in the art”. The claim is not patent eligible.
Regarding claim 12, the rejection of claim 11 is incorporated, and further, claim 12 is substantially similar to claim 2 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 13, the rejection of claim 11 is incorporated, and further, claim 13 is substantially similar to claim 3 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 14, the rejection of claim 11 is incorporated, and further, claim 14 is substantially similar to claim 4 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 15, the rejection of claim 11 is incorporated, and further, claim 15 is substantially similar to claim 5 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 16, the rejection of claim 11 is incorporated, and further, claim 16 is substantially similar to claim 6 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 17, the rejection of claim 11 is incorporated, and further, claim 17 is substantially similar to claim 7 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 18, the rejection of claim 11 is incorporated, and further, the claim recites: “wherein computing the first adjustment for the first set of values for the first set of hyperparameters comprises: in response at least in part to interpreting a configuration to determine that a first hyperparameter of the first set of hyperparameters is configured to be value-restricted to a first set of values: computing a first value for the first hyperparameter of the first set of hyperparameters”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0126 and 0165, recites mathematical concepts in addition to those identified in the rejection of the parent claim. Further, the claim recites: “adjusting the first value to generate a second value, wherein the second value is in the first set of values”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0126 and 0165, recites mathematical concepts in addition to those identified in the rejection of the parent claim. Further, the claim recites: “in response at least in part to interpreting a configuration to determine that a third value for a second hyperparameter of the first set of hyperparameters is adjustable, computing an adjustment for the second value for the second hyperparameter; including the second value and the third value in the first adjustment”. This limitation recites mental processes in addition to those identified in the rejection of the parent claim, and additionally, in light of applicant’s specification paragraphs 0126 and 0165, recites mathematical concepts in addition to those identified in the rejection of the parent claim.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 19, the rejection of claim 11 is incorporated, and further, claim 19 is substantially similar to claim 10 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 20:
Step 1 Statutory Category: Claim 20 is directed to a system, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial exception: Claim 20 recites, in part, “obtaining a first set of performance metrics corresponding to the first output”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case a judgment. See MPEP § 2106.04(a)(2)(III). Further, the claim recites: “based at least on the first set of performance metrics, computing a first adjustment for the first set of values for the first set of hyperparameters, the first adjustment comprising modifying at least one value of the first set of values”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case an evaluation. See MPEP § 2106.04(a)(2)(III). Additionally, in light of applicant’s specification paragraphs 0126 and 0165, the limitation covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, the claim recites: “applying the first adjustment to the first set of values for the first set of hyperparameters to generate a second set of values for the first set of hyperparameters”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion), in this case an evaluation. See MPEP § 2106.04(a)(2)(III). Additionally, in light of applicant’s specification paragraphs 0166 and 0172, the limitation covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C).
Step 2A Prong 2 Integration into a practical application: This judicial exception is not integrated into a practical application. In particular the claim recites: “A system comprising: at least one device including a hardware processor; the system being configured to perform operations”. This limitation is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “applying a first machine learning model, with a first set of values for a first set of hyperparameters, to a first set of input to generate a first output”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g). Further, the claim recites: “applying the first machine learning model, with the second set of values for the first set of hyperparameters, to a second set of input”. This limitation is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §2106.05(g).
Step 2B Significantly more: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element: “A system comprising: at least one device including a hardware processor; the system being configured to perform operations” amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the additional elements: “applying a first machine learning model, with a first set of values for a first set of hyperparameters, to a first set of input to generate a first output” and “applying the first machine learning model, with the second set of values for the first set of hyperparameters, to a second set of input” amount to adding insignificant extra-solution activity to the judicial exception, and further, are well‐understood, routine, and conventional as taught by activity is supported under Berkheimer Option 2, Gervais et al., U.S. Patent No. 12632736, Col 7, Lines 59-62, “The neural network inference engine 112 provides functionalities of a neural network, allowing to infer output(s) based on inputs using the predictive model (generated by the training server 300), as is well known in the art”. The claim is not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 3-4, 11, 13-14, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Jomaa et al., Hyp-RL : Hyperparameter Optimization by Reinforcement Learning, 06/27/2019, https://arxiv.org/pdf/1906.11527, hereinafter referred to as “Jomaa”.
Regarding claim 1, Jomaa teaches One or more non-transitory computer readable media comprising instructions which, when executed by one or more hardware processors (Jomaa, Page 9, Section 6, “The netowork is implemented in Tensorflow [1] and the reported results are the averaged over 5 environments representing 5-splits. The environments are wrapped in the OpenAI framework [5]. The code is available here”; A person of ordinary skill would recognize this requires the use of a generic computer, providing evidence for “one or more non-transitory computer readable media”, “instructions”, and “one or more hardware processors”), cause performance of operations comprising:
applying a first machine learning model, with a first set of values for a first set of hyperparameters, to a first set of input to generate a first output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
obtaining a first set of performance metrics corresponding to the first output (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
based at least on the first set of performance metrics, computing a first adjustment for the first set of values for the first set of hyperparameters, the first adjustment comprising modifying at least one value of the first set of values; applying the first adjustment to the first set of values for the first set of hyperparameters to generate a second set of values for the first set of hyperparameters (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”);
applying the first machine learning model, with the second set of values for the first set of hyperparameters, to a second set of input (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively, so in the next iteration the model is applied to a second set of input).
Regarding claim 3, the rejection of claim 1 is incorporated, and further, Jomaa teaches wherein the operations further comprise: applying the first machine learning model, with a third set of values for a second set of hyperparameters, to a third set of input to generate a second output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively);
obtaining a second set of performance metrics corresponding to the second output (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively);
based at least on the second set of performance metrics, computing a second adjustment for the third set of values for the second set of hyperparameters, the second adjustment comprising an increase or decrease to at least one value of the third set of values; applying the second adjustment to the third set of values for the second set of hyperparameters to generate a fourth set of values for the second set of hyperparameters (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”; Jomaa, Page 4, Section 3, Lines 5-6, “We define Λ = Λ1 ×···×ΛP as the P-dimensional hyperparameter space that can include continuous or discrete values”; The steps of algorithm 1 are performed iteratively and a person of ordinary skill in the art would recognize that different hyperparameter configurations have values that are larger or smaller than the previous configuration);
applying the first machine learning model, with the fourth set of values for the second set of hyperparameters, to a fourth set of input (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively).
Regarding claim 4, the rejection of claim 1 is incorporated, and further, Jomaa teaches wherein applying the first adjustment to the first set of values for the first set of hyperparameters comprises removing an effect of a hyperparameter of the first set of hyperparameters at least by: adjusting the weight of a hyperparameter (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
It is noted the claim recites alternative language and Jomaa teaches at least one of the alternatives.
Regarding claim 11, Jomaa teaches A method comprising: applying a first machine learning model, with a first set of values for a first set of hyperparameters, to a first set of input to generate a first output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
obtaining a first set of performance metrics corresponding to the first output (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
based at least on the first set of performance metrics, computing a first adjustment for the first set of values for the first set of hyperparameters, the first adjustment comprising modifying at least one value of the first set of values; applying the first adjustment to the first set of values for the first set of hyperparameters to generate a second set of values for the first set of hyperparameters (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”);
applying the first machine learning model, with the second set of values for the first set of hyperparameters, to a second set of input (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively, so in the next iteration the model is applied to a second set of input);
wherein the method is performed by at least one device including a hardware processor (Jomaa, Page 9, Section 6, “The netowork is implemented in Tensorflow [1] and the reported results are the averaged over 5 environments representing 5-splits. The environments are wrapped in the OpenAI framework [5]. The code is available here”; A person of ordinary skill would recognize this requires the use of a generic computer, providing evidence for “at least one device including a hardware processor”).
Regarding claim 13, the rejection of claim 11 is incorporated, and further, Jomaa teaches applying the first machine learning model, with a third set of values for a second set of hyperparameters, to a third set of input to generate a second output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively);
obtaining a second set of performance metrics corresponding to the second output (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively);
based at least on the second set of performance metrics, computing a second adjustment for the third set of values for the second set of hyperparameters, the second adjustment comprising an increase or decrease to at least one value of the third set of values; applying the second adjustment to the third set of values for the second set of hyperparameters to generate a fourth set of values for the second set of hyperparameters (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”; Jomaa, Page 4, Section 3, Lines 5-6, “We define Λ = Λ1 ×···×ΛP as the P-dimensional hyperparameter space that can include continuous or discrete values”; The steps of algorithm 1 are performed iteratively and a person of ordinary skill in the art would recognize that different hyperparameter configurations have values that are larger or smaller than the previous configuration);
applying the first machine learning model, with the fourth set of values for the second set of hyperparameters, to a fourth set of input (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively).
Regarding claim 14, the rejection of claim 11 is incorporated, and further, Jomaa teaches wherein applying the first adjustment to the first set of values for the first set of hyperparameters comprises removing an effect of a hyperparameter of the first set of hyperparameters at least by: adjusting the weight of a hyperparameter (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
It is noted the claim recites alternative language and Jomaa teaches at least one of the alternatives.
Regarding claim 20, Jomaa teaches A system comprising: at least one device including a hardware processor (Jomaa, Page 9, Section 6, “The netowork is implemented in Tensorflow [1] and the reported results are the averaged over 5 environments representing 5-splits. The environments are wrapped in the OpenAI framework [5]. The code is available here”; A person of ordinary skill would recognize this requires the use of a generic computer, providing evidence for “at least one device” and “a hardware processors”); the system being configured to perform operations comprising:
applying a first machine learning model, with a first set of values for a first set of hyperparameters, to a first set of input to generate a first output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
obtaining a first set of performance metrics corresponding to the first output (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
based at least on the first set of performance metrics, computing a first adjustment for the first set of values for the first set of hyperparameters, the first adjustment comprising modifying at least one value of the first set of values; applying the first adjustment to the first set of values for the first set of hyperparameters to generate a second set of values for the first set of hyperparameters (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”);
applying the first machine learning model, with the second set of values for the first set of hyperparameters, to a second set of input (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively, so in the next iteration the model is applied to a second set of input).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Jomaa in view of Bettosi, Reinforcement Learning: an Easy Introduction to Value Iteration, 09/10/2023, https://towardsdatascience.com/reinforcement-learning-an-easy-introduction-to-value-iteration-e4cfe0731fd5/, hereinafter referred to as “Bettosi”.
Regarding claim 2, the rejection of claim 1 is incorporated, and further, Jomaa teaches obtaining a second set of performance metrics corresponding to a second output generated by the application of the first machine learning model to the second set of input (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively, so in the next iteration another set of performance metrics corresponding to a second output would be generated);
…
computing a second adjustment for the second set of values for the first set of hyperparameters …; applying the second adjustment to the second set of values for the first set of hyperparameters to generate a third set of values for the first set of hyperparameters (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”; The steps of algorithm 1 are performed iteratively);
configuring the first machine learning model with the second set of values for the first set of hyperparameters; applying the first machine learning model, with the third set of hyperparameters, to a third set of input to generate a third output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively).
Jomaa does not explicitly teach determining a performance effect of the first adjustment based at least in part on first set of performance metrics and the second set of performance metrics nor computing a second adjustment based at least in part on the performance effect of the first adjustment.
Bettosi teaches determining a performance effect of the first adjustment based at least in part on first set of performance metrics and the second set of performance metrics (Bettosi, Page 8, How does the Value Iteration algorithm work?, The Value Iteration Algorithm, Line 8, “
∆
←
max
∆
,
v
-
V
s
”; Bettosi, Page 13, (3), Paragraph 1, “Before moving to the next pass in the inner loop, we perform a comparison between the current value of Δ and the difference between the previous value of this state v and the new value for this state we just calculated V(s). We update Δ to the larger of these two: Δ ← max(Δ,| v - V(s)|). This helps us track how close we are to convergence”);
computing a second adjustment for the second set of values for the first set of hyperparameters based at least in part on the performance effect of the first adjustment (Bettosi, Page 13, (3), Paragraph 2, “We perform step (3) for each s in S before breaking out of the inner loop and performing a check on the convergence condition Δ < θ. If this condition is met, we break out the outer loop, if not, we go back to step (2)”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include determining a performance effect and using it to compute an adjustment as taught by Bettosi. The motivation to do so would have been the ability to leverage dynamic programming, improving the efficiency of the model (Bettosi, Pages 7-8, What is Value Iteration?).
Regarding claim 12, the rejection of claim 11 is incorporated, and further, Jomaa teaches obtaining a second set of performance metrics corresponding to a second output generated by the application of the first machine learning model to the second set of input (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively, so in the next iteration another set of performance metrics corresponding to a second output would be generated);
…
computing a second adjustment for the second set of values for the first set of hyperparameters …; applying the second adjustment to the second set of values for the first set of hyperparameters to generate a third set of values for the first set of hyperparameters (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”; The steps of algorithm 1 are performed iteratively);
configuring the first machine learning model with the second set of values for the first set of hyperparameters; applying the first machine learning model, with the third set of hyperparameters, to a third set of input to generate a third output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; The steps of algorithm 1 are performed iteratively).
Jomaa does not explicitly teach determining a performance effect of the first adjustment based at least in part on first set of performance metrics and the second set of performance metrics nor computing a second adjustment based at least in part on the performance effect of the first adjustment.
Bettosi teaches determining a performance effect of the first adjustment based at least in part on first set of performance metrics and the second set of performance metrics (Bettosi, Page 8, How does the Value Iteration algorithm work?, The Value Iteration Algorithm, Line 8, “
∆
←
max
∆
,
v
-
V
s
”; Bettosi, Page 13, (3), Paragraph 1, “Before moving to the next pass in the inner loop, we perform a comparison between the current value of Δ and the difference between the previous value of this state v and the new value for this state we just calculated V(s). We update Δ to the larger of these two: Δ ← max(Δ,| v - V(s)|). This helps us track how close we are to convergence”);
computing a second adjustment for the second set of values for the first set of hyperparameters based at least in part on the performance effect of the first adjustment (Bettosi, Page 13, (3), Paragraph 2, “We perform step (3) for each s in S before breaking out of the inner loop and performing a check on the convergence condition Δ < θ. If this condition is met, we break out the outer loop, if not, we go back to step (2)”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include determining a performance effect and using it to compute an adjustment as taught by Bettosi. The motivation to do so would have been the ability to leverage dynamic programming, improving the efficiency of the model (Bettosi, Pages 7-8, What is Value Iteration?).
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Jomaa in view of Kaufmann et al., A SURVEY OF REINFORCEMENT LEARNING FROM HUMAN FEEDBACK, 12/22/2023, https://arxiv.org/pdf/2312.14925v1, hereinafter referred to as “Kaufmann”.
Regarding claim 5, the rejection of claim 1 is incorporated, and further, Jomaa teaches adjusting a value of the second set of values for a hyperparameter of the first set of hyperparameters (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
Jomaa does not explicitly teach adjusting a value in response at least in part to receiving a user instruction to adjust a value of a hyperparameter.
Kaufmann teaches adjusting a value in response at least in part to receiving a user instruction to adjust a value of a hyperparameter (Kaufmann, Page 11, Section 2.4, Algorithm 1, Lines 4-7, “4: //Reward learning 5: Generate queries from B 6: Update D with answers to queries from the oracle 7: Update
ψ
using D (e.g., to maximize Eq. (2))”; Kaufmann, Page 12, Paragraph 4, Lines 1, “the oracle is a human or group of humans”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include adjusting a hyperparameter in response to receiving a user interaction as taught by Kaufmann. The motivation to do so would have been to avoid reward engineering challenges and fostering robust training (Kaufmann, Page 3, Section 1.1, Lines 5-7, “Learning the agent’s objective from human feedback circumvents reward engineering challenges and fosters robust training, with the reward function dynamically refined and adjusted to distributional shifts as the agent learns”).
Regarding claim 15, the rejection of claim 11 is incorporated, and further, Jomaa teaches adjusting a value of the second set of values for a hyperparameter of the first set of hyperparameters (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
Jomaa does not explicitly teach adjusting a value in response at least in part to receiving a user instruction to adjust a value of a hyperparameter.
Kaufmann teaches adjusting a value in response at least in part to receiving a user instruction to adjust a value of a hyperparameter (Kaufmann, Page 11, Section 2.4, Algorithm 1, Lines 4-7, “4: //Reward learning 5: Generate queries from B 6: Update D with answers to queries from the oracle 7: Update
ψ
using D (e.g., to maximize Eq. (2))”; Kaufmann, Page 12, Paragraph 4, Lines 1, “the oracle is a human or group of humans”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include adjusting a hyperparameter in response to receiving a user interaction as taught by Kaufmann. The motivation to do so would have been to avoid reward engineering challenges and fostering robust training (Kaufmann, Page 3, Section 1.1, Lines 5-7, “Learning the agent’s objective from human feedback circumvents reward engineering challenges and fosters robust training, with the reward function dynamically refined and adjusted to distributional shifts as the agent learns”).
Claims 6-7 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Jomaa in view of Katkoori, U.S. Patent Application Publication No. 20250259077, hereinafter referred to as “Katkoori”.
Regarding claim 6, the rejection of claim 1 is incorporated, and further, Jomaa teaches computing an adjustment for the value for the first hyperparameter (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
Jomaa does not explicitly teach computing an adjustment in response at least in part to interpreting configuration data to determine that a first hyperparameter of the first set of hyperparameters is configured to be adjustable.
Katkoori teaches computing an adjustment in response at least in part to interpreting configuration data to determine that a first hyperparameter of the first set of hyperparameters is configured to be adjustable (Katkoori, Paragraph 0034, Lines 1-6, “In some embodiments, the hyperparameters applicable to the data reduction processing may be designated as adjustable, non-adjustable, or adjustable only within a range. These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include interpreting configuration data to determine if hyperparameters are configured to be adjustable as taught by Katkoori. The motivation to do so would have been to ensure any adjustments made to hyperparameters do not result in the model to be unsuitable for deployment (Katkoori, Paragraph 0034, Lines 4-9, “These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model, so that any hyperparameter refinements and adjusting (as described below) does not cause the resulting model to be unsuitable for its ultimate deployment”).
Regarding claim 7, the rejection of claim 1 is incorporated, and further, Jomaa teaches computing an adjustment for the second value for the second hyperparameter (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
Jomaa does not explicitly teach in response at least in part to interpreting a configuration to determine that a first value for a first hyperparameter of the first set of hyperparameters is configured to be non-adjustable, retaining the first value for the first hyperparameter nor computing an adjustment in response at least in part to interpreting a configuration to determine that a second value for a second hyperparameter of the first set of hyperparameters is configured to be adjustable.
Katkoori teaches in response at least in part to interpreting a configuration to determine that a first value for a first hyperparameter of the first set of hyperparameters is configured to be non-adjustable, retaining the first value for the first hyperparameter (Katkoori, Paragraph 0034, Lines 1-9, “In some embodiments, the hyperparameters applicable to the data reduction processing may be designated as adjustable, non-adjustable, or adjustable only within a range. These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model, so that any hyperparameter refinements and adjusting (as described below) does not cause the resulting model to be unsuitable for its ultimate deployment”); and
computing an adjustment in response at least in part to interpreting a configuration to determine that a second value for a second hyperparameter of the first set of hyperparameters is configured to be adjustable (Katkoori, Paragraph 0034, Lines 1-6, “In some embodiments, the hyperparameters applicable to the data reduction processing may be designated as adjustable, non-adjustable, or adjustable only within a range. These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include interpreting configuration data to determine if hyperparameters are configured to be adjustable as taught by Katkoori. The motivation to do so would have been to ensure any adjustments made to hyperparameters do not result in the model to be unsuitable for deployment (Katkoori, Paragraph 0034, Lines 4-9, “These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model, so that any hyperparameter refinements and adjusting (as described below) does not cause the resulting model to be unsuitable for its ultimate deployment”).
Regarding claim 16, the rejection of claim 11 is incorporated, and further, Jomaa teaches computing an adjustment for the value for the first hyperparameter (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
Jomaa does not explicitly teach computing an adjustment in response at least in part to interpreting a configuration to determine that a first hyperparameter of the first set of hyperparameters is configured to be adjustable.
Katkoori teaches computing an adjustment in response at least in part to interpreting a configuration to determine that a first hyperparameter of the first set of hyperparameters is configured to be adjustable (Katkoori, Paragraph 0034, Lines 1-6, “In some embodiments, the hyperparameters applicable to the data reduction processing may be designated as adjustable, non-adjustable, or adjustable only within a range. These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include interpreting configuration data to determine if hyperparameters are configured to be adjustable as taught by Katkoori. The motivation to do so would have been to ensure any adjustments made to hyperparameters do not result in the model to be unsuitable for deployment (Katkoori, Paragraph 0034, Lines 4-9, “These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model, so that any hyperparameter refinements and adjusting (as described below) does not cause the resulting model to be unsuitable for its ultimate deployment”).
Regarding claim 17, the rejection of claim 11 is incorporated, and further, Jomaa teaches computing an adjustment for the second value for the second hyperparameter (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
Jomaa does not explicitly teach in response at least in part to interpreting a configuration to determine that a first value for a first hyperparameter of the first set of hyperparameters is configured to be non-adjustable, retaining the first value for the first hyperparameter nor computing an adjustment in response at least in part to interpreting a configuration to determine that a second value for a second hyperparameter of the first set of hyperparameters is configured to be adjustable.
Katkoori teaches in response at least in part to interpreting a configuration to determine that a first value for a first hyperparameter of the first set of hyperparameters is configured to be non-adjustable, retaining the first value for the first hyperparameter (Katkoori, Paragraph 0034, Lines 1-9, “In some embodiments, the hyperparameters applicable to the data reduction processing may be designated as adjustable, non-adjustable, or adjustable only within a range. These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model, so that any hyperparameter refinements and adjusting (as described below) does not cause the resulting model to be unsuitable for its ultimate deployment”); and
computing an adjustment in response at least in part to interpreting a configuration to determine that a second value for a second hyperparameter of the first set of hyperparameters is configured to be adjustable (Katkoori, Paragraph 0034, Lines 1-6, “In some embodiments, the hyperparameters applicable to the data reduction processing may be designated as adjustable, non-adjustable, or adjustable only within a range. These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include interpreting configuration data to determine if hyperparameters are configured to be adjustable as taught by Katkoori. The motivation to do so would have been to ensure any adjustments made to hyperparameters do not result in the model to be unsuitable for deployment (Katkoori, Paragraph 0034, Lines 4-9, “These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model, so that any hyperparameter refinements and adjusting (as described below) does not cause the resulting model to be unsuitable for its ultimate deployment”).
Claims 8-9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Jomaa in view of Katkoori in further view of Wu et al., Efficient hyperparameter optimization through model-based reinforcement learning, 10/07/2020, Neurocomputing, Volume 409, Pages 381-393, ISSN 0925-2312, https://doi.org/10.1016/j.neucom.2020.06.064, hereinafter referred to as “Wu”.
Regarding claim 8, the rejection of claim 1 is incorporated, and further, Jomaa teaches computing an adjustment for the second value for the second hyperparameter; including the second value and the third value in the first adjustment (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
Jomaa does not explicitly teach in response at least in part to interpreting a configuration to determine that a first hyperparameter of the first set of hyperparameters is configured to satisfy a value-restricting condition: computing a value that does not satisfy the value-restricting condition for the first hyperparameter of the first set of hyperparameters; adjusting the first value to generate a value that does satisfy the value-restricting condition, wherein the second value is in the first set of values nor computing an adjustment for a value in response at least in part to interpreting a configuration to determine that a third value for a second hyperparameter of the first set of hyperparameters is adjustable.
Wu teaches in response at least in part to interpreting a configuration to determine that a first hyperparameter of the first set of hyperparameters is configured to satisfy a value-restricting condition: computing a value that does not satisfy the value-restricting condition for the first hyperparameter of the first set of hyperparameters; adjusting the first value to generate a value that does satisfy the value-restricting condition, wherein the second value is in the first set of values (Wu, Page 384, Section 3.4, Lines 1-2, “The value of a hyperparameter
λ
is determined by sampling from the distribution”; Wu, Page 384, Section 3.4, Col 2, “the following customized normalization procedure is taken: Scale the means of the distributions
μ
to
μ
'
by the tanh function in the range (-1, 1); Sample values h from the new distributions
N
(
μ
'
,
σ
)
; Scale h into the range of value [
h
L
,
h
U
] by the following method:
h
'
=
h
L
+
h
U
-
h
L
×
1
+
h
2
4
λ
=
c
l
i
p
_
a
n
d
_
c
o
n
v
e
r
t
h
'
,
h
L
,
h
U
(
5
)
where
h
U
and
h
L
represent the upper and lower bounds of a hyperparameter Table 1, respectively. The clip_and_convert function removes the value h’ outside of the interval [
h
L
,
h
U
] and performs a type of conversion”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include ensuring the hyperparameters satisfy a value-restricting condition as taught by Wu. The motivation to do so would have been to make it easier for the agent to efficiently explore the search space (Wu, Page 384, Section 3.4, Lines 2-8, “Random sampling causes large variations during training since the ranges of the possible values of hyperparameters are significantly different, e.g., for the extreme gradient boosting [38], the range of n_estimators is from 50 to 1200 and learning rate varies from 0.001 to 0.1 (as shown in Table1). As a result, it is difficult for the agent to efficiently explore the search space”).
The proposed combination thus far does not explicitly teach computing an adjustment for a value in response at least in part to interpreting a configuration to determine that a third value for a second hyperparameter of the first set of hyperparameters is adjustable.
Katkoori teaches computing an adjustment for a value in response at least in part to interpreting a configuration to determine that a third value for a second hyperparameter of the first set of hyperparameters is adjustable (Katkoori, Paragraph 0034, Lines 1-6, “In some embodiments, the hyperparameters applicable to the data reduction processing may be designated as adjustable, non-adjustable, or adjustable only within a range. These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of the proposed combination to include interpreting configuration data to determine if hyperparameters are configured to be adjustable as taught by Katkoori. The motivation to do so would have been to ensure any adjustments made to hyperparameters do not result in the model to be unsuitable for deployment (Katkoori, Paragraph 0034, Lines 4-9, “These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model, so that any hyperparameter refinements and adjusting (as described below) does not cause the resulting model to be unsuitable for its ultimate deployment”).
Regarding claim 9, the rejection of claim 8 is incorporated, and further, the proposed combination teaches in response at least in part to interpreting a configuration to determine that a fourth value for a third hyperparameter of the first set of hyperparameters is configured to be non-adjustable, retaining the fourth value for the third hyperparameter (Katkoori, Paragraph 0034, Lines 1-9, “In some embodiments, the hyperparameters applicable to the data reduction processing may be designated as adjustable, non-adjustable, or adjustable only within a range. These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model, so that any hyperparameter refinements and adjusting (as described below) does not cause the resulting model to be unsuitable for its ultimate deployment”).
Regarding claim 18, the rejection of claim 11 is incorporated, and further, Jomaa teaches computing an adjustment for the second value for the second hyperparameter; including the second value and the third value in the first adjustment (Jomaa, Page 7, Algorithm 1, Steps 8-11, “Generate new state
s
t
+
1
=
τ
s
t
,
λ
t
,
r
t
⊳
e
q
.
10
/ Store
B
⟵
B
∪
s
t
,
s
t
+
1
,
a
t
,
r
t
and replace oldest tuple if |B|
>
N
b
/ … / Update the
Q
^
network by minimizing
θ
=
argmin
θ
'
∑
(
s
,
a
,
Q
)
∈
B
Q
-
Q
^
s
,
a
;
θ
'
2
”; Jomaa, Page 7, Algorithm 1, Step 6, “Determine next action
a
t
:
a
t
=
~
U
n
i
f
∧
,
i
f
p
~
U
n
i
f
(
0,1
)
<
ϵ
max
a
Q
^
s
t
,
a
;
θ
,
o
t
h
e
r
w
i
s
e
”).
Jomaa does not explicitly teach in response at least in part to interpreting a configuration to determine that a first hyperparameter of the first set of hyperparameters is configured to be value-restricted to a first set of values: computing a first value for the first hyperparameter of the first set of hyperparameters; adjusting the first value to generate a second value, wherein the second value is in the first set of values; nor computing an adjustment for a value in response at least in part to interpreting a configuration to determine that a third value for a second hyperparameter of the first set of hyperparameters is adjustable.
Wu teaches in response at least in part to interpreting a configuration to determine that a first hyperparameter of the first set of hyperparameters is configured to be value-restricted to a first set of values: computing a first value for the first hyperparameter of the first set of hyperparameters; adjusting the first value to generate a second value, wherein the second value is in the first set of values (Wu, Page 384, Section 3.4, Lines 1-2, “The value of a hyperparameter
λ
is determined by sampling from the distribution”; Wu, Page 384, Section 3.4, Col 2, “the following customized normalization procedure is taken: Scale the means of the distributions
μ
to
μ
'
by the tanh function in the range (-1, 1); Sample values h from the new distributions
N
(
μ
'
,
σ
)
; Scale h into the range of value [
h
L
,
h
U
] by the following method:
h
'
=
h
L
+
h
U
-
h
L
×
1
+
h
2
4
λ
=
c
l
i
p
_
a
n
d
_
c
o
n
v
e
r
t
h
'
,
h
L
,
h
U
(
5
)
where
h
U
and
h
L
represent the upper and lower bounds of a hyperparameter Table 1, respectively. The clip_and_convert function removes the value h’ outside of the interval [
h
L
,
h
U
] and performs a type of conversion”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include ensuring the hyperparameters satisfy a value-restricting condition as taught by Wu. The motivation to do so would have been to make it easier for the agent to efficiently explore the search space (Wu, Page 384, Section 3.4, Lines 2-8, “Random sampling causes large variations during training since the ranges of the possible values of hyperparameters are significantly different, e.g., for the extreme gradient boosting [38], the range of n_estimators is from 50 to 1200 and learning rate varies from 0.001 to 0.1 (as shown in Table1). As a result, it is difficult for the agent to efficiently explore the search space”).
The proposed combination thus far does not explicitly teach computing an adjustment for a value in response at least in part to interpreting a configuration to determine that a third value for a second hyperparameter of the first set of hyperparameters is adjustable.
Katkoori teaches computing an adjustment for a value in response at least in part to interpreting a configuration to determine that a third value for a second hyperparameter of the first set of hyperparameters is adjustable (Katkoori, Paragraph 0034, Lines 1-6, “In some embodiments, the hyperparameters applicable to the data reduction processing may be designated as adjustable, non-adjustable, or adjustable only within a range. These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of the proposed combination to include interpreting configuration data to determine if hyperparameters are configured to be adjustable as taught by Katkoori. The motivation to do so would have been to ensure any adjustments made to hyperparameters do not result in the model to be unsuitable for deployment (Katkoori, Paragraph 0034, Lines 4-9, “These categories may be determined according to the type of input data and the type of device (e.g., resource constrained IoT device) that will be utilizing the model, so that any hyperparameter refinements and adjusting (as described below) does not cause the resulting model to be unsuitable for its ultimate deployment”).
Claims 10 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Jomaa in view of Smith, U.S. Patent Application Publication No. 20250292091, hereinafter referred to as “Smith”.
Regarding claim 10, the rejection of claim 1 is incorporated, and further, Jomaa teaches wherein the operations further comprise: based at least in part on the application of the first machine learning model to the second set of input, generating a second output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
applying a second machine learning model, with the second set of values for the first set of hyperparameters, to the second set of input to generate a third output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; Algorithm 1 is performed iteratively, each time a model is trained with a new hyperparameter configuration it can be considered a new model);
obtaining a second set of performance metrics corresponding to the second output (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
obtaining a third set of performance metrics corresponding to the third output (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; Algorithm 1 is performed iteratively).
Jomaa does not explicitly teach generating a model value score based at least on: a) the second set of performance metrics; b) the third set of performance metrics; c) a resource usage metric associated with the first machine learning model; and d) a resource usage metric associated with the second machine learning model.
Smith teaches generating a model value score based at least on: a) the second set of performance metrics; b) the third set of performance metrics; c) a resource usage metric associated with the first machine learning model; and d) a resource usage metric associated with the second machine learning model (Smith, Paragraph 0304, Lines 1-5, “the processing system may use reinforcement learning techniques to guide adjustments to the neural network architecture. The processing system may reward configurations that improve accuracy or reduce latency while penalizing those that increase resource usage”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include generating a model value score as taught by Smith. The motivation to do so would have been enabling the system to adapt to varying workloads and resource availability (Smith, Paragraph 0304, Lines 6-7, “This approach may enable the system to adapt autonomously to varying workloads and resource availability”).
Regarding claim 19, the rejection of claim 11 is incorporated, and further, Jomaa teaches wherein the operations further comprise: based at least in part on the application of the first machine learning model to the second set of input, generating a second output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
applying a second machine learning model, with the second set of values for the first set of hyperparameters, to the second set of input to generate a third output (Jomaa, Page 6, Lines 3-4 and Equation 9, “The reward function is set as the hyperparameter response function, and depends on the data set D and the action selected, as shown below:
R
D
,
a
=
-
f
(
D
,
λ
=
a
)
”; Jomaa, Page 5, Section 4.1, Lines 21-23 and Equation 6, “The response can be any meaningful performance metric, so without loss of generality, we consider it to be the validation loss of the model M trained with hyperparameters λ:
f
D
,
λ
=
L
(
M
λ
(
D
t
r
a
i
n
,
D
v
a
l
i
d
)
”; Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; Algorithm 1 is performed iteratively, each time a model is trained with a new hyperparameter configuration it can be considered a new model);
obtaining a second set of performance metrics corresponding to the second output (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
);
obtaining a third set of performance metrics corresponding to the third output (Jomaa, Page 7, Algorithm 1, Step 7, “Receive reward
r
t
=
R
D
,
λ
=
a
t
⊳
e
q
.
9
; Algorithm 1 is performed iteratively).
Jomaa does not explicitly teach generating a model value score based at least on: a) the second set of performance metrics; b) the third set of performance metrics; c) a resource usage metric associated with the first machine learning model; and d) a resource usage metric associated with the second machine learning model.
Smith teaches generating a model value score based at least on: a) the second set of performance metrics; b) the third set of performance metrics; c) a resource usage metric associated with the first machine learning model; and d) a resource usage metric associated with the second machine learning model (Smith, Paragraph 0304, Lines 1-5, “the processing system may use reinforcement learning techniques to guide adjustments to the neural network architecture. The processing system may reward configurations that improve accuracy or reduce latency while penalizing those that increase resource usage”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the hyperparameter optimization method of Jomaa to include generating a model value score as taught by Smith. The motivation to do so would have been enabling the system to adapt to varying workloads and resource availability (Smith, Paragraph 0304, Lines 6-7, “This approach may enable the system to adapt autonomously to varying workloads and resource availability”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Boue et al., U.S. Patent Application Publication No. 20240273400 teaches a hyperparameter tuning system which allocates a weight to each hyperparameter based on a performance attribution statistic of the hyperparameter and updates, in a series of experiments, the hyperparameters based on their weights and selects a set of hyperparameters for the machine learning model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOLLY CLARKE SIPPEL whose telephone number is (571)272-3270. The examiner can normally be reached Monday - Friday, 7:30 a.m. - 4:30 p.m. ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571)272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.C.S./ Examiner, Art Unit 2122
/KAKALI CHAKI/ Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights