DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Application filed on 12/22/2022. Claims 1-20 are pending in the case. Claims 1 and 19-20 are independent claims.
Claim Objections
Claims 19-20 are objected to because of the following informalities:
Claims 19-20 recite “includes respective retraining criteria” where “includes a respective retraining criteria” was apparently intended.
Claims 19-20 recite “receiving, from one or more computing devices” where “receiving, from the one or more computing devices” was apparently intended.
Claim 20 recites “monitoring data for industrial machine-learning operations model” where “monitoring data for an industrial machine-learning operations model” was apparently intended.
Appropriate correction is required.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
As to claim 1:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining, from the monitoring data, to retrain the industrial machine-learning operations model, the determining comprising computing drift parameters, each of the drift parameters being indicative of a type of observable drift of the industrial machine-learning operations model, wherein the drift parameters comprise (i) a usage drift, (ii) a performance drift, (iii) a data drift, and (iv) a prediction drift, and wherein each drift parameter includes a respective retraining criteria” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “confirming, from the drift parameters, the respective retraining criteria is met by at least one of the drift parameters” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “triggering, in response to the determining to retrain the industrial machine-learning operations model, [an update]” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “receiving, from the one or more computing devices, monitoring data for an industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “receiving, from the one or more computing devices, monitoring data for an industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 2:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “monitoring data for the industrial machine-learning operations model comprises monitoring (A) model usage data, (B) model performance data, (C) sensor data, and (D) prediction data” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “monitoring data for the industrial machine-learning operations model comprises monitoring (A) model usage data, (B) model performance data, (C) sensor data, and (D) prediction data” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 3:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “generating an updated industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “providing, to the one or more computing devices, the updated industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
No, the limitation “providing, to the one or more computing devices, the updated industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “generating an updated industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “providing, to the one or more computing devices, the updated industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
No, the limitation “providing, to the one or more computing devices, the updated industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 4:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “generating a refined training data set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “generating a refined training data set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
No, the limitation “retraining the industrial machine-learning operations model to generate the updated industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “retraining the industrial machine-learning operations model to generate the updated industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “generating a refined training data set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “generating a refined training data set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
No, the limitation “retraining the industrial machine-learning operations model to generate the updated industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “retraining the industrial machine-learning operations model to generate the updated industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 5:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “(i) relabeling and/or reannotating an original training set” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “(ii) generating a new training set including new prediction data collected by the one or more computing devices” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “(ii) generating a new training set including new prediction data collected by the one or more computing devices” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “(ii) generating a new training set including new prediction data collected by the one or more computing devices” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “(ii) generating a new training set including new prediction data collected by the one or more computing devices” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 6:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining a first performance parameter for the updated industrial machine-learning operations model exceeds a second performance parameter for the industrial machine-learning operations model” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “providing, to the one or more computing devices, the updated industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “providing, to the one or more computing devices, the updated industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 7:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining the first performance parameter for the updated industrial machine-learning operations model exceeds the second performance parameter for the industrial machine-learning operations model comprises comparing a first output of the updated industrial machine-learning operations model utilizing an exemplary data set and a second output of the industrial machine-learning operations model utilizing the exemplary data set” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 8:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining the respective retraining criteria is met by at least one of the drift parameters comprises determining that a weighted retraining criteria is met by the weighted drift parameters” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “drift parameters comprise weighted drift parameters” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “drift parameters comprise weighted drift parameters” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 9:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “the data drift includes metadata drift” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “the data drift includes metadata drift” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 10:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “meeting the respective retraining criteria for each drift parameter of the drift parameters depends in part on the type of observable drift of the drift parameter” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “meeting the respective retraining criteria for each drift parameter of the drift parameters depends in part on the type of observable drift of the drift parameter” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 11:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “the respective retraining criteria is met by at least two of the drift parameters” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “the respective retraining criteria is met by at least two of the drift parameters” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 12:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “triggering the update comprises providing an alert to initiate a retraining pipeline” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “triggering the update comprises providing an alert to initiate a retraining pipeline” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “triggering the update comprises providing an alert to initiate a retraining pipeline” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “triggering the update comprises providing an alert to initiate a retraining pipeline” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 13:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “triggering the update comprises triggering an automatic retraining of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “triggering the update comprises triggering an automatic retraining of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “triggering the update comprises triggering an automatic retraining of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “triggering the update comprises triggering an automatic retraining of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 14:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining the drift parameters based on usage drift comprises determining a frequency of utilization of the industrial machine-learning operations model by the one or more computing devices over a first period of time, and wherein the respective retraining criteria for the drift parameter based on the usage drift comprises a minimum threshold usage of the industrial machine-learning operations model for a second period of time” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 15:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining the drift parameters based on performance drift comprises determining a compute time for the industrial machine-learning operations model on available hardware of the one or more computing devices, and wherein the respective retraining criteria for the drift parameters based on the performance drift comprises a deviation of the compute time from an average compute time for the industrial machine-learning operations model on the available hardware of the one or more computing devices” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 16:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining the drift parameters based on data drift comprises determining a deviation of the prediction data generated utilizing the industrial machine-learning operations model from training data utilized to train the industrial machine-learning operations model” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “monitoring data comprises prediction data” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “monitoring data comprises prediction data” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 17:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining the drift parameters based on prediction drift comprises determining an accuracy in the prediction data is below a threshold prediction accuracy” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 18:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
The analysis of the parent claim is incorporated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “triggering the update comprises providing an alert to a user” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
No, the limitation “in response to receiving a confirmation from the user to initiate a retraining pipeline, initiating the retraining pipeline” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “in response to receiving a confirmation from the user to initiate a retraining pipeline, initiating the retraining pipeline” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “triggering the update comprises providing an alert to a user” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
No, the limitation “in response to receiving a confirmation from the user to initiate a retraining pipeline, initiating the retraining pipeline” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “in response to receiving a confirmation from the user to initiate a retraining pipeline, initiating the retraining pipeline” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 19:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining, from the monitoring data, to retrain the industrial machine-learning operations model, the determining comprising computing drift parameters, each of the drift parameters being indicative of a type of observable drift of the industrial machine-learning operations model, wherein the drift parameters comprise (i) a usage drift, (ii) a performance drift, (iii) a data drift, and (iv) a prediction drift, and wherein each drift parameter includes respective retraining criteria” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “confirming, from the drift parameters, the respective retraining criteria is met by at least one of the drift parameters” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “triggering, in response to the determining to retrain the industrial machine-learning operations model, [an update]” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
No, the limitation “receiving, from one or more computing devices, monitoring data for an industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
No, the limitation “receiving, from one or more computing devices, monitoring data for an industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 20:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a manufacture.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining, from the monitoring data, to retrain the industrial machine-learning operations model, the determining comprising computing drift parameters, each of the drift parameters being indicative of a type of observable drift of the industrial machine-learning operations model, wherein the drift parameters comprise (i) a usage drift, (ii) a performance drift, (iii) a data drift, and (iv) a prediction drift, and wherein each drift parameter includes respective retraining criteria” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “confirming, from the drift parameters, the respective retraining criteria is met by at least one of the drift parameters” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “triggering, in response to the determining to retrain the industrial machine-learning operations model, [an update]” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “one or more non-transitory computer storage media encoded with computer program instructions that, when executed by one or more computers, cause the one or more computers to perform operations” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
No, the limitation “receiving, from one or more computing devices, monitoring data for industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “one or more non-transitory computer storage media encoded with computer program instructions that, when executed by one or more computers, cause the one or more computers to perform operations” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
No, the limitation “receiving, from one or more computing devices, monitoring data for industrial machine-learning operations model” is an additional element that amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP § 2106.05(g). Furthermore the additional element is directed to receiving or transmitting data over a network, which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “update of the industrial machine-learning operations model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. § 103 as being unpatentable over Singh et al. (US 2022/0024032 A1, hereinafter Singh) in view of Salganik et al. (“Bit By Bit: Social Research in the Digital Age,” 12 June 2019, https://web.archive.org/web/20190610033023/https://www.bitbybitbook.com/en/1st-ed/observing-behavior/characteristics/drift/, hereinafter Salganik).
As to independent claim 1, Singh teaches a method for an industrial machine-learning operations model monitoring system, the method comprising:
receiving, from the one or more computing devices, monitoring data for an industrial machine-learning operations model (“information is obtained with respect to AI/ML model performance. … This information may assist in determining how well an AI/ML model is performing over time,” paragraph 0066 lines 1-2, 10-12);
determining, from the monitoring data, to retrain the industrial machine-learning operations model, the determining comprising computing drift parameters, each of the drift parameters being indicative of a type of observable drift of the industrial machine-learning operations model, wherein the drift parameters comprise (iii) a data drift, and (iv) a prediction drift, and wherein each drift parameter includes a respective retraining criteria (“After this information has been collected, the information is analyzed to determine whether data drift, model drift, or both, have occurred. When a change condition is found (e.g., predictions or input data fall outside of a historical range) and/or a change threshold is met or exceeded (e.g., model statistical performance deviates from historical performance by at least a certain amount), an alert or a retraining trigger is generated,” paragraph 0067 lines 1-8), and
confirming, from the drift parameters, the respective retraining criteria is met by at least one of the drift parameters (“After this information has been collected, the information is analyzed to determine whether data drift, model drift, or both, have occurred. When a change condition is found (e.g., predictions or input data fall outside of a historical range) and/or a change threshold is met or exceeded (e.g., model statistical performance deviates from historical performance by at least a certain amount), an alert or a retraining trigger is generated,” paragraph 0067 lines 1-8); and
triggering, in response to the determining to retrain the industrial machine-learning operations model, an update of the industrial machine-learning operations model (“The AI/ML model can then be retrained using the collected information and/or other information to attempt to improve AI/ML model performance,” paragraph 0068 lines 1-3).
Singh does not appear to expressly teach a method wherein the drift parameters comprise (i) a usage drift, and (ii) a performance drift.
Salganik teaches a method wherein the drift parameters comprise (i) a usage drift, and (ii) a performance drift (“these systems change in three main ways: population drift (change in who is using them), behavioral drift (change in how people are using them), and system drift (change in the system itself),” section 2.3.7 “Drifting” paragraph 2 lines 2-4).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the drift parameters of Singh to comprise the usage drift and performance drift of Salganik. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely monitoring usage drift and performance drift (“these systems change in three main ways: population drift (change in who is using them), behavioral drift (change in how people are using them), and system drift (change in the system itself),” section 2.3.7 “Drifting” paragraph 2 lines 2-4). Therefore, the rationale to support a conclusion that the claim would have been obvious is that the combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 2, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein monitoring data for the industrial machine-learning operations model comprises monitoring (A) model usage data, (B) model performance data (“these systems change in three main ways: population drift (change in who is using them), behavioral drift (change in how people are using them), and system drift (change in the system itself),” section 2.3.7 “Drifting” Salganik paragraph 2 lines 2-4), (C) sensor data, and (D) prediction data (“After this information has been collected, the information is analyzed to determine whether data drift, model drift, or both, have occurred. When a change condition is found (e.g., predictions or input data fall outside of a historical range) and/or a change threshold is met or exceeded (e.g., model statistical performance deviates from historical performance by at least a certain amount), an alert or a retraining trigger is generated,” Singh paragraph 0067 lines 1-8).
As to dependent claim 3, the rejection of claim 2 is incorporated. Singh/Salganik further teaches a method wherein triggering the updated of the industrial machine-learning operations model comprises:
generating an updated industrial machine-learning operations model (“The AI/ML model can then be retrained using the collected information and/or other information to attempt to improve AI/ML model performance,” Singh paragraph 0068 lines 1-3); and
providing, to the one or more computing devices, the updated industrial machine-learning operations model (“when the retrained AI/ML model meets certain thresholds, it could be automatically deployed alongside the existing AI/ML model,” Singh paragraph 0068 lines 4-6).
As to dependent claim 4, the rejection of claim 3 is incorporated. Singh/Salganik further teaches a method wherein generating an updated industrial machine-learning operations model comprises:
generating a refined training data set (“The AI/ML model can then be retrained using the collected information and/or other information to attempt to improve AI/ML model performance,” Singh paragraph 0068 lines 1-3, emphasis added); and
retraining the industrial machine-learning operations model to generate the updated industrial machine-learning operations model (“The AI/ML model can then be retrained using the collected information and/or other information to attempt to improve AI/ML model performance,” Singh paragraph 0068 lines 1-3).
As to dependent claim 5, the rejection of claim 4 is incorporated. Singh/Salganik further teaches a method wherein generating the refined training data set comprises one or more of (i) relabeling and/or reannotating an original training set, and (ii) generating a new training set including new prediction data collected by the one or more computing devices (“The AI/ML model can then be retrained using the collected information and/or other information to attempt to improve AI/ML model performance,” Singh paragraph 0068 lines 1-3, emphasis added).
As to dependent claim 6, the rejection of claim 4 is incorporated. Singh/Salganik further teaches a method comprising:
determining a first performance parameter for the updated industrial machine-learning operations model exceeds a second performance parameter for the industrial machine-learning operations model (“drift of the models can be measured side-by-side to see whether the new version of the AI/ML model is performing better on real data. In certain embodiments, the new version of the AI/ML model is not deployed unless it meets these thresholds,” Singh paragraph 0068 lines 6-10); and
providing, to the one or more computing devices, the updated industrial machine-learning operations model (“when the retrained AI/ML model meets certain thresholds, it could be automatically deployed alongside the existing AI/ML model,” Singh paragraph 0068 lines 4-6).
As to dependent claim 7, the rejection of claim 6 is incorporated. Singh/Salganik further teaches a method wherein determining the first performance parameter for the updated industrial machine-learning operations model exceeds the second performance parameter for the industrial machine-learning operations model comprises comparing a first output of the updated industrial machine-learning operations model utilizing an exemplary data set and a second output of the industrial machine-learning operations model utilizing the exemplary data set (“drift of the models can be measured side-by-side to see whether the new version of the AI/ML model is performing better on real data. In certain embodiments, the new version of the AI/ML model is not deployed unless it meets these thresholds,” Singh paragraph 0068 lines 6-10).
As to dependent claim 8, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein drift parameters comprise weighted drift parameters, and wherein determining the respective retraining criteria is met by at least one of the drift parameters comprises determining that a weighted retraining criteria is met by the weighted drift parameters (“When a change condition is found (e.g., predictions or input data fall outside of a historical range),” Singh paragraph 0067 lines 3-5).
As to dependent claim 9, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein the data drift includes metadata drift (“the distance between the unidimensional distributions of an original dataset and a new dataset can be measured for covariate drift,” Singh paragraph 0023 lines 3-6).
As to dependent claim 10, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein meeting the respective retraining criteria for each drift parameter of the drift parameters depends in part on the type of observable drift of the drift parameter (“When a change condition is found (e.g., predictions or input data fall outside of a historical range),” Singh paragraph 0067 lines 3-5).
As to dependent claim 11, the rejection of claim 10 is incorporated. Singh/Salganik further teaches a method wherein the respective retraining criteria is met by at least two of the drift parameters (When two criteria are met at the same time, either one will trigger retraining.).
As to dependent claim 12, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein triggering the update comprises providing an alert to initiate (“After this information has been collected, the information is analyzed to determine whether data drift, model drift, or both, have occurred. When a change condition is found (e.g., predictions or input data fall outside of a historical range) and/or a change threshold is met or exceeded (e.g., model statistical performance deviates from historical performance by at least a certain amount), an alert or a retraining trigger is generated,” Singh paragraph 0067 lines 1-8) a retraining pipeline (“the AI/ML model pipeline,” Singh paragraph 0021 line 8).
As to dependent claim 13, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein triggering the update comprises triggering an automatic retraining of the industrial machine-learning operations model (“After this information has been collected, the information is analyzed to determine whether data drift, model drift, or both, have occurred. When a change condition is found (e.g., predictions or input data fall outside of a historical range) and/or a change threshold is met or exceeded (e.g., model statistical performance deviates from historical performance by at least a certain amount), an alert or a retraining trigger is generated,” Singh paragraph 0067 lines 1-8).
As to dependent claim 14, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein determining the drift parameters based on usage drift comprises determining a frequency of utilization of the industrial machine-learning operations model by the one or more computing devices over a first period of time, and wherein the respective retraining criteria for the drift parameter based on the usage drift comprises a minimum threshold usage of the industrial machine-learning operations model for a second period of time (“When a change condition is found (e.g., predictions or input data fall outside of a historical range),” Singh paragraph 0067 lines 3-5).
As to dependent claim 15, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein determining the drift parameters based on performance drift comprises determining a compute time for the industrial machine-learning operations model on available hardware of the one or more computing devices, and wherein the respective retraining criteria for the drift parameters based on the performance drift comprises a deviation of the compute time from an average compute time for the industrial machine-learning operations model on the available hardware of the one or more computing devices (“When a change condition is found (e.g., predictions or input data fall outside of a historical range),” Singh paragraph 0067 lines 3-5).
As to dependent claim 16, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein monitoring data comprises prediction data, and wherein determining the drift parameters based on data drift comprises determining a deviation of the prediction data generated utilizing the industrial machine-learning operations model from training data utilized to train the industrial machine-learning operations model (“When a change condition is found (e.g., predictions or input data fall outside of a historical range),” Singh paragraph 0067 lines 3-5).
As to dependent claim 17, the rejection of claim 16 is incorporated. Singh/Salganik further teaches a method wherein determining the drift parameters based on prediction drift comprises determining an accuracy in the prediction data is below a threshold prediction accuracy (“When a change condition is found (e.g., predictions or input data fall outside of a historical range),” Singh paragraph 0067 lines 3-5).
As to dependent claim 18, the rejection of claim 1 is incorporated. Singh/Salganik further teaches a method wherein triggering the update comprises providing an alert to a user; and in response to receiving a confirmation from the user to initiate a retraining pipeline, initiating the retraining pipeline (“the trigger may cause predictions made by the AI/ML model and/or the input data to be transmitted and stored for a human reviewer to analyze. An action center queue may allow a human to validate and automatically retrain a new version of the model,” Singh paragraph 0026 lines 2-7).
As to independent claim 19, Singh teaches a system for updating an industrial machine-learning operations model, the system comprising:
one or more computers (figure 5) and one or more storage devices (figure 5 part 515 “Memory”) on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving, from one or more computing devices, monitoring data for an industrial machine-learning operations model (“information is obtained with respect to AI/ML model performance. … This information may assist in determining how well an AI/ML model is performing over time,” paragraph 0066 lines 1-2, 10-12);
determining, from the monitoring data, to retrain the industrial machine-learning operations model, the determining comprising computing drift parameters, each of the drift parameters being indicative of a type of observable drift of the industrial machine-learning operations model, wherein the drift parameters comprise (iii) a data drift, and (iv) a prediction drift, and wherein each drift parameter includes respective retraining criteria (“After this information has been collected, the information is analyzed to determine whether data drift, model drift, or both, have occurred. When a change condition is found (e.g., predictions or input data fall outside of a historical range) and/or a change threshold is met or exceeded (e.g., model statistical performance deviates from historical performance by at least a certain amount), an alert or a retraining trigger is generated,” paragraph 0067 lines 1-8), and
confirming, from the drift parameters, the respective retraining criteria is met by at least one of the drift parameters (“After this information has been collected, the information is analyzed to determine whether data drift, model drift, or both, have occurred. When a change condition is found (e.g., predictions or input data fall outside of a historical range) and/or a change threshold is met or exceeded (e.g., model statistical performance deviates from historical performance by at least a certain amount), an alert or a retraining trigger is generated,” paragraph 0067 lines 1-8); and
triggering, in response to the determining to retrain the industrial machine-learning operations model, an update of the industrial machine-learning operations model (“The AI/ML model can then be retrained using the collected information and/or other information to attempt to improve AI/ML model performance,” paragraph 0068 lines 1-3).
Singh does not appear to expressly teach a system wherein the drift parameters comprise (i) a usage drift, and (ii) a performance drift.
Salganik teaches a system wherein the drift parameters comprise (i) a usage drift, and (ii) a performance drift (“these systems change in three main ways: population drift (change in who is using them), behavioral drift (change in how people are using them), and system drift (change in the system itself),” section 2.3.7 “Drifting” paragraph 2 lines 2-4).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the drift parameters of Singh to comprise the usage drift and performance drift of Salganik. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely monitoring usage drift and performance drift (“these systems change in three main ways: population drift (change in who is using them), behavioral drift (change in how people are using them), and system drift (change in the system itself),” section 2.3.7 “Drifting” paragraph 2 lines 2-4). Therefore, the rationale to support a conclusion that the claim would have been obvious is that the combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to independent claim 20, Singh teaches one or more non-transitory computer storage media (figure 5 part 515 “Memory”) encoded with computer program instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:
receiving, from one or more computing devices, monitoring data for industrial machine-learning operations model (“information is obtained with respect to AI/ML model performance. … This information may assist in determining how well an AI/ML model is performing over time,” paragraph 0066 lines 1-2, 10-12);
determining, from the monitoring data, to retrain the industrial machine-learning operations model, the determining comprising computing drift parameters, each of the drift parameters being indicative of a type of observable drift of the industrial machine-learning operations model, wherein the drift parameters comprise (iii) a data drift, and (iv) a prediction drift, and wherein each drift parameter includes respective retraining criteria (“After this information has been collected, the information is analyzed to determine whether data drift, model drift, or both, have occurred. When a change condition is found (e.g., predictions or input data fall outside of a historical range) and/or a change threshold is met or exceeded (e.g., model statistical performance deviates from historical performance by at least a certain amount), an alert or a retraining trigger is generated,” paragraph 0067 lines 1-8), and
confirming, from the drift parameters, the respective retraining criteria is met by at least one of the drift parameters (“After this information has been collected, the information is analyzed to determine whether data drift, model drift, or both, have occurred. When a change condition is found (e.g., predictions or input data fall outside of a historical range) and/or a change threshold is met or exceeded (e.g., model statistical performance deviates from historical performance by at least a certain amount), an alert or a retraining trigger is generated,” paragraph 0067 lines 1-8); and
triggering, in response to the determining to retrain the industrial machine-learning operations model, an update of the industrial machine-learning operations model (“The AI/ML model can then be retrained using the collected information and/or other information to attempt to improve AI/ML model performance,” paragraph 0068 lines 1-3).
Singh does not appear to expressly teach a media wherein the drift parameters comprise (i) a usage drift, and (ii) a performance drift.
Salganik teaches a media wherein the drift parameters comprise (i) a usage drift, and (ii) a performance drift (“these systems change in three main ways: population drift (change in who is using them), behavioral drift (change in how people are using them), and system drift (change in the system itself),” section 2.3.7 “Drifting” paragraph 2 lines 2-4).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the drift parameters of Singh to comprise the usage drift and performance drift of Salganik. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely monitoring usage drift and performance drift (“these systems change in three main ways: population drift (change in who is using them), behavioral drift (change in how people are using them), and system drift (change in the system itself),” section 2.3.7 “Drifting” paragraph 2 lines 2-4). Therefore, the rationale to support a conclusion that the claim would have been obvious is that the combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant’s disclosure:
US 2016/0371601 A1 disclosing drift-triggered retraining
Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
In the interests of compact prosecution, Applicant is invited to contact the examiner via electronic media pursuant to USPTO policy outlined MPEP § 502.03. All electronic communication must be authorized in writing. Applicant may wish to file an Internet Communications Authorization Form PTO/SB/439. Applicant may wish to request an interview using the Interview Practice website: http://www.uspto.gov/patent/laws-and-regulations/interview-practice.
Applicant is reminded Internet e-mail may not be used for communication for matters under 35 U.S.C. § 132 or which otherwise require a signature. A reply to an Office action may NOT be communicated by Applicant to the USPTO via Internet e-mail. If such a reply is submitted by Applicant via Internet e-mail, a paper copy will be placed in the appropriate patent application file with an indication that the reply is NOT ENTERED. See MPEP § 502.03(II).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan Barrett whose telephone number is 571 270 3311. The examiner can normally be reached 9:00am to 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Michelle Bechtold can be reached at 571 431 0762. The fax phone number for the organization where this application or proceeding is assigned is 571 273 8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ryan Barrett/
Primary Examiner, Art Unit 2148
Prosecution Insights