DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is responsive to the Application filed on 2/15/2023. Claims 1-20 are pending in the case. Claims 1, 8, and 15 are independent claims.
Claim Rejections - 35 U.S.C. § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
As to claim 1:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a manufacture.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “identify a plurality of data samples of a data set” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “in response to a trigger, identify a first subset of samples of the plurality of data samples that are outside a range associated with a decision boundary of a machine learning model” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “remove the first subset of samples from the data set to generate a modified training set” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “at least one computer readable storage medium comprising a set of instructions, which when executed by a computing device, cause the computing device to” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
No, the limitation “train the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “train the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “at least one computer readable storage medium comprising a set of instructions, which when executed by a computing device, cause the computing device to” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
No, the limitation “train the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “train the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 2:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a manufacture.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “categorize a first sample of the first subset of samples as being a confident sample based on a label of the first sample and a classification of the first sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “group the first sample into the first subset based on the first sample being categorized as being the confident sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 3:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a manufacture.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein to categorize the first sample of the first subset of samples as being the confident sample, [] compare the label to the classification to determine that the label matches the classification” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 4:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a manufacture.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “categorize a first sample of the first subset of samples as being a suspicious sample based on a label of the first sample and a classification of the first sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “group the first sample into the first subset based on the first sample being categorized as being the suspicious sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 5:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a manufacture.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein to categorize the first sample as being the suspicious sample, [] compare the label to the classification to determine that the label does not match the classification” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 6:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a manufacture.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determine an accuracy of the machine learning model during the processing of the set of samples” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “set the trigger based on the accuracy exceeding a threshold” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “process a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “process a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “process a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “process a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 7:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a manufacture.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “bypass a bypass sample of the plurality of data samples from being grouped into the first subset of samples based on a classification probability of the bypass sample being within the range of the decision boundary” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 8:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “identify a plurality of data samples of a data set” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “in response to a trigger, identify a first subset of samples of the plurality of data samples that are outside a range associated with a decision boundary of a machine learning model” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “remove the first subset of samples from the data set to generate a modified training set” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “one or more processors” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
No, the limitation “a memory coupled to the one or more processors, the memory comprising instructions executable by the one or more processors, the one or more processors being operable when executing the instructions to” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §§ 2106.04(d), 2106.05(h).
No, the limitation “train the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “train the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “one or more processors” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
No, the limitation “a memory coupled to the one or more processors, the memory comprising instructions executable by the one or more processors, the one or more processors being operable when executing the instructions to” is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP § 2106.05(h).
No, the limitation “train the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “train the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 9:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “categorize a first sample of the first subset of samples as being a confident sample based on a label of the first sample and a classification of the first sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “group the first sample into the first subset based on the first sample being categorized as being the confident sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 10:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein to categorize the first sample of the first subset of samples as being the confident sample, [] compare the label to the classification to determine that the label matches the classification” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 11:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “categorize a first sample of the first subset of samples as being a suspicious sample based on a label of the first sample and a classification of the first sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “group the first sample into the first subset based on the first sample being categorized as being the suspicious sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 12:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein to categorize the first sample as being the suspicious sample, [] compare the label to the classification to determine that the label does not match the classification” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 13:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determine an accuracy of the machine learning model during the processing of the set of samples” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “set the trigger based on the accuracy exceeding a threshold” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “process a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “process a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “process a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “process a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 14:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a machine.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “bypass a bypass sample of the plurality of data samples from being grouped into the first subset of samples based on a classification probability of the bypass sample being within the range of the decision boundary” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 15:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “identifying a plurality of data samples of a data set” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “in response to a trigger, identifying a first subset of samples of the plurality of data samples that are outside a range associated with a decision boundary of a machine learning model” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “removing the first subset of samples from the data set to generate a modified training set” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “training the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “training the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “training the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “training the machine learning model based on the modified training set” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
As to claim 16:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “categorizing a first sample of the first subset of samples as being a confident sample based on a label of the first sample and a classification of the first sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “grouping the first sample into the first subset based on the first sample being categorized as being the confident sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 17:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein the categorizing the first sample further comprises comparing the label to the classification to determine that the label matches the classification” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 18:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “categorizing a first sample of the first subset of samples as being a suspicious sample based on a label of the first sample and a classification of the first sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “grouping the first sample into the first subset based on the first sample being categorized as being the suspicious sample” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 19:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “wherein the categorizing the first sample as being the suspicious sample comprises comparing the label to the classification to determine that the label does not match the classification” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
The analysis of the parent claim is incorporated.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
The analysis of the parent claim is incorporated.
As to claim 20:
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Yes, the claim is to a process.
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Yes, the limitation “determining an accuracy of the machine learning model during the processing of the set of samples” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “setting the trigger based on the accuracy exceeding a threshold” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Yes, the limitation “bypassing a bypass sample of the plurality of data samples from being grouped into the first subset of samples based on a classification probability of the bypass sample being within the range of the decision boundary” is the abstract idea of a mental process that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
No, the limitation “processing a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP §§ 2106.04(d), 2106.05(f)(1).
No, the limitation “processing a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §§ 2106.04(d), 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to integrate the judicial exception into a practical application.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
No, the limitation “processing a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer. See MPEP § 2106.05(f)(1).
No, the limitation “processing a set of samples with the machine learning model” is an additional element that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP § 2106.05(f)(2).
The additional elements, taken alone or in combination, fail to amount to significantly more than the judicial exception.
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. § 103 as being unpatentable over Swaroop et al. (US 2024/0211798 A1, hereinafter Swaroop) in view of Northcutt et al. (“Learning with Confident Examples: Rank Pruning for Robust Classification with Noisy Labels,” 9 August 2017, https://arxiv.org/abs/1705.01936, hereinafter Northcutt).
As to independent claim 1, Swaroop teaches at least one computer readable storage medium (“Storage Device,” figure 6 part 630) comprising a set of instructions, which when executed by a computing device, cause the computing device to:
identify a plurality of data samples of a data set (“At 508, updating the training datasets can include reannotating/relabeling existing or new training datasets to consolidate with an existing training dataset,” paragraph 0065 lines 17-20);
in response to a trigger (“At 506, a retraining of the MLOPs model is triggered, e.g., by model health monitoring engine 104, in response to retraining criteria 128 being met by one or more computed drift parameters,” paragraph 0065 lines 12-16),
identify a first subset of samples of the plurality of data samples [] (“At 510, reannotated/relabeled training datasets are split (e.g., divided) into three categories: training dataset 512, validation dataset 514, and out of sample (OOS) test dataset 516,” paragraph 0065 lines 20-23); and
remove the first subset of samples from the data set to generate a modified training set (“At 510, reannotated/relabeled training datasets are split (e.g., divided) into three categories: training dataset 512, validation dataset 514, and out of sample (OOS) test dataset 516,” paragraph 0065 lines 20-23); and
train the machine learning model based on the modified training set (“At 515, the MLOPs model retraining is performed by training the model on the training dataset 512 and the validation dataset 514,” paragraph 0065 lines 32-34).
Swaroop does not appear to expressly teach a medium comprising instructions to:
identify a first subset of samples of the plurality of data samples that are outside a range associated with a decision boundary of a machine learning model.
Northcutt teaches a medium comprising instructions to:
identify a first subset of samples of the plurality of data samples that are outside a range associated with a decision boundary of a machine learning model (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” paragraph 3 lines 1-2); and
remove the first subset of samples from the data set to generate a modified training set (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” paragraph 3 lines 1-2).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the subset of Swaroop to comprise the range of Northcutt. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely pruning range (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” Northcutt paragraph 3 lines 1-2). Therefore, the rationale to support a conclusion that the claim would have been obvious is that the combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 2, the rejection of claim 1 is incorporated. Swaroop/Northcutt further teaches a medium wherein the instructions, when executed, cause the computing device to:
categorize a first sample of the first subset of samples as being a confident sample based on a label of the first sample and a classification of the first sample (“we first find two thresholds, LBy=1 and UBy=0, to confidently guess the correctly and incorrectly labeled examples in each of
P
~
and
N
~
, forming four sets,” Northcutt page 8 section “3.4 Rank Pruning: A simple summary” paragraph 1 lines 3-5); and
group the first sample into the first subset based on the first sample being categorized as being the confident sample (“remove that number of examples from
P
~
by removing those with lowest predicted probability g(x). We prune
N
~
similarly.,” Northcutt page 8 section “3.4 Rank Pruning: A simple summary” paragraph 1 lines 6-7).
As to dependent claim 3, the rejection of claim 2 is incorporated. Swaroop/Northcutt further teaches a medium wherein to categorize the first sample of the first subset of samples as being the confident sample, the instructions, when executed, cause the computing device to:
compare the label to the classification to determine that the label matches the classification (“count the number of examples with label s = 0 that we are ‘confident’ have label y = 1,” Northcutt page 5 section “3.1 Deriving Noise Rate Estimators
p
^
1
c
o
n
f
and
p
^
0
c
o
n
f
” paragraph 1 lines 3).
As to dependent claim 4, the rejection of claim 1 is incorporated. Swaroop/Northcutt further teaches a medium wherein the instructions, when executed, cause the computing device to:
categorize a first sample of the first subset of samples as being a suspicious sample based on a label of the first sample and a classification of the first sample (“count the number of examples with label s = 0 that we are ‘confident’ have label y = 1,” Northcutt page 5 section “3.1 Deriving Noise Rate Estimators
p
^
1
c
o
n
f
and
p
^
0
c
o
n
f
” paragraph 1 lines 3); and
group the first sample into the first subset based on the first sample being categorized as being the suspicious sample (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” Northcutt paragraph 3 lines 1-2).
As to dependent claim 5, the rejection of claim 4 is incorporated. Swaroop/Northcutt further teaches a medium wherein to categorize the first sample as being the suspicious sample, the instructions, when executed, cause the computing device to:
compare the label to the classification to determine that the label does not match the classification (“count the number of examples with label s = 0 that we are ‘confident’ have label y = 1,” Northcutt page 5 section “3.1 Deriving Noise Rate Estimators
p
^
1
c
o
n
f
and
p
^
0
c
o
n
f
” paragraph 1 lines 3).
As to dependent claim 6, the rejection of claim 1 is incorporated. Swaroop/Northcutt further teaches a medium wherein the instructions, when executed, cause the computing device to:
process a set of samples with the machine learning model (“At 502, A MLOPs model deployed in a production environment, e.g., production environment 114, is monitored,” Swaroop paragraph 0065 lines 8-10);
determine an accuracy of the machine learning model during the processing of the set of samples (“At 506, a retraining of the MLOPs model is triggered, e.g., by model health monitoring engine 104, in response to retraining criteria 128 being met by one or more computed drift parameters,” Swaroop paragraph 0065 lines 12-16); and
set the trigger based on the accuracy exceeding a threshold (“At 506, a retraining of the MLOPs model is triggered, e.g., by model health monitoring engine 104, in response to retraining criteria 128 being met by one or more computed drift parameters,” Swaroop paragraph 0065 lines 12-16).
As to dependent claim 7, the rejection of claim 1 is incorporated. Swaroop/Northcutt further teaches a medium wherein the instructions, when executed, cause the computing device to:
bypass a bypass sample of the plurality of data samples from being grouped into the first subset of samples based on a classification probability of the bypass sample being within the range of the decision boundary (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” Northcutt paragraph 3 lines 1-2).
As to independent claim 8, Swaroop teaches a system comprising:
one or more processors (“Processor,” figure 6 part 610); and
a memory (“Storage Device,” figure 6 part 630) coupled to the one or more processors, the memory comprising instructions executable by the one or more processors, the one or more processors being operable when executing the instructions to:
identify a plurality of data samples of a data set (“At 508, updating the training datasets can include reannotating/relabeling existing or new training datasets to consolidate with an existing training dataset,” paragraph 0065 lines 17-20);
in response to a trigger (“At 506, a retraining of the MLOPs model is triggered, e.g., by model health monitoring engine 104, in response to retraining criteria 128 being met by one or more computed drift parameters,” paragraph 0065 lines 12-16),
identify a first subset of samples of the plurality of data samples [] (“At 510, reannotated/relabeled training datasets are split (e.g., divided) into three categories: training dataset 512, validation dataset 514, and out of sample (OOS) test dataset 516,” paragraph 0065 lines 20-23); and
remove the first subset of samples from the data set to generate a modified training set (“At 510, reannotated/relabeled training datasets are split (e.g., divided) into three categories: training dataset 512, validation dataset 514, and out of sample (OOS) test dataset 516,” paragraph 0065 lines 20-23); and
train the machine learning model based on the modified training set (“At 515, the MLOPs model retraining is performed by training the model on the training dataset 512 and the validation dataset 514,” paragraph 0065 lines 32-34).
Swaroop does not appear to expressly teach a system comprising:
identify a first subset of samples of the plurality of data samples that are outside a range associated with a decision boundary of a machine learning model.
Northcutt teaches a system comprising:
identify a first subset of samples of the plurality of data samples that are outside a range associated with a decision boundary of a machine learning model (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” paragraph 3 lines 1-2); and
remove the first subset of samples from the data set to generate a modified training set (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” paragraph 3 lines 1-2).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the subset of Swaroop to comprise the range of Northcutt. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely pruning range (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” Northcutt paragraph 3 lines 1-2). Therefore, the rationale to support a conclusion that the claim would have been obvious is that the combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 9, the rejection of claim 8 is incorporated. Swaroop/Northcutt further teaches a system wherein the one or more processors are further operable when executing the instructions to:
categorize a first sample of the first subset of samples as being a confident sample based on a label of the first sample and a classification of the first sample (“we first find two thresholds, LBy=1 and UBy=0, to confidently guess the correctly and incorrectly labeled examples in each of
P
~
and
N
~
, forming four sets,” Northcutt page 8 section “3.4 Rank Pruning: A simple summary” paragraph 1 lines 3-5); and
group the first sample into the first subset based on the first sample being categorized as being the confident sample (“remove that number of examples from
P
~
by removing those with lowest predicted probability g(x). We prune
N
~
similarly.,” Northcutt page 8 section “3.4 Rank Pruning: A simple summary” paragraph 1 lines 6-7).
As to dependent claim 10, the rejection of claim 9 is incorporated. Swaroop/Northcutt further teaches a system wherein to categorize the first sample of the first subset of samples as being the confident sample, the one or more processors are further operable when executing the instructions to:
compare the label to the classification to determine that the label matches the classification (“count the number of examples with label s = 0 that we are ‘confident’ have label y = 1,” Northcutt page 5 section “3.1 Deriving Noise Rate Estimators
p
^
1
c
o
n
f
and
p
^
0
c
o
n
f
” paragraph 1 lines 3).
As to dependent claim 11, the rejection of claim 8 is incorporated. Swaroop/Northcutt further teaches a system wherein the one or more processors are further operable when executing the instructions to:
categorize a first sample of the first subset of samples as being a suspicious sample based on a label of the first sample and a classification of the first sample (“count the number of examples with label s = 0 that we are ‘confident’ have label y = 1,” Northcutt page 5 section “3.1 Deriving Noise Rate Estimators
p
^
1
c
o
n
f
and
p
^
0
c
o
n
f
” paragraph 1 lines 3); and
group the first sample into the first subset based on the first sample being categorized as being the suspicious sample (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” Northcutt paragraph 3 lines 1-2).
As to dependent claim 12, the rejection of claim 11 is incorporated. Swaroop/Northcutt further teaches a system wherein to categorize the first sample as being the suspicious sample, the one or more processors are further operable when executing the instructions to:
compare the label to the classification to determine that the label does not match the classification (“count the number of examples with label s = 0 that we are ‘confident’ have label y = 1,” Northcutt page 5 section “3.1 Deriving Noise Rate Estimators
p
^
1
c
o
n
f
and
p
^
0
c
o
n
f
” paragraph 1 lines 3).
As to dependent claim 13, the rejection of claim 8 is incorporated. Swaroop/Northcutt further teaches a system wherein the one or more processors are further operable when executing the instructions to:
process a set of samples with the machine learning model (“At 502, A MLOPs model deployed in a production environment, e.g., production environment 114, is monitored,” Swaroop paragraph 0065 lines 8-10);
determine an accuracy of the machine learning model during the processing of the set of samples (“At 506, a retraining of the MLOPs model is triggered, e.g., by model health monitoring engine 104, in response to retraining criteria 128 being met by one or more computed drift parameters,” Swaroop paragraph 0065 lines 12-16); and
set the trigger based on the accuracy exceeding a threshold (“At 506, a retraining of the MLOPs model is triggered, e.g., by model health monitoring engine 104, in response to retraining criteria 128 being met by one or more computed drift parameters,” Swaroop paragraph 0065 lines 12-16).
As to dependent claim 14, the rejection of claim 8 is incorporated. Swaroop/Northcutt further teaches a system wherein the one or more processors are further operable when executing the instructions to:
bypass a bypass sample of the plurality of data samples from being grouped into the first subset of samples based on a classification probability of the bypass sample being within the range of the decision boundary (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” Northcutt paragraph 3 lines 1-2).
As to independent claim 15 Swaroop teaches a method comprising:
identifying a plurality of data samples of a data set (“At 508, updating the training datasets can include reannotating/relabeling existing or new training datasets to consolidate with an existing training dataset,” paragraph 0065 lines 17-20);
in response to a trigger (“At 506, a retraining of the MLOPs model is triggered, e.g., by model health monitoring engine 104, in response to retraining criteria 128 being met by one or more computed drift parameters,” paragraph 0065 lines 12-16),
identifying a first subset of samples of the plurality of data samples [] (“At 510, reannotated/relabeled training datasets are split (e.g., divided) into three categories: training dataset 512, validation dataset 514, and out of sample (OOS) test dataset 516,” paragraph 0065 lines 20-23); and
removing the first subset of samples from the data set to generate a modified training set (“At 510, reannotated/relabeled training datasets are split (e.g., divided) into three categories: training dataset 512, validation dataset 514, and out of sample (OOS) test dataset 516,” paragraph 0065 lines 20-23); and
training the machine learning model based on the modified training set (“At 515, the MLOPs model retraining is performed by training the model on the training dataset 512 and the validation dataset 514,” paragraph 0065 lines 32-34).
Swaroop does not appear to expressly teach a method comprising:
identifying a first subset of samples of the plurality of data samples that are outside a range associated with a decision boundary of a machine learning model.
Northcutt teaches a method comprising:
identifying a first subset of samples of the plurality of data samples that are outside a range associated with a decision boundary of a machine learning model (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” paragraph 3 lines 1-2); and
removing the first subset of samples from the data set to generate a modified training set (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” paragraph 3 lines 1-2).
Accordingly, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the subset of Swaroop to comprise the range of Northcutt. (1) The Examiner finds that the prior art included each claim element listed above, although not necessarily in a single prior art reference, with the only difference between the claimed invention and the prior art being the lack of actual combination of the elements in a single prior art reference. (2) The Examiner finds that one of ordinary skill in the art could have combined the elements as claimed by known software development methods, and that in combination, each element merely performs the same function as it does separately. (3) The Examiner finds that one of ordinary skill in the art would have recognized that the results of the combination were predictable, namely pruning range (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” Northcutt paragraph 3 lines 1-2). Therefore, the rationale to support a conclusion that the claim would have been obvious is that the combining prior art elements according to known methods to yield predictable results to one of ordinary skill in the art. See MPEP § 2143(I)(A).
As to dependent claim 16, the rejection of claim 15 is incorporated. Swaroop/Northcutt further teaches a method comprising:
categorizing a first sample of the first subset of samples as being a confident sample based on a label of the first sample and a classification of the first sample (“we first find two thresholds, LBy=1 and UBy=0, to confidently guess the correctly and incorrectly labeled examples in each of
P
~
and
N
~
, forming four sets,” Northcutt page 8 section “3.4 Rank Pruning: A simple summary” paragraph 1 lines 3-5); and
grouping the first sample into the first subset based on the first sample being categorized as being the confident sample (“remove that number of examples from
P
~
by removing those with lowest predicted probability g(x). We prune
N
~
similarly.,” Northcutt page 8 section “3.4 Rank Pruning: A simple summary” paragraph 1 lines 6-7).
As to dependent claim 17, the rejection of claim 16 is incorporated. Swaroop/Northcutt further teaches a method comprising:
comparing the label to the classification to determine that the label matches the classification (“count the number of examples with label s = 0 that we are ‘confident’ have label y = 1,” Northcutt page 5 section “3.1 Deriving Noise Rate Estimators
p
^
1
c
o
n
f
and
p
^
0
c
o
n
f
” paragraph 1 lines 3).
As to dependent claim 18, the rejection of claim 15 is incorporated. Swaroop/Northcutt further teaches a method comprising:
categorizing a first sample of the first subset of samples as being a suspicious sample based on a label of the first sample and a classification of the first sample (“count the number of examples with label s = 0 that we are ‘confident’ have label y = 1,” Northcutt page 5 section “3.1 Deriving Noise Rate Estimators
p
^
1
c
o
n
f
and
p
^
0
c
o
n
f
” paragraph 1 lines 3); and
grouping the first sample into the first subset based on the first sample being categorized as being the suspicious sample (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” Northcutt paragraph 3 lines 1-2).
As to dependent claim 19, the rejection of claim 18 is incorporated. Swaroop/Northcutt further teaches a method wherein the categorizing the first sample as being the suspicious sample comprises:
comparing the label to the classification to determine that the label does not match the classification (“count the number of examples with label s = 0 that we are ‘confident’ have label y = 1,” Northcutt page 5 section “3.1 Deriving Noise Rate Estimators
p
^
1
c
o
n
f
and
p
^
0
c
o
n
f
” paragraph 1 lines 3).
As to dependent claim 20, the rejection of claim 15 is incorporated. Swaroop/Northcutt further teaches a method comprising:
processing a set of samples with the machine learning model (“At 502, A MLOPs model deployed in a production environment, e.g., production environment 114, is monitored,” Swaroop paragraph 0065 lines 8-10);
determining an accuracy of the machine learning model during the processing of the set of samples (“At 506, a retraining of the MLOPs model is triggered, e.g., by model health monitoring engine 104, in response to retraining criteria 128 being met by one or more computed drift parameters,” Swaroop paragraph 0065 lines 12-16); and
setting the trigger based on the accuracy exceeding a threshold (“At 506, a retraining of the MLOPs model is triggered, e.g., by model health monitoring engine 104, in response to retraining criteria 128 being met by one or more computed drift parameters,” Swaroop paragraph 0065 lines 12-16).
bypassing a bypass sample of the plurality of data samples from being grouped into the first subset of samples based on a classification probability of the bypass sample being within the range of the decision boundary (“we prune the
π
^
1
|
P
~
|
examples in
P
~
with smallest g(x) and the
π
^
0
|
N
~
|
examples in
N
~
with highest g(x) and denote the pruned sets
P
~
c
o
n
f
and
N
~
c
o
n
f
,” page 7 section “3.3 The Rank Pruning Algorithm” Northcutt paragraph 3 lines 1-2).
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant’s disclosure:
US 2015/0379428 A1 disclosing ranked pruning of data sets to train machine learning model models
Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
In the interests of compact prosecution, Applicant is invited to contact the examiner via electronic media pursuant to USPTO policy outlined MPEP § 502.03. All electronic communication must be authorized in writing. Applicant may wish to file an Internet Communications Authorization Form PTO/SB/439. Applicant may wish to request an interview using the Interview Practice website: http://www.uspto.gov/patent/laws-and-regulations/interview-practice.
Applicant is reminded Internet e-mail may not be used for communication for matters under 35 U.S.C. § 132 or which otherwise require a signature. A reply to an Office action may NOT be communicated by Applicant to the USPTO via Internet e-mail. If such a reply is submitted by Applicant via Internet e-mail, a paper copy will be placed in the appropriate patent application file with an indication that the reply is NOT ENTERED. See MPEP § 502.03(II).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ryan Barrett whose telephone number is 571 270 3311. The examiner can normally be reached 9:00am to 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Michelle Bechtold can be reached at 571 431 0762. The fax phone number for the organization where this application or proceeding is assigned is 571 273 8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Ryan Barrett/
Primary Examiner, Art Unit 2148