DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities:
Paragraph [0027] said “subset N” versus “Subset N 125” in Fig. 1. In order to be consistent, both recitations need to be the same.
Paragraph [0033] said “predictions 3550” versus “Predictions 355” in Fig. 3. In order to be consistent, both recitations need to be the same.
Paragraph [0034] said “(APIs) 410” versus “API(s) 410” in Fig. 4. In order to be consistent, both recitations need to be the same.
Paragraph [0047] “the type of CPUs” is written twice. Removal of one of the recitations is required.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-20 are rejected under 35 U.S.C. 101 for containing an abstract idea without significantly more.
Regarding Claim 1:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a process.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural
phenomenon? - Yes, the claim recites the abstract ideas:
labeling, by the computing device and using the one or more machine learning models, the random training set of unlabeled data to produce a pseudo labeled training set: - This limitation is directed to the abstract idea of a mental process, as the process of labeling unlabeled data is a thought process that can be performed in a human mind by observing and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
correcting, by the computing device, the labels on a random subset of the pseudo labeled training set: - This limitation is directed to the abstract idea of a mental process, as the process of correcting the labels is a thought process that can be performed in a human mind by observing and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
evaluating, by the computing device, the accuracy of the one or more machine learning models using an evaluation set of labeled data: - This limitation is directed to the abstract idea of a mental process, as the process of evaluating the accuracy is a thought process that can be performed in a human mind by evaluating and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
receiving, by a computing device, an accuracy target for one or more machine learning models: - This limitation is directed to mere data gathering and outputting. The courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) have recognized mere data gathering and outputting as insignificant extra-solution activity (see MPEP 2106.05(g)(3)) and therefore fails to integrate the exception into a practical application.
training, by the computing device, the one or more machine learning models on a labeled training set of labeled data: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application.
until the accuracy of the one or more machine learning models satisfies the accuracy target: - This limitation is simply manipulating data on iterative manner which does not impose a meaningful limit on the claim, being insignificant as extra solution activity.
sampling, by the computing device, a set of unlabeled data to obtain a random training set of unlabeled data: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application. This limitation is directed to manipulating data by sampling or clustering it together being an insignificant extra solution activity.
training, by the computing device, the one or more machine learning models on the labeled training set, the corrected random subset, and the pseudo labeled training set: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application.
deploying, by the computing device, the one or more machine learning models based at least in part on the accuracy of the one or more machine learning models satisfying the accuracy target based at least in part on the evaluating: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application. Additionally, this limitation is directed to a mathematical concept.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional elements:
receiving, by a computing device, an accuracy target for one or more machine learning models: - This limitation is directed to receiving or transmitting data over a network. The courts (as per Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362) have recognized receiving or transmitting data over a network as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) (se MPEP 2106.05(d) II.).
training, by the computing device, the one or more machine learning models on a labeled training set of labeled data: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
until the accuracy of the one or more machine learning models satisfies the accuracy target: sampling, by the computing device, a set of unlabeled data to obtain a random training set of unlabeled data: - This limitation is directed to performing repetitive calculations (see MPEP 2106.05(d)(II)(ii)) as well as invoking a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
training, by the computing device, the one or more machine learning models on the labeled training set, the corrected random subset, and the pseudo labeled training set: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
deploying, by the computing device, the one or more machine learning models based at least in part on the accuracy of the one or more machine learning models satisfying the accuracy target based at least in part on the evaluating: This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
Regarding Claim 2:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a process.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural
phenomenon? - Yes, the claim is dependent on claim 1 which included an abstract idea (see rejection for
claim 1). Additionally, claim 2 recites the abstract ideas:
wherein the accuracy target comprises a threshold determined based at least in part on a performance of the one or more machine learning models in classifying unlabeled data: - This claim is directed to a mathematical concept, as the process of determining a threshold is a step of “determining” a variable or number using mathematical method (see MPEP 2106.04(a)(2) subsection C).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception.
Regarding Claim 3:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a process.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural
phenomenon? - Yes, the claim is dependent on claim 1 which included an abstract idea (see rejection for
claim 1).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein the labeling comprises ensemble learning with multiple machine learning models: - This limitation amounts to adding the words "apply it" (or an equivalent) with the judicial exception, as the GUI is considered as a generic computer that is used in its ordinary capacity to display data (see MPEP 2106.05(f)) and therefore fails to integrate the exception into a practical application. Additionally, this limitation is directed to mere data gathering and outputting.
Step 2B - Does the claim recite additional elements that amount to significantly more than the
judicial exception? - No, there are no additional elements that amount to significantly more than the
judicial exception. The additional element:
wherein the labeling comprises ensemble learning with multiple machine learning models: - This limitation amounts to adding the words "apply it" (or an equivalent) with the judicial exception, as the GUI is considered as a generic computer that is used in its ordinary capacity to display data. (see MPEP 2106.05(f)) and therefore fails to amount to significantly more than the judicial exception.
Regarding Claim 4:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a process.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural
phenomenon? - Yes, the claim is dependent on claim 3 and claim 3 is dependent on claim 1 which included an abstract idea (see rejection for claim 1).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein ensemble learning comprises at least one of bagging, stacking, or boosting: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of bagging, stacking or boosting (see MPEP 2106.05(h)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
wherein ensemble learning comprises at least one of bagging, stacking, or boosting: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of bagging, stacking or boosting (see MPEP 2106.05(h)).
Regarding Claim 5:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a process.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural
phenomenon? - Yes, the claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein the labeling comprises test time augmentation with at least one of vertical/horizontal flipping, blurring, random cropping, or histogram equalization: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of image editing (see MPEP 2106.05(h)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
wherein the labeling comprises test time augmentation with at least one of vertical/horizontal flipping, blurring, random cropping, or histogram equalization: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of image editing (see MPEP 2106.05(h)).
Regarding Claim 6:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a process.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural
phenomenon? - Yes, the claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). Additionally, claim 6 recites the abstract idea:
wherein the evaluating further comprises: identifying a performance deficiency where the accuracy of the one or more models in classifying a class of unlabeled data is below a threshold: - This limitation is directed to the abstract idea of a mental process, as the process of identifying a performance deficiency where the accuracy of the one or more models in classifying a class of unlabeled data is below a threshold is a thought process that can be performed in a human mind by observing, evaluating and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
augmenting the random subset of the pseudo labeled training set with a targeted subset comprising data from the pseudo labeled training set that are labeled with the class: - This limitation is Selecting a particular data source or type of data to be manipulated as it is merely adding data to the training set, being insignificant extra-solution activity (see MEPEP n2106.05(g)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the
judicial exception? - No, there are no additional elements that amount to significantly more than the
judicial exception. The additional element:
augmenting the random subset of the pseudo labeled training set with a targeted subset comprising data from the pseudo labeled training set that are labeled with the class: - This limitation is analogous to electronic recordkeeping because it’s adding data to the training set and keeping record of it (see MPEP 2106.05(d) II (iii)).
Regarding Claim 7:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a process.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein the labeled training set of labeled data is smaller than the pseudo labeled training set so that a small amount of labeled data can be used to create a larger pseudo labeled training set: - This limitation is Selecting a particular data source or type of data to be manipulated as it is merely adding data to the training set, being insignificant extra-solution activity (see MEPEP n2106.05(g)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the
judicial exception? - No, there are no additional elements that amount to significantly more than the
judicial exception. The additional element:
wherein the labeled training set of labeled data is smaller than the pseudo labeled training set so that a small amount of labeled data can be used to create a larger pseudo labeled training set: - This limitation is analogous to electronic recordkeeping because it’s adding data to the training set and keeping record of it (see MPEP 2106.05(d) II (iii)).
Regarding Claim 8:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim recites the abstract ideas:
labeling, by the computing device and using the one or more machine learning models, the random training set of unlabeled data to produce a pseudo labeled training set: - This limitation is directed to the abstract idea of a mental process, as the process of labeling unlabeled data is a thought process that can be performed in a human mind by observing and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
correcting, by the computing device, the labels on a random subset of the pseudo labeled training set: - This limitation is directed to the abstract idea of a mental process, as the process of correcting the labels is a thought process that can be performed in a human mind by observing and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
evaluating, by the computing device, the accuracy of the one or more machine learning models using an evaluation set of labeled data: - This limitation is directed to the abstract idea of a mental process, as the process of evaluating the accuracy is a thought process that can be performed in a human mind by evaluating and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
receiving, by a computing device, an accuracy target for one or more machine learning models: - This limitation is directed to mere data gathering and outputting. The courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) have recognized mere data gathering and outputting as insignificant extra-solution activity (see MPEP 2106.05(g)(3)) and therefore fails to integrate the exception into a practical application.
training, by the computing device, the one or more machine learning models on a labeled training set of labeled data: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application.
until the accuracy of the one or more machine learning models satisfies the accuracy target: - This limitation is simply manipulating data on iterative manner which does not impose a meaningful limit on the claim, being insignificant as extra solution activity.
sampling, by the computing device, a set of unlabeled data to obtain a random training set of unlabeled data: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application. This limitation is directed to manipulating data by sampling or clustering it together being an insignificant extra solution activity.
training, by the computing device, the one or more machine learning models on the labeled training set, the corrected random subset, and the pseudo labeled training set: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application.
deploying, by the computing device, the one or more machine learning models based at least in part on the accuracy of the one or more machine learning models satisfying the accuracy target based at least in part on the evaluating: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application. Additionally, this limitation is directed to a mathematical concept.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional elements:
receiving, by a computing device, an accuracy target for one or more machine learning models: - This limitation is directed to receiving or transmitting data over a network. The courts (as per Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362) have recognized receiving or transmitting data over a network as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) (se MPEP 2106.05(d) II.).
training, by the computing device, the one or more machine learning models on a labeled training set of labeled data: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
until the accuracy of the one or more machine learning models satisfies the accuracy target: sampling, by the computing device, a set of unlabeled data to obtain a random training set of unlabeled data: - This limitation is directed to performing repetitive calculations (see MPEP 2106.05(d)(II)(ii)) as well as invoking a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
training, by the computing device, the one or more machine learning models on the labeled training set, the corrected random subset, and the pseudo labeled training set: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
deploying, by the computing device, the one or more machine learning models based at least in part on the accuracy of the one or more machine learning models satisfying the accuracy target based at least in part on the evaluating: This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
Regarding Claim 9:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 8 which included an abstract idea (see rejection for claim 8). Additionally, claim 9 recites the abstract ideas:
wherein the accuracy target comprises a threshold determined based at least in part on a performance of the one or more machine learning models in classifying unlabeled data: - This claim is directed to a mathematical concept, as the process of determining a threshold is a step of “determining” a variable or number using mathematical method (see MPEP 2106.04(a)(2) subsection C).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception.
Regarding Claim 10:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 8 which included an abstract idea (see rejection for claim 8).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial
exception into a practical application? - No, there are no additional elements that integrate the judicial
exception into a practical application. The additional element:
wherein the labeling comprises ensemble learning with multiple machine learning models: - This limitation amounts to adding the words "apply it" (or an equivalent) with the judicial exception, as the GUI is considered as a generic computer that is used in its ordinary capacity to display data (see MPEP 2106.05(f)) and therefore fails to integrate the exception into a practical application. Additionally, this limitation is directed to mere data gathering and outputting.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
wherein the labeling comprises ensemble learning with multiple machine learning models: - This limitation amounts to adding the words "apply it" (or an equivalent) with the judicial exception, as the GUI is considered as a generic computer that is used in its ordinary capacity to display data. (see MPEP 2106.05(f)) and therefore fails to amount to significantly more than the judicial exception.
Regarding Claim 11:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of
matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 10 and claim 10 is dependent on claim 8 which included an abstract idea (see rejection for claim 8).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein ensemble learning comprises at least one of bagging, stacking, or boosting: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of bagging, stacking or boosting (see MPEP 2106.05(h)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
wherein ensemble learning comprises at least one of bagging, stacking, or boosting: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of bagging, stacking or boosting (see MPEP 2106.05(h)).
Regarding Claim 12:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 8 which included an abstract idea (see rejection for claim 8).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein the labeling comprises test time augmentation with at least one of vertical/horizontal flipping, blurring, random cropping, or histogram equalization: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of image editing (see MPEP 2106.05(h)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
wherein the labeling comprises test time augmentation with at least one of vertical/horizontal flipping, blurring, random cropping, or histogram equalization: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of image editing (see MPEP 2106.05(h)).
Regarding Claim 13:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 8 which included an abstract idea (see rejection for claim 8). Additionally, claim 13 recites the abstract idea:
wherein the evaluating further comprises: identifying a performance deficiency where the accuracy of the one or more models in classifying a class of unlabeled data is below a threshold: - This limitation is directed to the abstract idea of a mental process, as the process of identifying a performance deficiency where the accuracy of the one or more models in classifying a class of unlabeled data is below a threshold is a thought process that can be performed in a human mind by observing, evaluating and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
augmenting the random subset of the pseudo labeled training set with a targeted subset comprising data from the pseudo labeled training set that are labeled with the class: - This limitation is Selecting a particular data source or type of data to be manipulated as it is merely adding data to the training set, being insignificant extra-solution activity (see MEPEP n2106.05(g)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
augmenting the random subset of the pseudo labeled training set with a targeted subset comprising data from the pseudo labeled training set that are labeled with the class: - This limitation is analogous to electronic recordkeeping because it’s adding data to the training set and keeping record of it (see MPEP 2106.05(d) II (iii)).
Regarding Claim 14:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of
matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 8 which included an abstract idea (see rejection for claim 8).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein the labeled training set of labeled data is smaller than the pseudo labeled training set so that a small amount of labeled data can be used to create a larger pseudo labeled training set: - This limitation is Selecting a particular data source or type of data to be manipulated as it is merely adding data to the training set, being insignificant extra-solution activity (see MEPEP n2106.05(g)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
wherein the labeled training set of labeled data is smaller than the pseudo labeled training set so that a small amount of labeled data can be used to create a larger pseudo labeled training set: - This limitation is analogous to electronic recordkeeping because it’s adding data to the training set and keeping record of it (see MPEP 2106.05(d) II (iii)).
Regarding Claim 15:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim recites the abstract ideas:
labeling, by the computing device and using the one or more machine learning models, the random training set of unlabeled data to produce a pseudo labeled training set: - This limitation is directed to the abstract idea of a mental process, as the process of labeling unlabeled data is a thought process that can be performed in a human mind by observing and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
correcting, by the computing device, the labels on a random subset of the pseudo labeled training set: - This limitation is directed to the abstract idea of a mental process, as the process of correcting the labels is a thought process that can be performed in a human mind by observing and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
evaluating, by the computing device, the accuracy of the one or more machine learning models using an evaluation set of labeled data: - This limitation is directed to the abstract idea of a mental process, as the process of evaluating the accuracy is a thought process that can be performed in a human mind by evaluating and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional elements:
receiving, by a computing device, an accuracy target for one or more machine learning models: - This limitation is directed to mere data gathering and outputting. The courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) have recognized mere data gathering and outputting as insignificant extra-solution activity (see MPEP 2106.05(g)(3)) and therefore fails to integrate the exception into a practical application.
training, by the computing device, the one or more machine learning models on a labeled training set of labeled data: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application.
until the accuracy of the one or more machine learning models satisfies the accuracy target: - This limitation is simply manipulating data on iterative manner which does not impose a meaningful limit on the claim, being insignificant as extra solution activity.
sampling, by the computing device, a set of unlabeled data to obtain a random training set of unlabeled data: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application. This limitation is directed to manipulating data by sampling or clustering it together being an insignificant extra solution activity.
training, by the computing device, the one or more machine learning models on the labeled training set, the corrected random subset, and the pseudo labeled training set: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application.
deploying, by the computing device, the one or more machine learning models based at least in part on the accuracy of the one or more machine learning models satisfying the accuracy target based at least in part on the evaluating: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to integrate the exception into a practical application. Additionally, this limitation is directed to a mathematical concept.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional elements:
receiving, by a computing device, an accuracy target for one or more machine learning models: - This limitation is directed to receiving or transmitting data over a network. The courts (as per Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362) have recognized receiving or transmitting data over a network as well-understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) (se MPEP 2106.05(d) II.).
training, by the computing device, the one or more machine learning models on a labeled training set of labeled data: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
until the accuracy of the one or more machine learning models satisfies the accuracy target: sampling, by the computing device, a set of unlabeled data to obtain a random training set of unlabeled data: - This limitation is directed to performing repetitive calculations (see MPEP 2106.05(d)(II)(ii)) as well as invoking a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
training, by the computing device, the one or more machine learning models on the labeled training set, the corrected random subset, and the pseudo labeled training set: - This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
deploying, by the computing device, the one or more machine learning models based at least in part on the accuracy of the one or more machine learning models satisfying the accuracy target based at least in part on the evaluating: This limitation invokes a computer merely as a tool for performing an existing process (see MPEP 2106.05(f)(2)) and therefore fails to amount to significantly more than the judicial exception.
Regarding Claim 16:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 15 which included an abstract idea (see rejection for claim 15). Additionally, claim 16 recites the abstract ideas:
wherein the accuracy target comprises a threshold determined based at least in part on a performance of the one or more machine learning models in classifying unlabeled data: - This claim is directed to a mathematical concept, as the process of determining a threshold is a step of “determining” a variable or number using mathematical method (see MPEP 2106.04(a)(2) subsection C).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception.
Regarding Claim 17:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of
matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 15 which included an abstract idea (see rejection for claim 15).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein the labeling comprises ensemble learning with multiple machine learning models: - This limitation amounts to adding the words "apply it" (or an equivalent) with the judicial exception, as the GUI is considered as a generic computer that is used in its ordinary capacity to display data (see MPEP 2106.05(f)) and therefore fails to integrate the exception into a practical application. Additionally, this limitation is directed to mere data gathering and outputting.
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
wherein the labeling comprises ensemble learning with multiple machine learning models: - This limitation amounts to adding the words "apply it" (or an equivalent) with the judicial exception, as the GUI is considered as a generic computer that is used in its ordinary capacity to display data. (see MPEP 2106.05(f)) and therefore fails to amount to significantly more than the judicial exception.
Regarding Claim 18:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 17 and claim 17 is dependent on claim 15 which included an abstract idea (see rejection for claim 15).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein ensemble learning comprises at least one of bagging, stacking, or boosting: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of bagging, stacking or boosting (see MPEP 2106.05(h)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
wherein ensemble learning comprises at least one of bagging, stacking, or boosting: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of bagging, stacking or boosting (see MPEP 2106.05(h)).
Regarding Claim 19:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 15 which included an abstract idea (see rejection for claim 15).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
wherein the labeling comprises test time augmentation with at least one of vertical/horizontal flipping, blurring, random cropping, or histogram equalization: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of image editing (see MPEP 2106.05(h)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
wherein the labeling comprises test time augmentation with at least one of vertical/horizontal flipping, blurring, random cropping, or histogram equalization: - This limitation does no more than generally link a judicial exception to a particular technological environment as this limitation recites the use of judicial exception to the technology of image editing (see MPEP 2106.05(h)).
Regarding Claim 20:
Step 1 - Is the claim directed to a process, a process, machine, manufacture or composition of matter? - Yes, the claim is directed to a manufacture.
Step 2A - Prong 1 - Does the claim recite an abstract idea, law of nature, or natural phenomenon? - Yes, the claim is dependent on claim 15 which included an abstract idea (see rejection for claim 15). Additionally, claim 20 recites the abstract idea:
wherein the evaluating further comprises: identifying a performance deficiency where the accuracy of the one or more models in classifying a class of unlabeled data is below a threshold: - This limitation is directed to the abstract idea of a mental process, as the process of identifying a performance deficiency where the accuracy of the one or more models in classifying a class of unlabeled data is below a threshold is a thought process that can be performed in a human mind by observing, evaluating and judging (concepts performed in the human mind (including an observation, evaluation, judgment, opinion) (see MPEP § 2106.04(a)(2), subsection III)).
Step 2A - Prong 2 - Does the claim recite additional elements that integrate the judicial exception into a practical application? - No, there are no additional elements that integrate the judicial exception into a practical application. The additional element:
augmenting the random subset of the pseudo labeled training set with a targeted subset comprising data from the pseudo labeled training set that are labeled with the class: - This limitation is Selecting a particular data source or type of data to be manipulated as it is merely adding data to the training set, being insignificant extra-solution activity (see MEPEP n2106.05(g)).
Step 2B - Does the claim recite additional elements that amount to significantly more than the judicial exception? - No, there are no additional elements that amount to significantly more than the judicial exception. The additional element:
augmenting the random subset of the pseudo labeled training set with a targeted subset comprising data from the pseudo labeled training set that are labeled with the class: - This limitation is analogous to electronic recordkeeping because it’s adding data to the training set and keeping record of it (see MPEP 2106.05(d) II (iii)).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 6-9, 13-16, 20 are rejected under 35 U.S.C. 103 as being unpatentable over Dasgupta et al (US Patent 12182227”- hereinafter Dasgupta) in view of Cascante et al (NPL: "Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning"- hereinafter Cascante).
Referring to Claim 1, Dasgupta teaches a method comprising:
receiving, by a computing device, an accuracy target for one or more machine learning models (see Dasgupta at Column 22 Lines 52 to 56: “Additionally, as shown, this section includes a setting to allow a user to specify whether (and when) the training should be automatically stopped, based on evaluation of the model's accuracy or loss metric over successive training epochs”. Examiner interprets the evaluation of the model’s accuracy or loss metric used as a stop criterion to be equivalent as the claimed “accuracy target”);
training, by the computing device, the one or more machine learning models on a labeled training set of labeled data (see Dasgupta at Column 31 Lines 40 to 43: “As may be understood, in training ML media models, the models are trained with a set of media samples which are labeled by one or more annotations”. Examiner interprets a set of media samples which are labeled to be equivalent as the claimed “a labeled training set of labeled data”);
until the accuracy of the one or more machine learning models satisfies the accuracy target (see Dasgupta at Column 28 Lines 3 to 8: “In some embodiments, the model experiment may perform training runs repeatedly until a stopping condition is reached. The stopping condition may be determined based on evaluation of checkpoints of the model during different points of the training, as discussed in connection with the process shown in FIG. 3”. Examiner interprets the evaluation of the checkpoints as the stopping condition to be equivalent as the claimed “satisfies the accuracy target”);
and evaluating, by the computing device, the accuracy of the one or more machine learning models using an evaluation set of labeled data (see Dasgupta Column 16 Line 21 to 26: “In some embodiments, the MDE may implement a checkpointing feature where, during the training of a ML media model, periodic checkpoints of the model are saved. The periodic checkpoints are then evaluated against an evaluation or validation data set, which is distinct and independent from the training data set”. Examiner interprets the evaluation of the periodic checkpoints against an evaluation or validation data set to be equivalent as the claimed “evaluating”);
and deploying, by the computing device, the one or more machine learning models based at least in part on the accuracy of the one or more machine learning models satisfying the accuracy target based at least in part on the evaluating (see Dasgupta Column 29 line 22 to 27: “At operation 1170, an approved iteration of a ML media model is deployed to a production environment as a production model. In some embodiments, the approval may be indicated via user input, for example, via the model diagnosis interface as discussed in connection with operation 1150 of FIG. 11A”. Examiner interprets an approved iteration of a ML media model is deployed to a production environment to be equivalent as the claimed “deploying”).
However, Dasgupta fails to teach:
sampling, by the computing device, a set of unlabeled data to obtain a random training set of unlabeled data;
labeling, by the computing device and using the one or more machine learning models, the random training set of unlabeled data to produce a pseudo labeled training set;
correcting, by the computing device, the labels on a random subset of the pseudo labeled training set;
training, by the computing device, the one or more machine learning models on the labeled training set, the corrected random subset, and the pseudo labeled training set.
Cascante teaches, in analogous system,
sampling, by the computing device, a set of unlabeled data to obtain a random training set of unlabeled data (see Cascante Figure 1: Examiner interprets the unlabeled samples to be equivalent as the claimed “random training set of unlabeled data”);
PNG
media_image1.png
375
887
media_image1.png
Greyscale
labeling, by the computing device and using the one or more machine learning models, the random training set of unlabeled data to produce a pseudo labeled training set (see Cascante Figure 1: Examiner interprets the unlabeled samples and the pseudo labeled samples to be equivalent as the claimed “random training set of unlabeled data” and “pseudo labeled training set” respectively);
PNG
media_image2.png
375
887
media_image2.png
Greyscale
correcting, by the computing device, the labels on a random subset of the pseudo labeled training set (see Cascante Page 4 Left column last paragraph: “Note that, in Algorithm 1 X̅t is selected from the whole unlabeled set DUL, enabling previous pseudo-annotated samples to enter or to leave the new training set. This is used to discourage concept drift or confirmation bias, as it can prevent erroneous labels predicted by an undertrained network during the early stages of training to be accumulated. To further alleviate the problem, we also reinitialize the model parameters θ with random initializations after each round, and empirically observe that, –as opposed to fine-tuning–, our reinitialization strategy leads to better performance”. Examiner interprets the modification by either entering or leaving the samples to be equivalent as the claimed “correcting”);
training, by the computing device, the one or more machine learning models on the labeled training set, the corrected random subset, and the pseudo labeled training set (see Cascante Figure 1: Examiner interprets labeled samples, the combination of labeled samples and selected samples between step (3) and step (4), and selected samples to be equivalent as the claimed “labeled training set”, “corrected random subset”, and “pseudo labeled training set” respectively);
PNG
media_image3.png
375
887
media_image3.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dasgupta with the above teachings of Cascante by training a machine learning model based on an accuracy target, as taught by Dasgupta, and including pseudo-labels training in the machine learning model, as taught by Cascante. The modification would have been obvious because one of ordinary skill in the art would be motivated to improve performance in machine leaning tasks using the pseudo-label approach as it delivers state-of-the-art accuracy (as suggested by Cascante at page 1, right column: “Prior work has shown that training with a curriculum improves performance in several machine learning”, and at page 2 left column :” We demonstrate that, with the proposed curriculum paradigm, the classic pseudo-labeling approach can deliver near state-of-the-art results on CIFAR-10, Imagenet ILSVRC top-1 accuracy, and SVHN– compared to very recently proposed methods. Additionally, compared to previous approaches, our version of pseudo-labeling leads to more consistent results in realistic evaluation settings, where out-of-distribution samples are present”).
Referring to Claim 2, the combination of Dasgupta and Cascante teaches the method of claim 1, wherein the accuracy target comprises a threshold determined based at least in part on a performance of the one or more machine learning models in classifying unlabeled data (see Dasgupta Column 13 line 9 to 18: “until certain conditions are met, for example, until the classifier's performance is above a certain threshold, or after the classifier has gone through at least a certain number of rounds of training. If the performance of the classifier model is not yet satisfactory, in some embodiments, the media annotation system may allow the user to perform additional rounds of manual annotation or classifier training.” Examiner interprets the classifier’s performance is above a certain threshold to be equivalent as the claimed “threshold determined based at least in part on a performance”).
Referring to Claim 6, Dasgupta teaches the method of claim 1, wherein the evaluating further comprises:
identifying a performance deficiency where the accuracy of the one or more models in classifying a class of unlabeled data is below a threshold (see Dasgupta Column 13 line 9 to 18: “In some embodiments, the extrapolation button 1780 may be disabled until certain conditions are met, for example, until the classifier's performance is above a certain threshold, or after the classifier has gone through at least a certain number of rounds of training. If the performance of the classifier model is not yet satisfactory, in some embodiments, the media annotation system may allow the user to perform additional rounds of manual annotation or classifier training”. Examiner interprets until the classifier's performance is above a certain threshold to be equivalent as the claimed “performance deficiency where the accuracy of the one or more models in classifying a class of unlabeled data is below a threshold”);
However, Dasgupta fails to teach:
augmenting the random subset of the pseudo labeled training set with a targeted subset comprising data from the pseudo labeled training set that are labeled with the class.
Cascante teaches, in analogous system,
augmenting the random subset of the pseudo labeled training set with a targeted subset comprising data from the pseudo labeled training set that are labeled with the class (see Cascante Figure 1: Examiner interprets the pseudo labeled samples and unlabeled samples to be equivalent as the claimed “pseudo labeled training set” and “targeted subset”).
PNG
media_image2.png
375
887
media_image2.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dasgupta with the above teachings of Cascante by training a machine learning model based on an accuracy target, as taught by Dasgupta, and including pseudo-labels training in the machine learning model, as taught by Cascante. The modification would have been obvious because one of ordinary skill in the art would be motivated to improve performance in machine leaning tasks using the pseudo-label approach as it delivers state-of-the-art accuracy (as suggested by Cascante at page 1, right column: “Prior work has shown that training with a curriculum improves performance in several machine learning”, and at page 2 left column :” We demonstrate that, with the proposed curriculum paradigm, the classic pseudo-labeling approach can deliver near state-of-the-art results on CIFAR-10, Imagenet ILSVRC top-1 accuracy, and SVHN– compared to very recently proposed methods. Additionally, compared to previous approaches, our version of pseudo-labeling leads to more consistent results in realistic evaluation settings, where out-of-distribution samples are present”).
Referring to Claim 7, the combination of Dasgupta and Cascante teaches the method of claim 1, wherein the labeled training set of labeled data is smaller than the pseudo labeled training set so that a small amount of labeled data can be used to create a larger pseudo labeled training set (see Cascante Figure 1. Examiner interprets the labeled samples and pseudo labeled samples to be equivalent as the claimed “labeled training set” and “pseudo labeled training set” respectively).
PNG
media_image4.png
375
887
media_image4.png
Greyscale
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Dasgupta with the above teachings of Cascante by training a machine learning model based on an accuracy target, as taught by Dasgupta, and including pseudo-labels training in the machine learning model, as taught by Cascante. The modification would have been obvious because one of ordinary skill in the art would be motivated to improve performance in machine leaning tasks using the pseudo-label approach as it delivers state-of-the-art accuracy (as suggested by Cascante at page 1, right column: “Prior work has shown that training with a curriculum improves performance in several machine learning”, and at page 2 left column :” We demonstrate that, with the proposed curriculum paradigm, the classic pseudo-labeling approach can deliver near state-of-the-art results on CIFAR-10, Imagenet ILSVRC top-1 accuracy, and SVHN– compared to very recently proposed methods. Additionally, compared to previous approaches, our version of pseudo-labeling leads to more consistent results in realistic evaluation settings, where out-of-distribution samples are present”).
Referring to independent Claim 8 and Claim 15, they are rejected on the same basis as independent claim 1 since they are analogous claims of Claim 1.
Referring to dependent Claim 9 and Claim 16, they are rejected on the same basis as dependent claim 2 since they are analogous claims.
Referring to dependent Claim 13 and Claim 20, they are rejected on the same basis as dependent claim 6 since they are analogous claims.
Referring to dependent Claim 14, claim 14 is rejected on the same basis as dependent claim 7 since they are analogous claims.
Claims 3-4, 10-11, 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Dasgupta et al (US Patent 12182227”- hereinafter Dasgupta) in view of Cascante et al (NPL: " Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning"- hereinafter Cascante) and in further view of Manian et al(NPL: " Hyperspectral Image Labeling and Classification Using an Ensemble Semi-Supervised Machine Learning Approach"- hereinafter Manian).
Referring to Claim 3, the combination of Dasgupta and Cascante teaches the method of claim 1, however, fails to teach wherein the labeling comprises ensemble learning with multiple machine learning models.
Manian teaches, in analogous system, wherein the labeling comprises ensemble learning with multiple machine learning models (see Manian Page 1 abstract line 8 to 10: “An ensemble of machine learning models takes the extracted features and groundtruth data from the unsupervised stage as input and a decision block then combines the output of the machines to label the image based on majority voting.” Examiner interprets the combination of the output of the machines to label the images to be equivalent as the claimed “labeling”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Dasgupta and Cascante with the above teachings of Manian by training a machine learning model based on an accuracy target and including pseudo-labels training in the machine learning model, as taught by Dasgupta and Cascante, and labeling images using an ensemble of machine learning models, as taught by Manian. The modification would have been obvious because one of ordinary skill in the art would be motivated to improve classification accuracy in ensemble learning approach (as suggested by Manian at page 20 paragraph 1 lines 18 to 20: “Our ensemble method improves the water and soil classification accuracies using 24.6% of the dataset for training and the remaining data for testing”).
Referring to Claim 4, the combination of Dasgupta, Cascante and Manian teaches the method of claim 3, wherein ensemble learning comprises at least one of bagging, stacking, or boosting (see Manian Page 1 abstract line 10 to 11: “The ensemble of machine learning methods includes support vector machines, gradient boosting, Gaussian classifier, and linear perceptron”. Examiner interprets gradient boosting to be equivalent as the claimed “boosting”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Dasgupta and Cascante with the above teachings of Manian by training a machine learning model based on an accuracy target and including pseudo-labels training in the machine learning model, as taught by Dasgupta and Cascante, and labeling images using an ensemble of machine learning models, as taught by Manian. The modification would have been obvious because one of ordinary skill in the art would be motivated to improve classification accuracy in ensemble learning approach (as suggested by Manian at page 20 paragraph 1 lines 18 to 20: “Our ensemble method improves the water and soil classification accuracies using 24.6% of the dataset for training and the remaining data for testing”).
Referring to dependent Claim 10 and Claim 17, they are rejected on the same basis as dependent claim 3 since they are analogous claims.
Referring to dependent Claim 11 and Claim 18, they are rejected on the same basis as dependent claim 4 since they are analogous claims.
Claims 5, 12, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Dasgupta et al (US Patent 12182227”- hereinafter Dasgupta) in view of Cascante et al (NPL: " Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning"- hereinafter Cascante) and in further view of Fourati et al(NPL: " AN ORIGINAL FRAMEWORK FOR WHEAT HEAD DETECTION USING DEEP, SEMI-SUPERVISED AND ENSEMBLE LEARNING WITHIN GLOBAL WHEAT HEAD DETECTION (GWHD) DATASET)"- hereinafter Fourati).
Referring to Claim 5, the combination of Dasgupta and Cascante teaches the method of claim 1, wherein the labeling comprises test time augmentation with at least one of vertical/horizontal flipping, blurring, random cropping, or histogram equalization (see Fourati Page 4: 3.3.1 Test Time Augmentation (TTA): “We have compared different transforms including varying brightness or contrast but later we have only focused on test time augmentation based on orientation transformations including vertical flips, horizontal flips and 90 degrees rotation”. Examiner interprets test time augmentation based on orientation transformations including vertical flips and horizontal flips to be equivalent as the claimed “test time augmentation with at least one of vertical/horizontal flipping”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combination of Dasgupta, Cascante and Manian with the above teachings of Fourati by iteratively pseudo-labeled images using an ensemble of machine learning models with early stopping based on accuracy target, as taught by Dasgupta, Cascante and Manian, and labeling by using test time augmentation including vertical flips and horizontal flips, as taught by Fourati. The modification would have been obvious because one of ordinary skill in the art would be motivated to use test time augmentation approach to simplify and effectively optimize results (as suggested by Fourati at page 4 paragraph 3.3.1: “It is a simple but yet a very effective way to optimize mAP results. In the context of object detection, test time augmentation could lead to more accurate results in terms of detecting more wheat heads”).
Referring to dependent Claim 12 and Claim 19, they are rejected on the same basis as dependent claim 5 since they are analogous claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AWADAGBE G HOUNTON whose telephone number is (571)270-0670. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AWADAGBE G HOUNTON/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights