Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is responsive to the Application filed on 03/09/2026
Claims 1-20 are pending in the case. Claims 1, 10 and 17 are independent claims. Claims 1, 10 and 17 have been currently amended.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/09/2026 has been entered.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to an abstract idea without significantly more.
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A). The Step 2A analysis is broken into two prongs. In the first prong (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application. If it is determined at step 2A, Prong 2 that the claims do not integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself. Applicant is advised to consult the 2019 PEG for more details of the analysis.
Step 1 Analysis: Is the claim to a process, machine, manufacture or composition of matter? See MPEP § 2106.03.
Claims 1-9 are drawn to a system, claims 10-16 is drawn to a method and claims 17-20 are drawn to a non-transitory computer-readable medium, therefore each of these claim groups falls under one of four categories of statutory subject matter (machine/products/apparatus, process/method, manufactures and compositions of mater; Step 1). Nonetheless, the claims are directed to a judicially recognized exception of an abstract idea without significant more (Step 2A, see below). Independent claims 1, 10 and 17 are nonverbatim but similar in claim construction, hence share the same rationale that the claimed inventions are directed to non-statutory subject matter as follows:
Regarding claim 1:
Claim 1 recites: A system for generating labeled datasets for training machine learning models, the system comprising:
one or more memories;
and one or more processors, communicatively coupled to the one or more memories, configured to:
receive, from one or more data sources, unlabeled data samples;
receive inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples;
train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels;
identify, using the first machine learning model, a second subset of the unlabeled data samples including data elements associated with a data profile indicating a structural similarity to the data elements included in the first subset of the unlabeled data samples,
wherein the structural similarity is determined based on a statistical profile of the data profile;
apply, using the first machine learning model, automatic labels to the data elements included in the second subset of the unlabeled data samples;
generate a labeled dataset that includes the data elements associated with the user- specified labels and the data elements associated with the automatic labels;
and train a second machine learning model using the labeled dataset
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Claim 1 is directed to an abstract idea, specifically, a mental process – concepts performed in the human mind or by a human using a pen and paper" (including an observation, evaluation, judgement, opinion).
Independent claim 1 recites in part:
“identify, …. , a second subset of the unlabeled data samples including data elements associated with a data profile indicating a structural similarity to the data elements included in the first subset of the unlabeled data samples”
The limitation above is broadly and reasonably interpreted as a mental process, as a form or mental evaluation or judgement. For example, one can mentally evaluate data received, and identify, based on judgement and opinion, unlabeled data that is structurally similar to the first labeled data.
“wherein the structural similarity is determined based on a statistical profile of the data profile”
The limitation above is broadly and reasonably interpreted as a mental process, as a form or mental evaluation or judgement. For example, one can mentally evaluate data received, and determine, based on judgement and opinion, structurally similar data.
apply, … , automatic labels to the data elements included in the second subset of the unlabeled data samples,
The limitation above is broadly and reasonably interpreted as a mental process, as a form of mental evaluation or judgement, and or by a human using a pen and paper. For example, one can assign a label to a data element.
“generate a labeled dataset that includes the data elements associated with the user- specified labels and the data elements associated with the automatic labels”
The limitation above is broadly and reasonably interpreted as a mental process, as a form of mental evaluation or judgement, and or by a human using a pen and paper. For example, with pen and paper, one can create a list of data that has both the labels chosen by a user and labels generated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
Independent claim 1 recites in part:
A system for generating labeled datasets for training machine learning models, the system comprising:
one or more memories;
and one or more processors, communicatively coupled to the one or more memories, configured to:, as drafted, amount to additional elements that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, such generic computing components recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations) MPEP §§ 2106.04(d), 2106.05(f)(2).
receive, from one or more data sources, unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity (pre-solution activity, a step of obtaining information)to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
receive inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, […] amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, […], amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
train a second machine learning model using the labeled dataset, as drafted, amount to insignificant extra-solution activity, as merely appending a type of environment to the beginning of the abstract idea in the form of a machine learning model for performing the mental activities of data observation and judgement.
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
First, the additional elements directed to generally linking the use of a judicial exception to a particular technological environment or field of use are deemed insufficient to transform the judicial exception to a patentable invention because the claimed limitations generally link the judicial exception to the technology environment, see MPEP 2106.05(h). However, they are included below for the sake of completeness.
Second, the additional elements mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception. See MPEP 2106.05(f). However, they are included below for the sake of completeness.
Independent claim 1 recites in part:
A system for generating labeled datasets for training machine learning models, the system comprising:
one or more memories;
and one or more processors, communicatively coupled to the one or more memories, configured to:, as drafted, amount to additional elements that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, such generic computing components recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations), which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II). MPEP §§ 2106.04(d), 2106.05(f)(2).
receive, from one or more data sources, unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity (pre-solution activity, a step of obtaining information)to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
receive inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, […] amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, […] amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
train a second machine learning model using the labeled dataset, as drafted, amount to insignificant extra-solution activity, as merely appending a type of environment to the beginning of the abstract idea in the form of a machine learning model for performing the mental activities of data observation and judgement.
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. The claims are not eligible subject matter.
Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole the independent claim limitations do not recite what have the courts have identified as “significantly more”.
Regarding claim 10:
Claim 10 recites: A method for generating a labeled dataset using automated data profiling, comprising:
receiving, by a data labeling system, unlabeled data samples;
receiving, by the data labeling system, inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples;
training, by the data labeling system, a first machine learning model using the user- specified labels;
identifying, by the data labeling system and using the first machine learning model, a second subset of the unlabeled data samples including data elements associated with a data profile indicating a structural similarity to the data elements included in the first subset of the unlabeled data samples,
wherein the structural similarity is determined based on a statistical profile of the data profile;
applying, by the data labeling system, automatic labels to the data elements included in the second subset of the unlabeled data samples using the first machine learning model;
and generating, by the data labeling system, a labeled dataset that includes the data elements associated with the user-specified labels and the data elements associated with the automatic labels
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Claim 10 is directed to an abstract idea, specifically, a mental process – concepts performed in the human mind or by a human using a pen and paper" (including an observation, evaluation, judgement, opinion).
identifying, by the data labeling system …, a second subset of the unlabeled data samples including data elements associated with a data profile indicating a structural similarity to the data elements included in the first subset of the unlabeled data samples
The limitation above is broadly and reasonably interpreted as a mental process, as a form or mental evaluation or judgement. For example, one can mentally evaluate data received, and identify, based on judgement and opinion, unlabeled data that is structurally similar to the first labeled data.
“wherein the structural similarity is determined based on a statistical profile of the data profile”
The limitation above is broadly and reasonably interpreted as a mental process, as a form or mental evaluation or judgement. For example, one can mentally evaluate data received, and determine, based on judgement and opinion, structurally similar data.
“applying, … , automatic labels to the data elements included in the second subset of the unlabeled data samples using the first machine learning model”
The limitation above is broadly and reasonably interpreted as a mental process, as a form of mental evaluation or judgement, and or by a human using a pen and paper. For example, one can assign a label to a data element.
“generating, by the data labeling system, a labeled dataset that includes the data elements associated with the user-specified labels and the data elements associated with the automatic labels”
The limitation above is broadly and reasonably interpreted as a mental process, as a form of mental evaluation or judgement, and or by a human using a pen and paper. For example, with pen and paper, one can create a list of data that has both the labels chosen by a user and labels generated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
Independent claim 10 recites in part:
A method for generating a labeled dataset using automated data profiling, comprising:
receiving, by a data labeling system, unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity (pre-solution activity, a step of obtaining information)to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
receiving, by the data labeling system, inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
training, by the data labeling system, a first machine learning model using the user- specified labels, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, […] amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
First, the additional elements directed to generally linking the use of a judicial exception to a particular technological environment or field of use are deemed insufficient to transform the judicial exception to a patentable invention because the claimed limitations generally link the judicial exception to the technology environment, see MPEP 2106.05(h). However, they are included below for the sake of completeness.
Second, the additional elements mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception. See MPEP 2106.05(f). However, they are included below for the sake of completeness.
Independent claim 10 recites in part:
A method for generating a labeled dataset using automated data profiling, comprising:
receiving, by a data labeling system, unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity (pre-solution activity, a step of obtaining information)to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
receiving, by the data labeling system, inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
training, by the data labeling system, a first machine learning model using the user- specified labels, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, […] amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole the independent claim limitations do not recite what have the courts have identified as “significantly more”.
Regarding claim 17:
Claim 17 recites: A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:
one or more instructions that, when executed by one or more processors of a data labeling system, cause the data labeling system to:
receive, from one or more data sources, unlabeled data samples;
receive inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples;
train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels;
identify, using the first machine learning model, a second subset of the unlabeled data samples including data elements associated with a data profile indicating a structural similarity to the data elements included in the first subset of the unlabeled data samples,
wherein the structural similarity is determined based on a statistical profile of the data profile;
apply automatic labels to the data elements included in the second subset of the unlabeled data samples using the first machine learning model;
present a user interface to request feedback related to the automatic labels based on confidence levels associated with the automatic labels; and
generate, based on the feedback related to the automatic labels, a labeled dataset that includes the data elements associated with the user-specified labels and the data elements associated with the automatic labels
Step 2A Prong One Analysis: Does the claim recite an abstract idea, law of nature, or natural phenomenon? See MPEP § 2106.04(II)(A)(1).
Claim 17 is directed to an abstract idea, specifically, a mental process – concepts performed in the human mind or by a human using a pen and paper" (including an observation, evaluation, judgement, opinion).
identifying, by the data labeling system …, a second subset of the unlabeled data samples including data elements associated with a data profile indicating a structural similarity to the data elements included in the first subset of the unlabeled data samples
The limitation above is broadly and reasonably interpreted as a mental process, as a form or mental evaluation or judgement. For example, one can mentally evaluate data received, and identify, based on judgement and opinion, unlabeled data that is structurally similar to the first labeled data.
“wherein the structural similarity is determined based on a statistical profile of the data profile”
The limitation above is broadly and reasonably interpreted as a mental process, as a form or mental evaluation or judgement. For example, one can mentally evaluate data received, and determine, based on judgement and opinion, structurally similar data.
“apply automatic labels to the data elements included in the second subset of the unlabeled data samples using […]
The limitation above is broadly and reasonably interpreted as a mental process, as a form of mental evaluation or judgement, and or by a human using a pen and paper. For example, one can assign a label to a data element.
“generate, based on the feedback related to the automatic labels, a labeled dataset that includes the data elements associated with the user-specified labels and the data elements associated with the automatic labels”
The limitation above is broadly and reasonably interpreted as a mental process, as a form of mental evaluation or judgement, and or by a human using a pen and paper. For example, with pen and paper, one can create a list of data that has both the labels chosen by a user and labels generated.
Step 2A Prong Two Analysis: Does the claim recite additional elements that integrate the judicial exception into a practical application? See MPEP § 2106.04(d).
Independent claim 17 recites in part:
A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:
one or more instructions that, when executed by one or more processors of a data labeling system, cause the data labeling system to, as drafted, amount to additional elements that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, such generic computing components recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations), which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II). MPEP §§ 2106.04(d), 2106.05(f)(2).
receive, from one or more data sources, unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity (pre-solution activity, a step of obtaining information)to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
receive inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, […], amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
present a user interface to request feedback related to the automatic labels based on confidence levels associated with the automatic labels, as drafted, amounts to adding insignificant extra-solution activity (post-solution activity, a step of outputting information) to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
Step 2B Analysis: Does the claim recite additional elements that amount to significantly more than the judicial exception? See MPEP § 2106.05.
First, the additional elements directed to generally linking the use of a judicial exception to a particular technological environment or field of use are deemed insufficient to transform the judicial exception to a patentable invention because the claimed limitations generally link the judicial exception to the technology environment, see MPEP 2106.05(h). However, they are included below for the sake of completeness.
Second, the additional elements mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception. See MPEP 2106.05(f). However, they are included below for the sake of completeness.
Independent claim 17 recites in part:
A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:
one or more instructions that, when executed by one or more processors of a data labeling system, cause the data labeling system to, as drafted, amount to additional elements that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, such generic computing components recited at a high-level of generality (i.e., as a generic processor performing data gathering and mathematical calculations), which the courts have recognized as well‐understood, routine, and conventional when they are claimed in a generic manner. See MPEP § 2106.05(d)(II). MPEP §§ 2106.04(d), 2106.05(f)(2).
receive, from one or more data sources, unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity (pre-solution activity, a step of obtaining information)to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
receive inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples, as drafted, amounts to adding insignificant extra-solution activity to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, […], amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
[…] using the first machine learning model, amounts to adding the words “apply it” (or an equivalent) with the judicial exception and reciting only the idea of a solution or outcome, i.e., the claim fails to recite details of how a solution to a problem is accomplished because it is unclear how the “ML model” is used nor the specification makes it clear how these actions are performed. Thus, these additional elements are recited in a manner that represent no more than mere instructions to apply the judicial exceptions on a computer. See MPEP § 2106.05(f) and § 2106.04(d).
present a user interface to request feedback related to the automatic labels based on confidence levels associated with the automatic labels, as drafted, amounts to adding insignificant extra-solution activity (post-solution activity, a step of outputting information) to the judicial exception. See MPEP §§ 2106.04(d), 2106.05(g).
Thus, considering the additional elements individually and in combination and the claims as a whole, the additional elements do not provide significantly more than the abstract idea. The claims are not eligible subject matter.
Therefore, in examining elements as recited by the limitations individually and as an ordered combination, as a whole the independent claim limitations do not recite what have the courts have identified as “significantly more”.
Furthermore, regarding dependent claims 2-9 are dependent on claim 1, claims 11-16 are dependent on claim 10 and claims 18-20 are dependent on claim 17, the claims are directed to a judicial exception without significantly more as highlighted below in the claim limitations by evaluating the claim limitations under Step 2A and 2B:
Claims 2 and 11 incorporates the rejection of independent claims 1 and 10 respectively, and does not integrate the judicial exception into a practical application.
Claims 3 and 12 incorporates the rejection of independent claims 1 and 10 respectively, and does not integrate the judicial exception into a practical application.
Claims 4, 13 and 18 incorporates the rejection of claims 3, 12 and 17 respectively, and does not integrate the judicial exception into a practical application.
Claims 5 and 14 incorporates the rejection of claims 3 and 12 respectively, and does not integrate the judicial exception into a practical application.
Claim 6 incorporates the rejection of claim 5 respectively, and does not integrate the judicial exception into a practical application.
Claims 7, 15 and 19 incorporates the rejection of claims 5, 14 and 17, does not integrate the judicial exception into a practical application.
Claim 8, 16 and 20 incorporates the rejection of claims 5, 14and 17, does not integrate the judicial exception into a practical application.
Claim 9 incorporates the rejection of independent claim 1 and does not integrate the judicial exception into a practical application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-5, 7, 9-15 and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Cavallari et al. (Pub No.: 20220075961 A1), hereinafter referred to as Cavallari, in view of Burbank et al (US Patent No. 11,630,835 B2), hereinafter referred to as Burbank and further in view of Leen et al. (US Patent No. 11,580,379 B1), hereinafter referred to as Leen.
With respect to claim 1, Cavallari disclose:
A system for generating labeled datasets for training machine learning models, the system comprising: one or more memories (In paragraph [0065], Cavallari discloses that processing unit 950 includes one or more processors containing a cache or other form of on-board memory.)
one or more processors, communicatively coupled to the one or more memories, configured to: receive, from one or more data sources, unlabeled data samples (In paragraph [0052], Cavallari discloses that a computer system receives unlabeled content. In some embodiments, receiving unlabeled content includes receiving a query. The unlabeled content included in the query may be a document, sentence, a set of terms, etc. In paragraph [0065], Cavallari discloses that processing unit 950 includes one or more processors containing a cache or other form of on-board memory.)
receive inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples (In paragraph [0054], Cavallari discloses that the computer system determines, from a plurality of labeled vectors stored in a vector index, a first set of labeled vectors that match the unlabeled vector, unlabeled content and the set of labeled content includes user-generated content in multiple different languages. )
identify, using the <computer system>, a second subset of the unlabeled data samples including data elements associated with a data profile indicating a structural similarity to the data elements included in the first subset of the unlabeled data samples, wherein the structural similarity is determined based on a statistical profile of the data profile (In paragraph [0021], Cavallari discloses automatically inferring labels for unlabeled content based on the semantic meaning of labeled content by comparing labeled and unlabeled content in the vector space. In paragraph [0055], Cavallari discloses that the computer system gives a new label to content that doesn't have one (unlabeled data). This new label comes from a group of labeled vectors. Sometimes, this means finding groups of labeled vectors that match the ones in the first group. It may also involve figuring out how confident we are about these matches by counting how many vectors in the second group have the same label as the matching vectors in the first group.)
apply, using the <computer system>, automatic labels to the data elements included in the second subset of the unlabeled data samples (In paragraph [0055], Cavallari disclose the computer system automatically gives a new label to content that doesn't have one, choosing from a group of labeled examples. Such as respective second sets of labeled vectors that match the vectors included in the first set. )
generate a labeled dataset that includes the data elements associated with the user- specified labels and the data elements associated with the automatic labels (In paragraph [0012], Cavallari discloses automatically propagating labels across large sets of unlabeled content, including content in various different languages, based on a small set of labeled content. Unlabeled content may include any of various forms of user-generated content for which relevant labels are not yet known or have not yet been generated.)
With respect to claim 1, Cavallari do not explicitly disclose:
Wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples
A < computer system> as being claimed “first machine learning model”
Train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels
Train a second machine learning model using the labeled dataset
However, Leen is known to disclose:
A < computer system> as being claimed “first machine learning model” (In Fig. 4 and Col. 8, lines 38-49, Leen disclose a first machine learning model of the plurality of machine learning models has been trained. The operations 500 include, at block 506, labeling, by the machine learning service, first unlabeled data using the first machine learning model for an inference request)
Train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels (In Fig. 4 and Cols. 8–9, lines 58–9, Leen discloses that a second machine learning model has been trained using a second subset of the training dataset, wherein the second subset is second unlabeled data using the second machine learning model. )
Train a second machine learning model using the labeled dataset (In Fig. 4 and Cols. 8–9, line 58-9, Leen discloses that a second machine learning model has been trained using a second subset of the training dataset, wherein the second subset is second unlabeled data using the second machine learning model. )
Cavallari and Leen are analogous pieces of art because both references concern the techniques for automatically labeling content based on semantic similarity e.g., for use in training natural language processing (NLP) systems. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Cavallari, matching labeled content, the disclosed IR system selects a label for the unlabeled content from this set of labeled content as taught by Cavallari, with a computer to automatically learn from the training data to generate a model that can make predictions for other data as taught by Leen. The motivation for doing so would have been to improve the accuracy of NLP systems relative to traditional language processing techniques (See [0015] of Cavallari.)
With respect to claim 1, Cavallari in view of Leen do explicitly disclose:
Wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples
However, Burbank is known to disclose:
Wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples (In Cols. 2-3, lines 62-13, Burbank disclose a first profile from a first corpus that includes first user data, a second profile from a second corpus including second user data wherein the first profile and the set of modified profiles includes respective sets of first and second attributes and determine a mathematical distance between the first profile and each modified profile of the set of modified profiles based at least in part on a comparison between the first set of attributes and the second set of attributes)
Cavallari in view Leen and Burbank are analogous pieces of art because all references concern obtaining labels/unlabeled data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Burbank, with modifications of user datasets to support statistical resemblance as taught by Burbank. The motivation for doing so would have been to modify a dataset to mimic a customer dataset such that the modified dataset may be used to support machine learning techniques (See (Col. 6, lines 53-56) of Burbank.)
Regarding claim 2, Cavallari in view Leen and Burbank disclose the elements of claim 1. In addition, Cavallari disclose:
The system of claim 1, wherein the one or more processors, to identify the second subset of the unlabeled data samples, are configured to: detect one or more attributes related to a structure associated with the data elements included in the first subset of the unlabeled data samples, wherein the second subset of the unlabeled data samples is identified based on the data profile indicating the structural similarity to the one or more attributes associated with the data elements included in the first subset of the unlabeled data samples (In paragraphs [0054-0055], Cavallari disclose the computer system looks at a group of labeled vectors in a vector index to find a first set of labeled vectors that are similar to an unlabeled vector. At step 740, the computer system gives a new label to the unlabeled content, choosing it from the first set of labeled vectors. Sometimes, this involves finding second sets of labeled vectors that match the first set.)
Regarding claim 3, Cavallari in view Leen and Burbank disclose the elements of claim 1. In addition, Cavallari disclose:
The system of claim 1, wherein the one or more processors are further configured to: determine a confidence level associated with each of the automatic labels applied to [[each ]]the data elements included in the second subset of the unlabeled data samples (In paragraph [0038], Cavallari disclosed confidence scores generated by module 430 indicating whether the labels of vectors in set 132 are accurate representations of their respective classes. )
Regarding claim 4, Cavallari in view Leen and Burbank disclose the elements of claim 3. In addition, Cavallari disclose:
The system of claim 3, wherein the one or more processors are further configured to: identify, among the automatic labels applied to the data elements included in the second subset of the unlabeled data samples, a subset of the automatic labels for which the associated confidence level satisfies a threshold (In paragraph [0056], Cavallari disclose assigning further includes comparing the propagation score with a propagation threshold, where the assigning is performed based on the propagation score satisfying the propagation threshold).
and maintain, in the labeled dataset, the subset of the automatic labels for which the associated confidence level satisfies the threshold without informing one or more users (In paragraph [0056], Cavallari discloses generating, based on the determination, a total confidence score, where the total confidence score is the sum of the confidence scores of vectors included in the first set of labels that match the selected new label.)
Regarding claim 5, Cavallari in view Leen and Burbank disclose the elements of claim 3. In addition, Cavallari disclose:
The system of claim 3, wherein the one or more processors are further configured to: identify, among the automatic labels applied to the data elements included in the second subset of the unlabeled data samples, a subset of the automatic labels for which the associated confidence level fails to satisfy a threshold (In paragraph [0044], where propagation scores 442 associated with various proposed labels for a given unlabeled vector do not satisfy propagation threshold 452, computer system 110 may throw out the unlabeled vector altogether and begin the labeling process with a different unlabeled vector.)
present a user interface to request feedback related to the subset of the automatic labels for which the associated confidence level fails to satisfy the threshold (In paragraph [0067], Cavallari disclose I/O interface 930 may represent one or more interfaces and may be any of various types of interfaces configured to couple to and communicate with other devices, according to various embodiments.)
Regarding claim 7, Cavallari in view Leen and Burbank disclose the elements of claim 5. In addition, Cavallari disclose:
The system of claim 5, wherein the one or more processors are further configured to: receive, via the user interface, feedback confirming the subset of the automatic labels for which the associated confidence level fails to satisfy the threshold (In paragraph [0042], Cavallari discloses that if propagation score 442 satisfies the threshold 452, then decision 454 will indicate that the proposed label y.sub. I should be assigned to the unlabeled vector 122. If the propagation score 442 does not satisfy the threshold 452, then assignment module 140 indicates that this label should be discarded.)
maintain, in the labeled dataset, the subset of the automatic labels based on the feedback confirming the subset of the automatic labels (In paragraph [0042], Cavallari discloses that the propagation score 442 may be a value that is less than the threshold 452 and, therefore, does not meet this threshold.)
and reinforce one or more rules used by the first machine learning model to predict the subset of the automatic labels based on the feedback confirming the subset of the automatic labels (In paragraph [0042], Cavallari discloses that the propagation threshold may be tuned by a system administrator based on a desired accuracy of labels. For example, for more accurate labels, a high propagation threshold may be selected. As one specific non-limiting example, on a scale of 0 to 1, the propagation threshold may be selected as 0.85. )
Regarding claim 9, Cavallari in view Leen and Burbank disclose the elements of claim 1. In addition, Leen disclose:
The system of claim 1, wherein the one or more processors are further configured to: detect, in the one or more data sources, data elements that contain sensitive information using the second machine learning model, wherein the second machine learning model is trained to predict whether a data element contains sensitive information using one or more of a training dataset or a test dataset created from the labeled dataset (In Fig. 4 and Cols. 8–9, lines 58–9, Leen discloses that a second machine learning model has been trained using a second subset of the training dataset, wherein the second subset is second unlabeled data using the second machine learning model. )
and conceal the sensitive information within the one or more data sources, wherein the one or more processors, to conceal the sensitive information, are configured to mask or delete the sensitive information in the one or more data sources (In Col. 8-9, lines 58–9, Leen discloses that determining the latency of the second machine learning model is less than or equal to the latency of the first machine learning model; and deletes the first machine learning model after the second machine learning model has been trained.)
With respect to claim 10, Cavallari disclose:
A method for generating a labeled dataset using automated data profiling, comprising: receiving, by a data labeling system, unlabeled data samples (In paragraph [0052], Cavallari discloses that a computer system receives unlabeled content. In some embodiments, receiving unlabeled content includes receiving a query. The unlabeled content included in the query may be a document, sentence, a set of terms, etc. In paragraph [0065], Cavallari discloses that processing unit 950 includes one or more processors containing a cache or other form of on-board memory.)
receiving, by the data labeling system, inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples (In paragraph [0054], Cavallari discloses that the computer system determines, from a plurality of labeled vectors stored in a vector index, a first set of labeled vectors that match the unlabeled vector, unlabeled content and the set of labeled content includes user-generated content in multiple different languages. )
identify, using the <computer system>, a second subset of the unlabeled data samples including data elements associated with a data profile indicating a structural similarity to the data elements included in the first subset of the unlabeled data samples, wherein the structural similarity is determined based on a statistical profile of the data profile (In paragraph [0021], Cavallari discloses automatically inferring labels for unlabeled content based on the semantic meaning of labeled content by comparing labeled and unlabeled content in the vector space. In paragraph [0055], Cavallari discloses that the computer system gives a new label to content that doesn't have one (unlabeled data). This new label comes from a group of labeled vectors. Sometimes, this means finding groups of labeled vectors that match the ones in the first group. It may also involve figuring out how confident we are about these matches by counting how many vectors in the second group have the same label as the matching vectors in the first group.)
applying, by the data labeling system, automatic labels to the data elements included in the second subset of the unlabeled data samples using the <computer system> (In paragraph [0055], Cavallari disclose the computer system automatically gives a new label to content that doesn't have one, choosing from a group of labeled examples. Such as respective second sets of labeled vectors that match the vectors included in the first set.)
generating, by the data labeling system, a labeled dataset that includes the data elements associated with the user-specified labels and the data elements associated with the automatic labels (In paragraph [0012], Cavallari discloses automatically propagating labels across large sets of unlabeled content, including content in various different languages, based on a small set of labeled content. Unlabeled content may include any of various forms of user-generated content for which relevant labels are not yet known or have not yet been generated.)
With respect to claim 10, Cavallari do not explicitly disclose:
Wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples
A < computer system> as being claimed “first machine learning model”
Training, by the data labeling system, a first machine learning model using the user- specified labels
However, Leen is known to disclose:
A < computer system> as being claimed “first machine learning model” (In Fig. 4 and Col. 8, lines 38-49, Leen disclose a first machine learning model of the plurality of machine learning models has been trained. The operations 500 include, at block 506, labeling, by the machine learning service, first unlabeled data using the first machine learning model for an inference request)
Training, by the data labeling system, a first machine learning model using the user- specified labels (In Col. 9, lines 19-40, Leen disclose the system training multiple machine learning models using training dataset. As each model finishes training, it is used automatically label a corresponding unlabeled dataset. Specifically, the first trained model labels a first unlabeled dataset, and after the second model is trained, it labels a second unlabeled dataset.)
Cavallari and Leen are analogous pieces of art because both references concern the techniques for automatically labeling content based on semantic similarity e.g., for use in training natural language processing (NLP) systems. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Cavallari, matching labeled content, the disclosed IR system selects a label for the unlabeled content from this set of labeled content as taught by Cavallari, with a computer to automatically learn from the training data to generate a model that can make predictions for other data as taught by Leen. The motivation for doing so would have been to improve the accuracy of NLP systems relative to traditional language processing techniques (See [0015] of Cavallari.)
With respect to claim 10, Cavallari in view of Leen do explicitly disclose:
Wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples
However, Burbank is known to disclose:
Wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples (In Cols. 2-3, lines 62-13, Burbank disclose a first profile from a first corpus that includes first user data, a second profile from a second corpus including second user data wherein the first profile and the set of modified profiles includes respective sets of first and second attributes and determine a mathematical distance between the first profile and each modified profile of the set of modified profiles based at least in part on a comparison between the first set of attributes and the second set of attributes.)
Cavallari in view Leen and Burbank are analogous pieces of art because all references concern obtaining labels/unlabeled data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Burbank, with modifications of user datasets to support statistical resemblance as taught by Burbank. The motivation for doing so would have been to modify a dataset to mimic a customer dataset such that the modified dataset may be used to support machine learning techniques (See (Col. 6, lines 53-56) of Burbank.)
Regarding claim 11, Cavallari in view Leen and Burbank disclose the elements of claim 10. In addition, Cavallari disclose:
The method of claim 10, wherein identifying the second subset of the unlabeled data samples comprises: detecting one or more attributes related to a structure associated with the data elements included in the first subset of the unlabeled data samples, wherein the second subset of the unlabeled data samples is identified based on the data profile indicating the structural similarity to the one or more attributes associated with the data elements included in the first subset of the unlabeled data samples (In paragraphs [0054-0055], Cavallari disclose the computer system looks at a group of labeled vectors in a vector index to find a first set of labeled vectors that are similar to an unlabeled vector. At step 740, the computer system gives a new label to the unlabeled content, choosing it from the first set of labeled vectors. Sometimes, this involves finding second sets of labeled vectors that match the first set.)
Regarding claim 12, Cavallari in view Leen and Burbank disclose the elements of claim 10. In addition, Cavallari disclose:
The method of claim 10, further comprising: determining a confidence level associated with each of the automatic labels applied to [[each ]]the data elements included in the second subset of the unlabeled data samples (In paragraph [0038], Cavallari disclosed confidence scores generated by module 430 indicating whether the labels of vectors in set 132 are accurate representations of their respective classes. )
Regarding claim 13, Cavallari in view Leen and Burbank disclose the elements of claim 12. In addition, Cavallari disclose:
The method of claim 12, further comprising: identifying, among the automatic labels applied to the data elements included in the second subset of the unlabeled data samples, a subset of the automatic labels for which the associated confidence level satisfies a threshold (In paragraph [0056], Cavallari disclose assigning further includes comparing the propagation score with a propagation threshold, where the assigning is performed based on the propagation score satisfying the propagation threshold.)
maintaining, in the labeled dataset, the subset of the automatic labels for which the associated confidence level satisfies the threshold without informing one or more users (In paragraph [0056], Cavallari discloses generating, based on the determination, a total confidence score, where the total confidence score is the sum of the confidence scores of vectors included in the first set of labels that match the selected new label)
Regarding claim 14, Cavallari in view Leen and Burbank disclose the elements of claim 12. In addition, Cavallari disclose:
The method of claim 12, further comprising: identifying, among the automatic labels applied to the data elements included in the second subset of the unlabeled data samples, a subset of the automatic labels for which the associated confidence level fails to satisfy a threshold (In paragraph [0044], where propagation scores 442 associated with various proposed labels for a given unlabeled vector do not satisfy propagation threshold 452, computer system 110 may throw out the unlabeled vector altogether and begin the labeling process with a different unlabeled vector.)
presenting a user interface to request feedback related to the subset of the automatic labels for which the associated confidence level fails to satisfy the threshold (In paragraph [0067], Cavallari disclose I/O interface 930 may represent one or more interfaces and may be any of various types of interfaces configured to couple to and communicate with other devices, according to various embodiments.)
Regarding claim 15, Cavallari in view Leen and Burbank disclose the elements of claim 14. In addition, Cavallari disclose:
The method of claim 14, further comprising: receiving, via the user interface, feedback confirming the subset of the automatic labels for which the associated confidence level fails to satisfy the threshold (In paragraph [0042], Cavallari discloses that if propagation score 442 satisfies the threshold 452, then decision 454 will indicate that the proposed label y.sub. I should be assigned to the unlabeled vector 122. If the propagation score 442 does not satisfy the threshold 452, then assignment module 140 indicates that this label should be discarded.)
maintaining, in the labeled dataset, the subset of the automatic labels based on the feedback confirming the subset of the automatic labels (In paragraph [0042], Cavallari discloses that the propagation score 442 may be a value that is less than the threshold 452 and, therefore, does not meet this threshold.)
reinforcing one or more rules used by the first machine learning model to predict the subset of the automatic labels based on the feedback confirming the subset of the automatic labels (In paragraph [0042], Cavallari discloses that the propagation threshold may be tuned by a system administrator based on a desired accuracy of labels. For example, for more accurate labels, a high propagation threshold may be selected. As one specific non-limiting example, on a scale of 0 to 1, the propagation threshold may be selected as 0.85. )
With respect to claim 17, Cavallari disclose:
A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising: one or more instructions that, when executed by one or more processors of a data labeling system, cause the data labeling system to: receive, from one or more data sources, unlabeled data samples (In paragraph [0052], Cavallari discloses that a computer system receives unlabeled content. In some embodiments, receiving unlabeled content includes receiving a query. The unlabeled content included in the query may be a document, sentence, a set of terms, etc. In paragraph [0065], Cavallari discloses that processing unit 950 includes one or more processors containing a cache or other form of on-board memory.)
receive inputs to apply user-specified labels to data elements included in a first subset of the unlabeled data samples (In paragraph [0054], Cavallari discloses that the computer system determines, from a plurality of labeled vectors stored in a vector index, a first set of labeled vectors that match the unlabeled vector, unlabeled content and the set of labeled content includes user-generated content in multiple different languages. )
identify, using the <computer system>, a second subset of the unlabeled data samples including data elements associated with a data profile indicating a structural similarity to the data elements included in the first subset of the unlabeled data samples, wherein the structural similarity is determined based on a statistical profile of the data profile (In paragraph [0021], Cavallari discloses automatically inferring labels for unlabeled content based on the semantic meaning of labeled content by comparing labeled and unlabeled content in the vector space. In paragraph [0055], Cavallari discloses that the computer system gives a new label to content that doesn't have one (unlabeled data). This new label comes from a group of labeled vectors. Sometimes, this means finding groups of labeled vectors that match the ones in the first group. It may also involve figuring out how confident we are about these matches by counting how many vectors in the second group have the same label as the matching vectors in the first group.)
apply automatic labels to the data elements included in the second subset of the unlabeled data samples using the <computer system> (In paragraph [0055], Cavallari disclose the computer system automatically gives a new label to content that doesn't have one, choosing from a group of labeled examples. Such as respective second sets of labeled vectors that match the vectors included in the first set. )
present a user interface to request feedback related to the automatic labels based on confidence levels associated with the automatic labels (In paragraph [0067], Cavallari disclose I/O interface 930 may represent one or more interfaces and may be any of various types of interfaces configured to couple to and communicate with other devices, according to various embodiments.)
generate, based on the feedback related to the automatic labels, a labeled dataset that includes the data elements associated with the user-specified labels and the data elements associated with the automatic labels (In paragraph [0012], Cavallari discloses automatically propagating labels across large sets of unlabeled content, including content in various different languages, based on a small set of labeled content. Unlabeled content may include any of various forms of user-generated content for which relevant labels are not yet known or have not yet been generated.)
With respect to claim 17, Cavallari do not explicitly disclose:
A < computer system> as being claimed “first machine learning model”
Train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels
Wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples
However, Leen is known to disclose:
A < computer system> as being claimed “first machine learning model” (In Fig. 4 and Col. 8, lines 38-49, Leen disclose a first machine learning model of the plurality of machine learning models has been trained. The operations 500 include, at block 506, labeling, by the machine learning service, first unlabeled data using the first machine learning model for an inference request)
Train a first machine learning model using a training dataset and a test dataset that are based on the user-specified labels (In Col. 9, lines 19-40, Leen disclose the system training multiple machine learning models using training dataset. As each model finishes training, it is used automatically label a corresponding unlabeled dataset. Specifically, the first trained model labels a first unlabeled dataset, and after the second model is trained, it labels a second unlabeled dataset.)
Cavallari and Leen are analogous pieces of art because both references concern the techniques for automatically labeling content based on semantic similarity e.g., for use in training natural language processing (NLP) systems. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Cavallari, matching labeled content, the disclosed IR system selects a label for the unlabeled content from this set of labeled content as taught by Cavallari, with a computer to automatically learn from the training data to generate a model that can make predictions for other data as taught by Leen. The motivation for doing so would have been to improve the accuracy of NLP systems relative to traditional language processing techniques (See [0015] of Cavallari.)
With respect to claim 17, Cavallari in view of Leen do explicitly disclose:
Wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples
However, Burbank is known to disclose:
Wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples (In Cols. 2-3, lines 62-13, Burbank disclose a first profile from a first corpus that includes first user data, a second profile from a second corpus including second user data wherein the first profile and the set of modified profiles includes respective sets of first and second attributes and determine a mathematical distance between the first profile and each modified profile of the set of modified profiles based at least in part on a comparison between the first set of attributes and the second set of attributes.)
Cavallari in view Leen and Burbank are analogous pieces of art because all references concern obtaining labels/unlabeled data. Accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Burbank, with modifications of user datasets to support statistical resemblance as taught by Burbank. The motivation for doing so would have been to modify a dataset to mimic a customer dataset such that the modified dataset may be used to support machine learning techniques (See (Col. 6, lines 53-56) of Burbank.)
Regarding claim 18, Cavallari in view Leen and Burbank disclose the elements of claim 17. In addition, Cavallari disclose:
The non-transitory computer-readable medium of claim 17, wherein the one or more instructions further cause the data labeling system to: identify, among the automatic labels applied to the data elements included in the second subset of the unlabeled data samples, a subset of the automatic labels for which the associated confidence level satisfies a threshold (In paragraph [0056], Cavallari disclose assigning further includes comparing the propagation score with a propagation threshold, where the assigning is performed based on the propagation score satisfying the propagation threshold.)
maintain, in the labeled dataset, the subset of the automatic labels for which the associated confidence level satisfies the threshold without requesting feedback via the user interface (In paragraph [0044], where propagation scores 442 associated with various proposed labels for a given unlabeled vector do not satisfy propagation threshold 452, computer system 110 may throw out the unlabeled vector altogether and begin the labeling process with a different unlabeled vector.)
Regarding claim 19, Cavallari in view Leen and Burbank disclose the elements of claim 17. In addition, Cavallari disclose:
The non-transitory computer-readable medium of claim 17, wherein the one or more instructions further cause the data labeling system to: receive, via the user interface, feedback confirming a subset of the automatic labels (In paragraph [0042], Cavallari discloses that if propagation score 442 satisfies the threshold 452, then decision 454 will indicate that the proposed label y.sub. I should be assigned to the unlabeled vector 122. If the propagation score 442 does not satisfy the threshold 452, then assignment module 140 indicates that this label should be discarded.)
maintain, in the labeled dataset, the subset of the automatic labels based on the feedback confirming the subset of the automatic labels (In paragraph [0042], Cavallari discloses that the propagation score 442 may be a value that is less than the threshold 452 and, therefore, does not meet this threshold.)
reinforce one or more rules used by the first machine learning model to predict the subset of the automatic labels based on the feedback confirming the subset of the automatic labels (In paragraph [0042], Cavallari discloses that the propagation threshold may be tuned by a system administrator based on a desired accuracy of labels. For example, for more accurate labels, a high propagation threshold may be selected. As one specific non-limiting example, on a scale of 0 to 1, the propagation threshold may be selected as 0.85.)
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Cavallari in view of Leen, Burbank and further in view of BALASUBRAMANIAN et al. (Pub No.: 20220172100 A1), hereinafter referred to as BALASUBRAMANIAN.
Regarding claim 6, Cavallari in view of Leen and Burbank disclose the elements of claim 5. Cavallari in view of Leen and Burbank do not explicitly disclose:
The system of claim 5, wherein the user interface includes one or more visual indicators to differentiate the automatic labels for which the feedback is requested
However, BALASUBRAMANIAN disclose the limitation (In paragraph [0053], BALASUBRAMANIAN discloses that a request is sent to a feedback device 145 (e.g., a user providing feedback via this device) to evaluate the results of the trained models 103 on the unlabeled testing dataset and feedback is received by the anomaly detection service/component. )
Accordingly, it would have been obvious to a person having ordinary skills in the art before the effective filling date of the claimed invention, having the teaching of Cavallari in view of Leen and Burbank to include BALASUBRAMANIAN, with adding data from the unlabeled dataset into the training dataset when the received requested feedback indicates a verified result. The motivation for doing so would have been to improve the model accuracy, thereby learning from human expertise (See [0036] of BALASUBRAMANIAN.)
Claims 8, 16 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Cavallari in view of Leen, Burbank and further in view of Cardella et al. (US Patent No. 11,580,379 B1), hereinafter referred to as Cardella.
Regarding claim 8, Cavallari in view of Leen and Burbank disclose the elements of claim 5. Cavallari in view of Leen and Burbank do not explicitly disclose:
The system of claim 5, wherein the one or more processors are further configured to: receive, via the user interface, feedback rejecting or modifying the subset of the automatic labels for which the associated confidence level fails to satisfy the threshold
modify, in the labeled dataset, the subset of the automatic labels based on the feedback rejecting or modifying the subset of the automatic labels
update one or more counter-rules used by the first machine learning model based on the feedback rejecting or modifying the subset of the automatic labels
However, Cardella disclose the limitation:
The system of claim 5, wherein the one or more processors are further configured to: receive, via the user interface, feedback rejecting or modifying the subset of the automatic labels for which the associated confidence level fails to satisfy the threshold (In Cols. 1-2, lines 66–17, Cardella discloses automated systems and techniques that can automatically label data. Training instances with label confidence scores below a threshold confidence score may be excluded from the training corpus.)
modify, in the labeled dataset, the subset of the automatic labels based on the feedback rejecting or modifying the subset of the automatic labels (In Fig. 3C and Col. 14, lines 22–56, Cardella discloses that after modifying the transcription label (e.g., generating a modified version of the label), the user may select the graphical control button 312 labeled as “Proceed” to continue to FIG. 3D.)
update one or more counter-rules used by the first machine learning model based on the feedback rejecting or modifying the subset of the automatic labels (In Col. 14, lines 23–25, Cardella discloses an updated confidence score relevant to modified transcription labels, according to various embodiments of the present disclosure.)
Accordingly, it would have been obvious to a person having ordinary skills in the art before the effective filling date of the claimed invention, having the teaching of Cavallari in view of Brin and Leen to include Cardella, generate a first confidence score associated with the first label data based at least in part on the first data and second data related to label generation by the first person. The motivation for doing so would have been to perform automatic labeling to train the algorithms (See (Col. 3, lines 18-31) of Cardella.)
Regarding claim 16, Cavallari in view of Leen and Burbank disclose the elements of claim 14. Cavallari in view of Leen and Burbank do not explicitly disclose:
The method of claim 14, further comprising: receiving, via the user interface, feedback rejecting or modifying the subset of the automatic labels for which the associated confidence level fails to satisfy the threshold
modifying, in the labeled dataset, the subset of the automatic labels based on the feedback rejecting or modifying the subset of the automatic labels
updating one or more counter-rules used by the first machine learning model based on the feedback rejecting or modifying the subset of the automatic labels
However, Cardella disclose the limitation:
The method of claim 14, further comprising: receiving, via the user interface, feedback rejecting or modifying the subset of the automatic labels for which the associated confidence level fails to satisfy the threshold (In Cols. 1-2, lines 66–17, Cardella discloses automated systems and techniques that can automatically label data. Training instances with label confidence scores below a threshold confidence score may be excluded from the training corpus.)
modifying, in the labeled dataset, the subset of the automatic labels based on the feedback rejecting or modifying the subset of the automatic labels (In Fig. 3C and Col. 14, lines 22–56, Cardella discloses that after modifying the transcription label (e.g., generating a modified version of the label), the user may select the graphical control button 312 labeled as “Proceed” to continue to FIG. 3D.)
updating one or more counter-rules used by the first machine learning model based on the feedback rejecting or modifying the subset of the automatic labels (In Col. 14, lines 23–25, Cardella discloses an updated confidence score relevant to modified transcription labels, according to various embodiments of the present disclosure.)
Accordingly, it would have been obvious to a person having ordinary skills in the art before the effective filling date of the claimed invention, having the teaching of Cavallari in view of Brin and Leen to include Cardella, generate a first confidence score associated with the first label data based at least in part on the first data and second data related to label generation by the first person. The motivation for doing so would have been to perform automatic labeling to train the algorithms (See (Col. 3, lines 18-31) of Cardella.)
Regarding claim 20, Cavallari in view of Leen and Burbank disclose the elements of claim 17. Cavallari in view of Leen and Burbank do not explicitly disclose:
The non-transitory computer-readable medium of claim 17, wherein the one or more instructions further cause the data labeling system to: receive, via the user interface, feedback rejecting or modifying a subset of the automatic labels
modify, in the labeled dataset, the subset of the automatic labels based on the feedback rejecting or modifying the subset of the automatic labels
update one or more counter-rules used by the first machine learning model based on the feedback rejecting or modifying the subset of the automatic labels
However, Cardella disclose the limitation:
The non-transitory computer-readable medium of claim 17, wherein the one or more instructions further cause the data labeling system to: receive, via the user interface, feedback rejecting or modifying a subset of the automatic labels (In Cols. 1-2, lines 66–17, Cardella discloses automated systems and techniques that can automatically label data. Training instances with label confidence scores below a threshold confidence score may be excluded from the training corpus.)
modify, in the labeled dataset, the subset of the automatic labels based on the feedback rejecting or modifying the subset of the automatic labels (In Fig. 3C and Col. 14, lines 22–56, Cardella discloses that after modifying the transcription label (e.g., generating a modified version of the label), the user may select the graphical control button 312 labeled as “Proceed” to continue to FIG. 3D.)
update one or more counter-rules used by the first machine learning model based on the feedback rejecting or modifying the subset of the automatic labels (In Col. 14, lines 23–25, Cardella discloses an updated confidence score relevant to modified transcription labels, according to various embodiments of the present disclosure.)
Accordingly, it would have been obvious to a person having ordinary skills in the art before the effective filling date of the claimed invention, having the teaching of Cavallari in view of Leen and Burbank to include Cardella, generate a first confidence score associated with the first label data based at least in part on the first data and second data related to label generation by the first person. The motivation for doing so would have been to perform automatic labeling to train the algorithms (See (Col. 3, lines 18-31) of Cardella.)
Response to Arguments
Applicant's arguments filed on 03/09/2026 have been fully considered, and in part are persuasive.
Pertaining to Rejection under 101
On pages 11-12, applicant argues claim 1 as an improvement to other technology or technical field of automated data profiling and automated labeling. However, the examiner does not believe the claim recites an improvement to the functioning of a computer or another technology/technical field under the framework of MPEP 2106.05(a). The Examiner respectfully believes based on the structure on claims 1, does not recite any improvement to those technologies themselves. The claim merely uses conventional machine learning, statistical profiling, and automatic labeling as tools to generate additional labeled training data. The claimed advance lies in the informational result than improvement to the operation of the profiling engine, machine learning model or computer technology.
The alleged improvement is occurring within the abstract idea itself. MPEP 2106.05(a) requires an improvement to technology or computer functionality, not merely an improvement to the abstract process. The limitation involving labels has already been identified as reciting an abstract idea and, under MPEP 2106.05(a), you cannot rely on improvements that are themselves part of the abstract idea.
The applicant cites paragraph 15 of the specification, states that “by using the automated data profiling … may significantly reduce the time that is expended…”. The examiners argues nowhere in the claims recite those improvement in a way that would ordinarily support an improvement as cited in paragraph 15 of the specification. See MPEP 2106.05(a).
Arguments are not persuasive and a full 101 analysis is set forth above.
Pertaining to Rejection under 103
On page 14, Applicant argues CAVALLARI, BRIN, and LEEN does not teach "a structure of the first subset of the unlabeled data samples," let alone "wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples,". However, the examiner believes applicant did not adequately explain why CAVALLARI, BRIN, and LEEN references fails to disclose the claimed limitation.
The Examiner notes the amended limitation “wherein the statistical profile comprises one or more attributes related to a structure of the first subset of the unlabeled data samples” is not found in CAVALLARI, BRIN, and LEEN. Nonetheless, LEEN does seem to discuss a first unlabeled data using the first machine learning model for an inference request, the request including a reference to a training dataset (See (Cols. 8-9) of Leen).
Applicant’s arguments with respect to claim(s) 1, 10 and 17 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EVEL HONORE whose telephone number is (703)756-1179. The examiner can normally be reached Monday-Friday 8 a.m. -5:30 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
EVEL HONORE
Examiner
Art Unit 2142
/Mariela Reyes/Supervisory Patent Examiner, Art Unit 2142
Prosecution Insights