Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The Action is responsive to the Amendments and Remarks filed on 2/27/2026. Claims 1-20 are pending claims. Claims 1, 9, and 16 are written in independent form.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55 for the foreign application IN202411002409 with a corresponding priority filing date of 1/12/2024
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claims 1, 9, and 16, the limitation “generating…using a natural language machine learning model, a labeled dataset from unlabeled data based on an understanding of a data labeling task description that is provided to the natural language machine learning model via one or more first prompts” and “the one or more first prompts describe… the labeled dataset as an output of the data labeling task” renders the claims indefinite because it is unclear how prompts being input to a natural language machine learning model can already describe a definitive/actual output that hasn’t yet been generated via the prompts.For purposes of compact prosecution and based on paragraph [0089] included in Applicant’s Remarks dated 2/27/2026 as cited support for the amended language, the claims are being understood as reciting “wherein: (i) the one or more first prompts describe…(b) [[the]] a desired labeled dataset as an output of the data labeling task…”.
Dependent Claims 2-8, 10-15, and 17-20 inherit the deficiencies of their parent claims and are therefore being rejected based upon the same reason(s) stated for their parent claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Yao et al. (U.S. Pre-Grant Publication No. 2023/0368077, hereinafter referred to as Yao) and further in view of Chintagunta et al. (U.S. Pre-Grant Publication No. 2024/0029714, hereinafter referred to as Chintagunta) and Do et al. (U.S. Pre-Grant Publication No. 2020/0285906, hereinafter Referred to as Do906).
Regarding Claim 1:
Yao teaches a computer-implemented method comprising:
Generating, by one or more processors and using a machine learning model, a labeled dataset from unlabeled data based on an understanding of a data labeling description that is provided to the machine learning model via one or more first prompts,
Yao teaches “the training data selection/filter 720 is configured to generate training, validation, and testing datasets for MLT (or ML model training)” where “the training data selection/filter 720 may label data in datasets for supervised learning” (Para. [0186]) thereby teaching generating a labeled dataset from unlabeled data for use in supervised learning/training, validation, and testing.
Yao further teaches labeling the data based on an understanding of a data labeling description provided to the machine learning model via one or more prompts by teaching “supervised learning involves learning a function or model that maps an input to an output based on example input-output pairs or some other form of labeled training data including a set of training examples” (Para. [0323]). It is noted that supervised learning is performed with respect to a machine learning model and learns to label data based on the input/prompts dictating the data labeling description and thus the labeling.
Yao also teaches a processor by teaching “hardware resources 600 including one or more processors (or processor cores) 610, one or more memory/storage devices 620, and one or more communication resources 630, each of which may be communicatively coupled via a interconnect (IX) 606 or other interface circuitry, which implement any suitable bus and/or IX technologies” (Para. [0175]).
Wherein:
(ii) the labeled dataset comprises a plurality of labels;
Yao teaches “the training data selection/filter 720 may label data in datasets for supervised learning” (Para. [0186]) and “ In ML classification, labels are assigned to instances, and models are trained to correctly predict the pre-assigned labels of from the training examples.” (Para. [0279]) thereby teaching a plurality of labels in the datasets for supervised learning.
Generating, by the one or more processors, a first instance of a classification machine learning model by determining one or more first parameter values for the first instance of the classification machine learning model based on a training portion of the labeled dataset;
Yao teaches “in ML classification, labels are assigned to instances, and models are trained to correctly predict the pre-assigned labels of from the training examples” (Para. [0279]) thereby teaching training/generating an ML classification model by determining parameter values of the ML classification model based on labeled training data because ML models contain “learned parameters” values determined from the training (Para. [0189]).
Generating, by the one or more processors and using the first instance of the classification machine learning model, a plurality of validation classification outputs for a validation portion of the labeled dataset;
Yao teaches “the training data and validation are normally split from the same data set with a certain ratio in terms of the quantity of the data examples, and therefore, they have the same pattern. The training data set is used to create (e.g., fine-tune) the ML entity, while the validation data set is used to qualify performance of the trained entity” (Para. [0037]) thereby teaching generating an output of the trained entity for validation of the labeled dataset.
Generating, by the one or more processors and using a natural language machine learning model, a refined labeled dataset by:
Yao also teaches “ ML models use one or more features to make predictions or inferences. In some implementations, new features can be derived from old features” (Para. [0202]) thereby teaching refining old features to derive new features.
(i) generating one or more second prompts by modifying the one or more first prompts based on an uncertainty score that exceeds a threshold, wherein the uncertainty score is associated with a validation classification output form the plurality of validation classification outputs, and
Yao teaches “The Nudr service-based interface may be exhibited by the UDR to allow the UDM 358, PCF 356, and NEF 352 to access a particular set of the stored data, as well as to read, update (e.g., add, modify), delete, and subscribe to notification of relevant data changes in the UDR” (Para. [0147]) and “in some implementations, new features can be derived from old features” (Para. [0202]) where “the existing attribute (e.g., model Performance Training) is enhanced or extended to include the performance scores of the ML entity when performing on the training data and validation data respectively. In some examples, the modelPerformanceTraining attribute is enhanced to include at least two information elements (IEs), including: a first IE is for indicating the performance score of the ML entity when performing on the training data, and a second IE is for indicating performance score of the ML entity when performing on the validation data.” (Para. [0082]). Yao further teaches “the ModelPerformance data type is also used to specify the performance of an ML entity when performing validation. Here, a validation performance score is provided for each inference output based on corresponding validation data.” (Para. [0101]).
Yao further teaches exceeding a threshold by teaching “ ML training management involves allowing the MnS consumer to request and/or manage the model training/retraining. For example, activating/deactivating, training performance management and setting policy for the producer-initiated ML training (e.g., the conditions to trigger the ML (re-)training based on the AI/ML inference performance or AI/ML inference trustworthiness). In some implementations, ML training management may be based on the performance evaluation results observed by the model performance monitoring (performance data and/or feedback). For example, if the model performance decreases, the AI/ML performance management capability may trigger the MLT to start retraining.” (Para. [0061])
(ii) providing the one or more second prompts to the machine learning model to replace a label of the plurality of labels in the labeled dataset with a corrective label, wherein the label corresponds to the validation classification output; and
Yao teaches “The Nudr service-based interface may be exhibited by the UDR to allow the UDM 358, PCF 356, and NEF 352 to access a particular set of the stored data, as well as to read, update (e.g., add, modify), delete, and subscribe to notification of relevant data changes in the UDR” (Para. [0147]) and “in some implementations, new features can be derived from old features” (Para. [0202]) where “If the variance is not acceptable, the entity would need to be tuned and/or re-trained before being made available to the consumer and/or used for inference/prediction.” (Para. [0036]) and “The MLT may be initiated by the MLT MnS producer, for example, as a result of performance evaluation of the ML model, based on feedback or new training data received from the MLT MnS consumer, and/or when new training data which are not from the MLT MnS consumer describing the new network status/events become available.” (Para. [0031]) where
Therefore, Yao teaches refining/replacing old features corresponding to the validation classification output to derive new features by modifying data (new features derived from old features) and then providing the modified data prompts/input to the machine learning model for retraining.
Generating, by the one or more processors, a second instance of the classification machine learning model by fine-tuning the first instance of the classification machine learning model based on the refined labeled dataset;
Yao teaches “during the MLT process, the generated ML entity…needs to be validated” and “if the variance is not acceptable, the entity would need to be tuned and/or re-trained before being made available to the consumer and/or used for inference/prediction” (Para. [0036]) thereby teaching generating a second instance of the ML model with different second parameter values. Yao further teaches “the training data set is used to create (e.g., fine-tune) the ML entity, while the validation data set is used to qualify performance of the trained entity.” (Para. [0037] where “In some implementations, new features can be derived from old features” (Para. [0202]).
Wherein:
(i) the second instance of the classification machine learning model generates an inference classification output based on inference input data,
Yao teaches “For purposes of the present disclosure, the term “inference” refers to the process of using trained AI/ML model(s) to generate predictions, decisions, data analytics, actions, policies, configurations, and/or the like based on new, unseen data (e.g., “input inference data”)” (Para. [0189]).
(ii) the inference input data comprises an unlabeled document data object,
Yao teaches “Each input-output pair includes an input object (e.g., a vector) and a desired output object or value (referred to as a “supervisory signal”).” (Para. [0323]) where “The training data selection/filter 720 may label data in datasets for supervised learning, or the data may remain unlabeled for unsupervised learning. The produced datasets may then be fed into the MLT function (MLTF) 725.” (Para. [0186]).
(iii) the inference classification output comprises a document classification,
Yao teaches “Classification algorithms may describe an individual (data) instance whose category is to be predicted using a feature vector. As an example, when the instance includes a collection (corpus) of text, each feature in a feature vector may be the frequency that specific words appear in the corpus of text.” and “ML algorithms for classification may be referred to as a ‘classifier.’” (Para. [0279]) thereby teaching the output of the the ML algorithm classifier as a classification of the document (corpus of text).
(iv) a performance of one or more prediction-based actions is initiated based on the inference classification output, and
Yao teaches “The inference engine 745 may consume the inference dataset provided by the inference data selection/filter 750, and generate one or more inferences. The inferences may be or include, for example, statistical inferences, predictions, probabilities and/or probability distributions, actions, configurations, policies, data analytics, outcomes, optimizations, and/or the like. The outcome(s) may be provided to the performance measurement function 730.” (Para. [0191]) thereby teaching inferring an action to be performed/initiated.
Yao explicitly teaches all of the elements of the claimed invention as recited above except:
The machine learning model being a natural language machine learning model for labeling data;
Wherein:
(i) the one or more first prompts describe:
(a) a data labeling task to perform on the unlabeled data, and
(b) a desired labeled dataset as an output of the data labeling task, and
Generating, by the one or more processors, a refined labeled dataset by modifying a label of the labeled dataset based on an uncertainty score of a plurality of uncertainty scores associated with the plurality of validation classification outputs;
(v) the one or more prediction-based actions comprise generating an abstractive summary.
However, in the related field of endeavor of utilizing artificial intelligence for processing and summarizing speech, Chintagunta teaches:
The machine learning model being a natural language machine learning model for labeling data;
Chintagunta teaches “the introduction of natural language understanding (NLU) systems for providing real-time decision support” (Para. [0031]) where “Example embodiments provide a medically-aware the summarization component 130, e.g., a Machine Learning (ML) model data labeler, GPT-3-ENS, that combines medical knowledge and an ensemble of GPT-3 for the purpose of medical dialogue summarization. While GPT-3 is used in an example, other machine learning models (large language models) may be used.” (Para. [0131]).
Wherein:
(i) the one or more first prompts describe:
(a) a data labeling task to perform on the unlabeled data, and
Chintagunta teaches “The component then dynamically constructs few-shot prompts for tasks by conditioning on relevant patient information and use a generative machine learning model (e.g., GPT-3) as the backbone.” (Para. [0030]) which few-shot prompting is understood as using a prompt that necessarily includes a task/instructions and pairs of a few exemplary inputs and their corresponding exemplary/desired outputs to define a desired output for a particular input.
(b) a desired labeled dataset as an output of the data labeling task, and
Chintagunta teaches “The component then dynamically constructs few-shot prompts for tasks by conditioning on relevant patient information and use a generative machine learning model (e.g., GPT-3) as the backbone.” (Para. [0030]) which few-shot prompting is understood as using a prompt that necessarily includes a task/instructions and pairs of a few exemplary inputs and their corresponding exemplary/desired outputs to define a desired output for a particular input.
(v) the one or more prediction-based actions comprise generating an abstractive summary.
It is noted in advance that the claim limitation is being given its broadest reasonable interpretation, and that the claim limitation does not specify what the abstractive summary is a summary of. Chintagunta teaches “For PEGASUS, the summarization performance improves drastically compared to model fine-tuned using only the human labeled data. We hypothesize that data generated from GPT-3-ENS can serve as quality training data for abstractive models such as PEGASUS but not so much for hybrid models such as DRSUM due to GPT-3 being a generative language model. The summaries written by our human doctors have writing structure similar to that of a hybrid summarization model such as DRSUM that is more extractive in nature. This can explain why DRSUM did not show performance gain when using generated data from GPT-3-ENS. The key, however, is that it still did perform on par.” (para. [0167]) and “From Table 9, we observe that for both PEGASUS and DRSUM, mixture of human labeled and GPT-3-ENS data consistently improves almost all automated metrics for all α values.sup.1 The lift in metrics is lower for DRSUM, again illustrating the idea we highlighted of GPT-3-ENS data being more amenable to abstractive models such as PEGASUS than for hybrid or extractive-biased models such as DRSUM. Table 8 provides qualitative comparison between summaries generated by each of these models.” (Para. [0171]).
Thus, it would have been obvious to one of ordinary skill in the art, having the teachings of Chintagunta and Yao at the time that the claimed invention was effectively filed, to have combined the natural language processing for labeling natural language speech data, as taught by Chintagunta, with the systems and methods for ML entity lifecycle management, as taught by Yao.
One would have been motivated to make such combination because Chintagunta teaches “the example iterative approach can improve labeling quality (and hence the model) even when the initial labels are quite noisy” (Para. [0075) and it would be obvious to a person having ordinary skill in the art to desire improved labeling quality even when initial labels are noisy.
Chintagunta and Yao explicitly teach all of the elements of the claimed invention as recited above except:
Generating, by the one or more processors, a refined labeled dataset by modifying a label of the labeled dataset based on an uncertainty score of a plurality of uncertainty scores associated with the plurality of validation classification outputs;
However, in the related field of endeavor of classifying unlabeled data, Do906 teaches:
Generating, by the one or more processors, a refined labeled dataset by modifying a label of the labeled dataset based on an uncertainty score of a plurality of uncertainty scores associated with the plurality of validation classification outputs;
Do906 teaches “If a desired level of performance is not achieved, then the labeled data may be refined at step 360, by determining if a confidence value for the data is above a certain threshold at step 370. A confidence value may be the same as the desired performance and use the same metrics, or a confidence value may be a classification accuracy that describes the percentage of the time or the frequency with which an image feature or region is identified correctly.” (Para. [0047]).
Thus, it would have been obvious to one of ordinary skill in the art, having the teachings of Do906, Chintagunta, and Yao at the time that the claimed invention was effectively filed, to have combined the label refinement based on a confidence value, as taught by Do906, with the natural language processing for labeling natural language speech data, as taught by Chintagunta, and the systems and methods for ML entity lifecycle management, as taught by Yao.
One would have been motivated to make such combination because Do906 explicitly teaches a quality standard/threshold for labeling data (Para. [0047]) and an “iterative data curation process can increase the quantity of annotated dataset and improve the quality of dataset.” (Para. [0048]) where “deep learning classification accuracy is historically dependent on the size of the initial training datasets. Quantifying the size of a dataset required to achieve a target accuracy is important when trying to decide the feasibility of a system.” (Para. [0049]).
Regarding Claim 2:
Do906, Chintagunta, and Yao further teach:
Wherein the natural language machine learning model comprises a generative pre-trained transformer.
Chintagunta teaches “our approach GPT-3-ENS is able to generate a large amount of training data for both pre-trained summarization models” (Para. [0166]) where GPT is understood as standing for Generative Pre-trained Transformer. Ghintagunta further teaches “train a transformer-based model, which is then applied on sentences to create noisy sentence-level labels” (Para. [0029]) and “The model itself comprises a transformer language model DeCLUTR sentence encoder, with a classification head comprising a single feed-forward layer with a softmax activation” (Para. [0070]) where DeCLUTR is understood as being a machine learning model designed for natural language processing.
Regarding Claim 3:
Do906, Chintagunta, and Yao further teach:
Wherein generating the labeled dataset comprises:
Providing the one or more first prompts to the natural language machine learning model; and
Chintagunta teaches “submitting at least one prompt including the text to a generative artificial intelligence.” (Para. [0262]).
assigning, by the natural language machine learning model, the plurality of labels to the unlabeled data based on the one or more first prompts.
Yao teaches “the training data selection/filter 720 is configured to generate training, validation, and testing datasets for MLT (or ML model training)” where “the training data selection/filter 720 may label data in datasets for supervised learning” (Para. [0186]) thereby teaching generating a labeled dataset from unlabeled data for use in supervised learning/training, validation, and testing.
Yao further teaches labeling the data based on prompts associated with a data labeling task by teaching “supervised learning involves learning a function or model that maps an input to an output based on example input-output pairs or some other form of labeled training data including a set of training examples” (Para. [0323]) and natural language processing and/or natural language understanding being learned through model parameters “in the context of ML” (Para. [0331]).Chintagunta more explicitly teaches “our approach GPT-3-ENS is able to generate a large amount of training data for both pre-trained summarization models” (Para. [0166]) where GPT is understood as standing for Generative Pre-trained Transformer. Ghintagunta further teaches “train a transformer-based model, which is then applied on sentences to create noisy sentence-level labels” (Para. [0029]) and “The model itself comprises a transformer language model DeCLUTR sentence encoder, with a classification head comprising a single feed-forward layer with a softmax activation” (Para. [0070]) where DeCLUTR is understood as being a machine learning model designed for natural language processing..
Therefore, Yao in combination with Chintagunta teach assigning one or more labels to the unlabeled data based on one or more prompts and natural language processing/understanding in the context of ML model (Yao) where the ML model is a natural language machine learning model that assigns the labels (Chintagunta).
Regarding Claim 4:
Do906, Chintagunta, and Yao further teach:
Wherein the one or more first prompts comprise an input sample that is associated with the data labeling task description.
Yao teaches “supervised learning involves learning a function or model that maps an input to an output based on example input-output pairs or some other form of labeled training data including a set of training examples” (Para. [0323]) thereby teaching the prompt comprising an input sample associated with labeled data
Regarding Claim 5:
Do906, Chintagunta, and Yao further teach:
Wherein the one or more first prompts comprise one or more input-output pair examples that are associated with the data labeling task decription.
Yao teaches “supervised learning involves learning a function or model that maps an input to an output based on example input-output pairs or some other form of labeled training data including a set of training examples” (Para. [0323]).
Regarding Claim 6:
Do906, Chintagunta, and Yao further teach:
Determining a plurality of uncertainty scores based on a plurality of prediction probabilities or a plurality of classification margins that are associated with the plurality of validation classification outputs.
Yao teaches “the inference engine 745 may consume the inference dataset provided by the inference data selection/filter 750 [from the data repository 715], and generate one or more inferences. The inferences may be or include, for example, statistical inferences, predictions, probabilities and/or probability distributions, actions, configurations, policies, data analytics, outcomes, optimizations, and/or the like. The outcome(s) may be provided to the performance measurement function 730” (Para. [0191]).
Yao further teaches “the existing attribute (e.g., model Performance Training) is enhanced or extended to include the performance scores of the ML entity when performing on the training data and validation data respectively. In some examples, the modelPerformanceTraining attribute is enhanced to include at least two information elements (IEs), including: a first IE is for indicating the performance score of the ML entity when performing on the training data, and a second IE is for indicating performance score of the ML entity when performing on the validation data.” (Para. [0082]).
Regarding Claim 7:
Do906, Chintagunta, and Yao further teach:
Determining the plurality of prediction probabilities;
Yao teaches “the inference engine 745 may consume the inference dataset provided by the inference data selection/filter 750 [from the data repository 715], and generate one or more inferences. The inferences may be or include, for example, statistical inferences, predictions, probabilities and/or probability distributions, actions, configurations, policies, data analytics, outcomes, optimizations, and/or the like. The outcome(s) may be provided to the performance measurement function 730” (Para. [0191]).
Generating the plurality of uncertainty scores for the plurality of validation classification outputs based on the plurality of prediction probabilities; and
Yao teaches “the inference engine 745 may consume the inference dataset provided by the inference data selection/filter 750 [from the data repository 715], and generate one or more inferences. The inferences may be or include, for example, statistical inferences, predictions, probabilities and/or probability distributions, actions, configurations, policies, data analytics, outcomes, optimizations, and/or the like. The outcome(s) may be provided to the performance measurement function 730” (Para. [0191]).
Yao further teaches “the existing attribute (e.g., model Performance Training) is enhanced or extended to include the performance scores of the ML entity when performing on the training data and validation data respectively. In some examples, the modelPerformanceTraining attribute is enhanced to include at least two information elements (IEs), including: a first IE is for indicating the performance score of the ML entity when performing on the training data, and a second IE is for indicating performance score of the ML entity when performing on the validation data.” (Para. [0082]).
Therefore, Yao teaches generating the performance scores for the validation data based on the inferences (predictions, probabilities and/or probability distributions) from inference engine 745.
Determining one or more of the plurality of validation classification outputs comprising either:
(i) one or more top percentile uncertainty scores or
(ii) one or more uncertainty scores from the plurality of uncertainty scores, that exceeds the threshold.
Yao teaches “the term ‘quantile function’ may also be referred to as a percentile function” (Para. [0316]) where “to obtain the valid training outcomes, consumers may also designate their requirements for model performance (e.g., accuracy, momentum, precision, quantile, recall/sensitivity, model bias, run-time latency, resource consumption (e.g., memory utilization, processor utilization, network utilization, and the like), and/or other suitable metrics/measures, such as any of those discussed herein) in the training request.” (Para. [0024]) and “the performance measurement function 730 is configured to measure model performance metrics (e.g., accuracy, momentum, precision, quantile, recall/sensitivity, model bias, run-time latency, resource consumption, and/or other suitable metrics/measures” (Para. [0192]).
Regarding Claim 8:
Do906, Chintagunta, and Yao further teach:
Determining the plurality of classification margins by:
For at least one of the plurality of validation classification outputs, determining a first prediction and a second prediction that are determined by the first instance of the classification machine learning model during the generating of at least one validation classification output of the plurality of validation classification outputs; and
Yao teaches “Example 7 includes the method of example 6 and/or some other example(s) herein, wherein performance score of the ML entity when performing on the validation data is provided by a dedicated (new) attribute of the MLTrainingReport IOC, e.g., the respective performance scores of the ML entity when performing on the training data and validation data are reported in different attributes.” (Para. [0223]). Therefore, Yao teaches determining first and second predictions by the ML entity during the generating of at least one validation classification output that is reported in different attributes, thus determining a difference between the two in the report.
Wherein the first prediction is associated with a first highest prediction score and the second prediction is associated with a second highest prediction score; and
Yao teaches “Example 7 includes the method of example 6 and/or some other example(s) herein, wherein performance score of the ML entity when performing on the validation data is provided by a dedicated (new) attribute of the MLTrainingReport IOC, e.g., the respective performance scores of the ML entity when performing on the training data and validation data are reported in different attributes.” (Para. [0222]). Yao further teaches “the existing attribute...is enhanced or extended to include the performance scores of the ML entity” (Para. [0082]) which because there are scores (plural), there must be a first highest and second highest score. Therefore, Yao teaches determining first and second predictions, each associated with prediction performance scores, by the ML entity during the generating of a validation classification output that is reported in different attributes, thus determining a difference between the two in the report.
Determining a difference between the first prediction and the second prediction.
Yao teaches “Example 7 includes the method of example 6 and/or some other example(s) herein, wherein performance score of the ML entity when performing on the validation data is provided by a dedicated (new) attribute of the MLTrainingReport IOC, e.g., the respective performance scores of the ML entity when performing on the training data and validation data are reported in different attributes.” (Para. [0223]). Therefore, Yao teaches determining first and second predictions by the ML entity during the generating of a validation classification output that is reported in different attributes, thus determining a difference between the two in the report.
Regarding Claim 9:
Some of the limitations herein are similar to some or all of the limitations of Claim 1.
Do906, Chintagunta, and Yao further teach:
A computing system comprising memory and one or more processors communicatively coupled to the memory, the one or more processors configured to perform steps.
Yao teaches “hardware resources 600 including one or more processors (or processor cores) 610, one or more memory/storage devices 620, and one or more communication resources 630, each of which may be communicatively coupled via a interconnect (IX) 606 or other interface circuitry, which implement any suitable bus and/or IX technologies” (Para. [0175]).
Regarding Claim 10:
All of the limitations herein are similar to some or all of the limitations of Claim 3.
Regarding Claim 11:
All of the limitations herein are similar to some or all of the limitations of Claim 4.
Regarding Claim 12:
All of the limitations herein are similar to some or all of the limitations of Claim 5.
Regarding Claim 13:
All of the limitations herein are similar to some or all of the limitations of Claim 6.
Regarding Claim 14:
All of the limitations herein are similar to some or all of the limitations of Claim 7.
Regarding Claim 15:
All of the limitations herein are similar to some or all of the limitations of Claim 8.
Regarding Claim 16:
Some of the limitations herein are similar to some or all of the limitations of Claim 1.
Do906, Chintagunta, and Yao further teach:
One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to perform steps.
Yao teaches “FIG. 6 illustrates components (hardware resources 600) capable of reading instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium) and perform any one or more of the methodologies discussed herein” and “hardware resources 600 including one or more processors (or processor cores) 610, one or more memory/storage devices 620, and one or more communication resources 630, each of which may be communicatively coupled via a interconnect (IX) 606 or other interface circuitry, which implement any suitable bus and/or IX technologies” (Para. [0175]).
Regarding Claim 17:
All of the limitations herein are similar to some or all of the limitations of Claim 3.
Regarding Claim 18:
All of the limitations herein are similar to some or all of the limitations of Claim 6.
Regarding Claim 19:
All of the limitations herein are similar to some or all of the limitations of Claim 7.
Regarding Claim 20:
All of the limitations herein are similar to some or all of the limitations of Claim 8.
Response to Amendment
Applicant’s Amendments, filed on 2/27/2026, are acknowledged and accepted.
Response to Arguments
On page 12 of the Remarks filed on 2/27/2026, Applicant argues that “Yao does not provide a teaching or suggestion of " ... one or more first prompts ... [that] describe: (a) a data labeling task to perform on the unlabeled data, and (b) the labeled dataset as an output of the data labeling task," as recited in amended claims 1, 9, and 16.”.Applicant’s argument is convincing that Yao does not teach the amended language, and thus necessitates the new grounds of rejection presented above addressing the amended language.
On pages 12-13 of the Remarks filed on 2/27/2026, Applicant argues that “An alleged deriving of new features from old features, as described by Yao, does not teach or suggest replacing a label. In particular, Yao fails to teach or suggest at least "(ii) providing the one or more second prompts to the natural language machine learning model to replace a label of the plurality of labels in the labeled dataset with a corrective label ... ," as recited in amended claims 1, 9, and 16.”Applicant’s argument of the amended language is not convincing to overcome the Yao reference and has been fully addressed in the rejection above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Gao et al. (U.S. Pre-Grant Publication No. 2022/0366317) teaches a field extraction system that does not require field-level annotations for training. Specifically, the training process is bootstrapped by mining pseudo-labels from unlabeled forms using simple rules. Then, a transformer-based structure is used to model interactions between text tokens in the input form and predict a field tag for each token accordingly. The pseudo-labels are used to supervise the transformer training. As the pseudo-labels are noisy, a refinement module that contains a sequence of branches is used to refine the pseudo-labels. Each of the refinement branches conducts field tagging and generates refined labels. At each stage, a branch is optimized by the labels ensembled from all previous branches to reduce label noise.The reference further teaches “In view of the need for an efficient system for information extraction from form documents, embodiments described a field extraction system that does not require field-level annotations for training. Specifically, the training process is bootstrapped by mining pseudo-labels from unlabeled forms using simple rules. Then, a transformer-based structure is used to model interactions between text tokens in the input form and predict a field tag for each token accordingly. The pseudo-labels are used to supervise the transformer training. As the pseudo-labels are noisy, a refinement module that contains a sequence of branches is used to refine the pseudo-labels. Each of the refinement branches conducts field tagging and generates refined labels. At each stage, a branch is optimized by the labels ensembled from all previous branches to reduce label noise.” (Para. [0018]).
Creed et al. (U.S. Pre-Grant Publication No. 2021/0117815) teaches filtering a set of data, the set of data comprising multiple data instances by: receiving a set of scores for the set of data; determining attention filtering information based on prior knowledge of one or more relationships between the data instances in said set of data and calculating attention relevancy weights corresponding to the data instances and the set of scores; and providing the attention filtering information to a machine learning, ML, technique or ML model.The reference further teaches “For simplicity, the following description uses MIL on natural language processing, by way of example only, to describe an attention mechanism according to the invention. Although the present invention may be described based on natural language processing in which the labelled training dataset is based on sentences from a corpus of literature/citations, it is to be appreciated by the skilled person that the invention may use any type of labelled training dataset as the application demands” (Para. [0089]).The reference further teaches “The attention mechanism may use an attention neural network to determine attention filtering information for each set of data based on calculating attention relevancy weights (also known as attention scores or attention relevancy scores) that represent the relevancy of each data instance (e.g. a labelled training data instance) in each set of data, a set of scores or potentials in relation to each set of data, and using prior knowledge of the relationships between pairs of data instances (e.g. every pair of training data instances)” (Para. [0090]).
Cheng et al. (U.S. Pre-Grant Publication No. 2021/0201076) teaches training a first machine learning model, through supervised training, based on the generated comparison matrices and based on probabilistic labels generated by a second machine learning model.
Su et al. (U.S. Pre-Grant Publication No. 2010/0169243) teaches a computer-implemented system and method for text classification is provided that applies a hybrid approach for text classification. The system and method includes a text pre-processor which prepares unclassified articles in a format which can be read by a two-stage classifier. The classifier employs a hybrid approach. A keyword-based model achieves machine-labelling of the articles. The machine-labelled articles are used to train a machine learning model. New articles can be applied against the trained model, and classified.The reference further teaches “a. receiving a set of unlabelled and a set of initially labelled documents; b. applying a keyword model to machine label the set of unlabelled documents, to produce a set of labelled (keyword) documents; c. training a machine learning model using the set of labelled (keyword) documents and the set of initially labelled documents; d. labelling a selected document using the machine learning model to produce an associated label; and e. storing the selected document and the associated label.” (Paras. [0013]-[0017]).The reference further teaches “ f the pre-defined threshold is not reached, then the machine (keyword model) labelled documents plus the human-labelled documents are used to train (or update) a supervised learning algorithm or model. In a further preferred embodiment, after the model has been trained, a few documents are selected at random (or through another approach) and human-labelled. Alternatively, further human labelled documents can otherwise be added to the training set. The machine learning model is updated using this augmented training set. Again, the AUC (or other measure) is scored and compared to a pre-defined threshold. These steps can be repeated to further train the model until a pre-defined threshold is reached. Thereafter, new documents can be applied against the trained model, and classified.” (Para. [0043]).
Zhang et al. (U.S. Pre-Grant Publication No. 2021/0056417) teaches a method for active learning includes obtaining a set of unlabeled training samples and for each unlabeled training sample, perturbing the unlabeled training sample to generate an augmented training sample. The method includes generating, using a machine learning model, a predicted label for both samples and determining an inconsistency value for the unlabeled training sample that represents variance between the predicted labels for the unlabeled and augmented training samples. The method includes sorting the unlabeled training samples based on the inconsistency values and obtaining, for a threshold number of samples selected from the sorted unlabeled training samples, a ground truth label. The method includes selecting a current set of labeled training samples including each selected unlabeled training samples paired with the corresponding ground truth label. The method includes training, using the current set and a proper subset of unlabeled training samples, the machine learning model.
Banis et al. (U.S. Patent No. 11,449,775) teaches a modular machine learning-as-a-service (MLAAS) system uses machine learning to respond to tasks without requiring machine learning modeling or design knowledge by its users. The MLAAS system receives an inference request including a model identifier and a target defining features for use in processing the inference request. The features correspond to a task for evaluation using a machine learning model associated with the model identifier. An inference outcome is generated by processing the inference request using the target as input to the model. Feedback indicating an accuracy of the inference outcome with respect to the task is later received and used to generate a training data set, which the MLAAS can use to further train model used to generate the inference outcome. As a result, the training of a machine learning model by the MLAAS system is limited to using data resulting from an inference performed using that model.
Foreign Publication CN117216668A teaches comparing whether the original label of the preset training data is consistent with the output label; if not, modifying the labels in the set of predetermined training data and adjusting the neural network model using the modified predetermined training data.
Ramakrishna et al. (U.S. Pre-Grant Publication No. 2024/0296838) teaches techniques for updating a machine learning (ML) model are described. A device or system may receive input data corresponding to a natural or non-natural language (e.g., gesture) input. Using a first ML model, the device or system may determine the input data corresponds to a data category of a plurality of data categories. Based on the data category, the device or system may select a ML training type from among a plurality of ML training types. Using the input data, the device or system may perform the selected ML training type with respect to a runtime ML model to generate an updated ML model.
Trummer (U.S. Pre-Grant Publication No. 2024/0281222) teaches at least one processing device is configured to receive natural language input from one or more user devices relating at least in part to data of one or more databases, to apply the natural language input to an artificial intelligence (AI) system, to generate in the AI system database code for the one or more databases based at least in part on the natural language input, and to execute the generated database code against at least a portion of the one or more databases. The natural language input may be received in association with a database query, which is decomposed into a sequence of processing steps formulated in natural language using corresponding text templates, and from which one or more prompts are generated for application to the AI system.
Rowan et al. (U.S. Patent No. 11,893,461) teaches systems and methods for labeling data are disclosed. An example method may be performed by one or more processors of a labeling system and include retrieving labeled data, identifying characteristics predictive of labels that would be entered for unlabeled data items having the respective characteristics based on the labeled data, training an analysis model to predict labels that would be entered for unlabeled data items, generating, for unlabeled data items, using the trained analysis model, a prediction of a label that will be entered for the respective unlabeled data item if the respective unlabeled data item is presented for labeling, selecting, based on the generated predictions, a subset of unlabeled data items to be presented for labeling, receiving labels for the subset of unlabeled data items, determining that a completion criteria associated with the trained analysis model is met, and generating labels for remaining unlabeled data items.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT F MAY whose telephone number is (571)272-3195. The examiner can normally be reached Monday-Friday 9:30am to 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached on 571-270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBERT F MAY/Examiner, Art Unit 2154 5/14/2026
/BORIS GORNEY/Supervisory Patent Examiner, Art Unit 2154
Prosecution Insights