DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d) based on an application filed in REPUBLIC OF KOREA on March 3, 2022 and on August 23, 2022. The certified copy has been filed in parent Application No. 18/173,661, filed on February 23, 2023. Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., an abstract idea) without significantly more.
Regarding claim 1 and analogous claim 19, 20:
Step 1 (whether a claim is to a statutory category):
Yes, the claim is within the four statutory categories (a process, machine, manufacture or composition of matter). Claim 1 recites a method, therefore, falls within a process category. Claim 19 recites a non-transitory computer-readable medium, therefore, falls within a manufacture category. Claim 20 recites an apparatus, therefore, falls within a machine category.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “acquiring, based on a classification model, classification information on training data included in a first dataset;” describes a mental process (observation, evaluation) wherein receiving a dataset and its labels recites concepts that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). And “acquiring, based on the classification information and a set of one or more classification information acquired based on the classification model in one or more previous epochs, target training dynamics information;” describes a mental process (observation, evaluation, judgement) wherein receiving information and past model information to compare and judge a target value based on previous data recites concepts that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III). And “acquiring, based on the training dynamics prediction model, predictive training dynamics information on the training data; and” describes a mental process (observation, evaluation) wherein receiving information about data recites concepts that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “training, based on the target training dynamics information and the predictive training dynamics information, the training dynamics prediction model” the claim does not recite additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Therefore, claim 1 is ineligible.
Regarding claim 2:
Further modifies the abstract idea of claim 1.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “wherein training the training dynamics prediction model includes acquiring loss information based on the target training dynamics information and the predictive training dynamics information” wherein acquiring loss information recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP 2106.04(a)(2), (I)).
Therefore, claim 2 is ineligible.
Regarding claim 3:
Further modifies the abstract idea of claim 2.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “wherein acquiring the loss information includes acquiring a Kullback-Leibler divergence value based on the target training dynamics information and the predictive training dynamics information.” wherein acquiring loss information using recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP 2106.04(a)(2), (I)).
Therefore, claim 3 is ineligible.
Regarding claim 4:
Further modifies the abstract idea of claim 1.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein the first dataset includes one or more data pre-labeled with a class.” the claim does not recite additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Therefore, claim 4 is ineligible.
Regarding claim 5:
Further modifies the abstract idea of claim 4.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “further comprising training the classification model based on the classification information and the pre-labeled class on the training data included in the first dataset.” the claim does not recite additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Therefore, claim 5 is ineligible.
Regarding claim 6:
Further modifies the abstract idea of claim 4.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “wherein training the training dynamics prediction model includes acquiring loss information based on the classification information and the pre-labeled class on the training data included in the first dataset.” wherein acquiring loss information using recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP 2106.04(a)(2), (I)).
Therefore, claim 6 is ineligible.
Regarding claim 7 and analogous claim 17:
Further modifies the abstract idea of claim 6.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “wherein acquiring the loss information includes acquiring a cross-entropy loss value based on the classification information and the pre- labeled class.” wherein acquiring loss information using recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP 2106.04(a)(2), (I)).
Therefore, claim 7 is ineligible.
Regarding claim 8:
Further modifies the abstract idea of claim 6.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “determining, based on the classification information, a class to which the training data included in the first dataset is most likely to belong; and checking whether the determined class matches the pre-labeled class.” describes a mental process (evaluation, judgement) wherein evaluating data and comparing values, then determining if it matches recites concepts that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Therefore, claim 8 is ineligible.
Regarding claim 9:
Further modifies the abstract idea of claim 1.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “wherein acquiring the target training dynamics information includes calculating, based on the classification information and the set of one or more classification information, average values of probability of data belonging to each class.” recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP 2106.04(a)(2), (I)), and describes a mental process (evaluation, judgement) wherein evaluating data and comparing values, then determining if it matches recites concepts that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Therefore claim 9 is ineligible.
Regarding claim 10 and analogous claim 14:
Further modifies the abstract idea of claim 1.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein acquiring the predictive training dynamics information includes: acquiring, based on the classification model, hidden feature information on the training data; and acquiring, based on the hidden feature information, the predictive training dynamics information.” does not recite additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Therefore, claim 10 is ineligible.
Regarding claim 11:
Step 1 (whether a claim is to a statutory category):
Yes, the claim is within the four statutory categories (a process, machine, manufacture or composition of matter). Claim 1 recites a method, therefore, falls within a process category.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “acquiring a result of labeling the selected data with a corresponding class;” describes a mental process (observation) wherein receiving data results with its labels recites concepts that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “a first operation of training, based on a first dataset, the classification model and a training dynamics prediction model;” And “selecting, using the training dynamics prediction model, some data from a second dataset;” And “a second operation of training, based on the result of labeling the selected data with the corresponding class, the classification model and the training dynamics prediction model, wherein, the first dataset includes one or more data pre-labeled with a class, and” And “the second dataset includes one or more data not pre-labeled with a class.” does not recite additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), such as mere (i.e., selecting a particular data source or type of data to be manipulated) to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Therefore, claim 11 is ineligible.
Regarding claim 12:
Further modifies the method of claim 11.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “acquiring, based on the classification model, target training dynamics information corresponding to training data included in the first dataset;” And “acquiring, based on the training dynamics prediction model, predictive training dynamics information corresponding to the training data included in the first dataset; and” And “acquiring, based on the target training dynamics information and the predictive training dynamics information, loss information” describes a mental process (observation) wherein receiving data information recites concepts that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Therefore claim 12 is ineligible.
Regarding claim 13:
Further modifies the method of claim 12.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “wherein training the classification model and the training dynamics prediction model includes acquiring a Kullback-Leibler divergence value based on the target training dynamics information and the predictive training dynamics information” recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP 2106.04(a)(2), (I)).
Therefore claim 13 is ineligible.
Regarding claim 15:
Further modifies the method of claim 11.
Step 2A Prong 2 (evaluate whether the claim recites additional elements that integrate the exception into a practical application):
No, “wherein the second operation includes: merging the result of labeling the selected data with the corresponding class into the first dataset; and performing the first operation again using the merged result as a new first dataset.” does not recite additional elements that integrate the judicial exception into a practical application with the words "apply it" (or an equivalent), to implement an abstract idea on a computer (see MPEP 2106.05(f)).
Step 2B (Inventive concept):
No, it does not add significantly more since the intended practical application is well-understood, routine, and conventional and stated at a generic level (i.e. “apply it”, see MPEP 2106.05(f)).
Therefore, claim 15 is ineligible.
Regarding claim 16:
Further modifies the method of claim 11.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “acquiring, based on the classification information and the pre-labeled class on the training data included in the first dataset, loss information, and” and “wherein the classification information includes a result of calculating, for each of a plurality of classes, probability of data belonging to that class” recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP 2106.04(a)(2), (I)). And “acquiring, based on the classification model, classification information on training data included in the first dataset; and” describes a mental process (observation) wherein receiving data information recites concepts that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Therefore claim 16 is ineligible.
Regarding claim 18:
Further modifies the method of claim 11.
Step 2A Prong 1 (whether a claim is directed to a judicial exception):
Yes, “calculating, based on the predictive training dynamics information, uncertainty for each of the one or more data included in the second dataset; and” and “determining some data to be selected from the second dataset based on a result of calculating the uncertainty, wherein, the predictive training dynamics information includes a result of calculating probability for each of a plurality of classes that the data belongs to that class” recites a mathematical concept, as it involves mathematical relationships, mathematical formulas or equations, mathematical calculations (see MPEP 2106.04(a)(2), (I)). And “acquiring, based on the training dynamics prediction model, corresponding predictive training dynamics information for each of the one or more data included in the second dataset;” describes a mental process (observation, evaluation) wherein receiving data information recites concepts that can practically be performed in the human mind, with or without the use of a physical aid such as pen and paper (including an observation, evaluation, judgment, opinion). See MPEP § 2106.04(a)(2)(III).
Therefore claim 18 is ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 11 and 15 is/are rejected under 35 U.S.C. 102 102(a)(1) as being anticipated by Zhang et al., Non-Patent Literature “Cartography Active Learning.”
Regarding claim 11:
Zhang teaches:
A method for training a classification model for outputting a classification result corresponding to input data, the method comprising: (Section 4.3, “this work uses two models for the AL setup solving the classification tasks. Input [corresponding to input data]: Labeled seed set L, Unlabeled set U, Total budget K, Number of queries n, Correctness threshold tcor = 0.2; 2 for i = 1, ..., n do 3 Ψ(L), Ψ(U) ← train main classifier θ on L [training a classification model], get representations of L and U; [outputting a classification result]”)
a first operation of training, based on a first dataset, the classification model and a training dynamics prediction model; (Section 4.3, “This work uses two models for the AL setup solving the classification tasks”…(Section 6, “we train a classifier on limited seed data (i.e., wherein the first operation of training is training the classifier, hence, ‘classification model’, based on seed data ‘first dataset’) maps to distinguish these regions from each other to select the most informative instances. We show empirically that our method is competitive or significantly outperforms various popular AL methods, and provide intuitions on why this is the case by using training dynamics (i.e., wherein uses two models is interpreted as using training dynamics prediction model)”))
selecting, using the training dynamics prediction model, some data from a second dataset; (Section 3, “we train a binary classifier on the seed set L (i.e., initial dataset) and apply it to U (i.e., wherein the U is interpreted as the second dataset) to select [selecting] the instances that are the closest to the decision boundary between ambiguous and hard-to-learn instances”…Algorithm 1, “6. for j=1, ..., K n do 7. xˆ ← argmin x∈Ψ(U) |0.5−Pθ 0(ˆy = 1 | x)|; 8. L ← L ∪ xˆ; 9. U ← U\xˆ; 10. End”)
acquiring a result of labeling the selected data with a corresponding class; and (Section 4, “The selected instances [selected data] are shown the withheld label (i.e., wherein label is interpreted as ‘corresponding class’) and added to the labeled set L and removed from U.”)
a second operation of training, based on the result of labeling the selected data with the corresponding class, the classification model and the training dynamics prediction model, (“The statistics of the instances are extracted after the selected top-50 batch is added to the seed set [labeling the selected data with the corresponding class]. Once the main model is trained again [second operation of training] on the increased seed set, we obtain the statistics of the previously added batch of 50.”)
wherein, the first dataset includes one or more data pre-labeled with a class, and the second dataset includes one or more data not pre-labeled with a class (Section 2, “a small set of labeled data L [first dataset] (i.e., wherein the dataset is pre-labeled ‘labeled data’) and a large pool of unlabeled data U [second dataset] (i.e., wherein the dataset is unlabeled data). Most AL algorithms start similarly: a model is fit to L to get access to Pθ(y | x), then apply a query strategy to get the best scored instance from U, label this instance and add it to L in an iterative process.”)
Regarding claim 15:
Zhang teaches the method of claim 11.
Zhang further teaches:
wherein the second operation includes: merging the result of labeling the selected data with the corresponding class into the first dataset; and performing the first operation again using the merged result as a new first dataset (Section 4, “The selected instances are shown the withheld label and added to the labeled set L and removed from U (i.e., wherein added to the labeled set L is interpreted as merging the result of the selected data to the first dataset)”…Section 5, “Once the main model is trained again on the increased seed set, we obtain the statistics of the previously added batch of 50 (i.e., wherein the main model is trained again is interpreted as performing the first operation again.)”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-7 and 12, 16-17, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al., Non-Patent Literature “Cartography Active Learning” in view of Yoo et al., Non-Patent Literature “Learning Loss for Active Learning.”
Regarding claim 1 analogous claim 19 and 20:
Zhang teaches:
acquiring, based on a classification model, classification information on training data included in a first dataset; (Section 3, paragraph 9, “To select presumably informative instances, we train a binary classifier [classification model] on the seed set L [first dataset] and apply it to U to select the instances that are the closest to the decision boundary between ambiguous and hard-to-learn instances (i.e., wherein classification information is interpreted as the measures of distance to the decision)”)
acquiring, based on the classification information and a set of one or more classification information acquired based on the classification model in one or more previous epochs, target training dynamics information; (Section 3, paragraph 1, “to use model-independent measures, from fitting the model on the seed data L [acquiring, based on the classification information], by using data maps (Swayamdipta et al., 2020) for AL. Data maps help identify characteristics [classification information] of instances within the broader trends of a dataset by leveraging their training dynamics [target training dynamics information] (i.e., the behavior of a model during training, such as mean and standard deviation of confidence and correctness with respect to the gold label) (i.e., wherein behavior of a model during training under the broadest reasonable interpretation (BRI) is interpreted as observations over time, hence, ‘previous epochs’). These model-dependent measures reveal distinct regions in a data map, by and large, reflecting instance properties (see Figure 1 and details below on easy-to-learn, ambiguous, and hard-to-learn instances). Training dynamics encapsulate information of data quality that has been largely ignored in AL: the sweet spot of instances at the boundary of hard-to-learn and ambiguous instances, which are quick to label while providing informative samples, as shown in full data training")
acquiring, based on the training dynamics prediction model, predictive training dynamics information on the training data; and (Algorithm 1, “Algorithm 1: Cartography Active Learning 1 input: Labeled seed set L, Unlabeled set U, Total budget K, Number of queries n, Correctness threshold tcor = 0.2; for i = 1, ..., n do 3 Ψ(L), Ψ(U) ← train main classifier θ on L, get representations of L and U; 4 µˆ, σˆ, φˆ ← get data map statistics of L with θ; [training dynamics information] 5 Pθ 0 ← train binary classifier θ 0 on Ψ(L) with yΨ(xˆi) = ( 1, if φˆ i > tcor 0, else [training dynamics prediction model] (i.e., wherein the classifier predicts, hence, ‘prediction model’) 6 for j=1, ..., K n do 7 xˆ ← argmin x∈Ψ(U) |0.5−Pθ 0(ˆy = 1 | x)|;”)
PNG
media_image1.png
648
463
media_image1.png
Greyscale
Zhang does not explicitly teach:
A method for training a training dynamics prediction model, the method comprising:
training, based on the target training dynamics information and the predictive training dynamics information, the training dynamics prediction model.
Yoo teaches:
A method for training a training dynamics prediction model, the method comprising: (Abstract, “active learning method that is simple but task-agnostic, and works efficiently with the deep networks. We attach a small parametric module, named “loss prediction module,” to a target network, and learn it to predict target losses of unlabeled inputs.”)
training, based on the target training dynamics information and the predictive training dynamics information, the training dynamics prediction model (Section 3.3, “We have a labeled dataset L s K·(s+1) and a model set composed of a target model Θtarget and a loss prediction module [training dynamics prediction model] Θloss. Our objective is to learn the model set for this stage s to obtain {Θs target, Θs loss}. Given a training data point x, we obtain a target prediction through the target model as yˆ = Θtarget(x), and also a predicted loss through the loss prediction module as ˆl = Θloss(h). With the target annotation y of x, the target loss can be computed as l = Ltarget(ˆy, y) to learn the target model. Since this loss l is a ground-truth target of h [target training dynamics information] for the loss prediction module, we can also compute the loss for the loss prediction module as Lloss( ˆl, l). Then, the final loss function to jointly learn both of the target model and the loss prediction module is defined as Ltarget(ˆy, y) + λ · Lloss( ˆl, l) (1)”)
Yoo and Zhang are both related to the same field of endeavor (i.e., active learning). In view of the teachings of Yoo it would have been obvious for a person of ordinary skill in the art to apply the teachings of Yoo to Zhang before the effective filing date of the claim invention in order to improve the efficiency of classification models (Yoo, Abstract, “The performance of deep neural networks improves with more annotated data. The problem is that the budget for annotation is limited. One solution to this is active learning, where a model asks human to annotate data that it perceived as uncertain. A variety of recent methods have been proposed to apply active learning to deep networks but most of them are either designed specific for their target tasks or computationally inefficient for large networks. In this paper, we propose a novel active learning method that is simple but task-agnostic, and works efficiently with the deep networks. We attach a small parametric module, named “loss prediction module,” to a target network, and learn it to predict target losses of unlabeled inputs. Then, this module can suggest data that the target model is likely to produce a wrong prediction.”)
Regarding claim 2:
Zhang, as modified by Yoo, teaches the method of claim 1.
Zhang does not explicitly teach:
wherein training the training dynamics prediction model includes acquiring loss information based on the target training dynamics information and the predictive training dynamics information.
Yoo further teaches:
wherein training the training dynamics prediction model includes acquiring loss information based on the target training dynamics information and the predictive training dynamics information (Section 3.3, paragraph 2, “Given a training data point x, we obtain a target prediction through the target model as yˆ = Θtarget(x), and also a predicted loss [loss information] through the loss prediction module as ˆl = Θloss(h) [predictive training dynamics information]. With the target annotation y of x, the target loss can be computed as l = Ltarget(ˆy, y) to learn the target model . Since this loss l is a ground-truth target of h [target training dynamics information] for the loss prediction module, we can also compute the loss for the loss prediction module as Lloss( ˆl, l). Then, the final loss function to jointly learn both of the target model and the loss prediction module is defined as Ltarget(ˆy, y) + λ · Lloss( ˆl, l) (1) where λ is a scaling constant. This procedure to define the final loss [training dynamics prediction model]”)
The motivation for claim 2 is the same motivation for claim 1.
Regarding claim 4:
Zhang, as modified by Yoo, teaches the method of claim 1.
Zhang further teaches:
wherein the first dataset includes one or more data pre-labeled with a class (Section 3, paragraph 2, “Then we introduce CAL, which pro poses to learn a data map from the seed labeled data L [first dataset] and identifying regions of instances with a binary classifier…To identify these regions, we require: (1) a data map can be learned from limited data, and (2) a classifier to identify informative instances (i.e., wherein seed labeled data is interpreted as the first dataset that is pre-labeled, hence, with a class)”)
The motivation for claim 4 is the same motivation for claim 1.
Regarding claim 5:
Zhang, as modified by Yoo, teaches the method of claim 4.
Zhang further teaches:
further comprising training the classification model based on the classification information and the pre-labeled class on the training data included in the first dataset (Section 3, paragraph 2, “Then we introduce CAL, which pro poses to learn a data map from the seed labeled data L and identifying regions of instances with a binary classifier…To identify these regions, we require: (1) a data map can be learned from limited data, and (2) a classifier to identify informative instances (i.e., wherein seed labeled data is interpreted as the first dataset that is pre-labeled, hence, with a class)”…“We start with a seed set size of 1,000 [first dataset] for AGNews and 500 for TREC, this means after the AL iterations we will have 2,500 labeled instances for AGNews and 2,000 for TREC. Our motivation here is to keep the AL simulation realistic. We assume enough annotation budget initially annotate 500-1,000 samples. Then, in every AL iteration annotate an additional 50 samples, which seems manageable for an annotator. Finally, we run 30 AL iterations to give a good overview of the performance of the acquisition functions over the iterations towards convergence”)
The motivation for claim 5 is the same motivation for claim 1.
Regarding claim 6:
Zhang, as modified by Yoo, teaches the method of claim 4.
Zhang further teaches:
based on the classification information and the pre-labeled class on the training data included in the first dataset (Section 3, paragraph 2, “Then we introduce CAL, which pro poses to learn a data map from the seed labeled data L [first dataset] and identifying regions of instances with a binary classifier…To identify these regions, we require: (1) a data map can be learned from limited data, and (2) a classifier to identify informative instances (i.e., wherein seed labeled data is interpreted as the first dataset that is pre-labeled, hence, with a class)”)
Zhang does not explicitly teach:
wherein training the training dynamics prediction model includes acquiring loss information
Yoo further teaches:
wherein training the training dynamics prediction model (Section 3.3, paragraph 2, “Given a training data point x, we obtain a target prediction through the target model as yˆ = Θtarget(x), and also a predicted loss through the loss prediction module as ˆl = Θloss(h). With the target annotation y of x, the target loss can be computed as l = Ltarget(ˆy, y) to learn the target model . Since this loss l is a ground-truth target of h for the loss prediction module, we can also compute the loss for the loss prediction module as Lloss( ˆl, l). Then, the final loss function to jointly learn both of the target model and the loss prediction module is defined as Ltarget(ˆy, y) + λ · Lloss( ˆl, l) (1) where λ is a scaling constant. This procedure to define the final loss [training dynamics prediction model]”)
includes acquiring loss information (Section 3.3, paragraph 2, “Given a training data point x, we obtain a target prediction through the target model as yˆ = Θtarget(x), and also a predicted loss [acquiring loss information] through the loss prediction module as ˆl = Θloss(h))
The motivation for claim 6 is the same motivation for claim 1.
Regarding claim 7 and analogous claim 17:
Zhang, as modified by Yoo, teaches the method of claim 6.
Zhang further teaches:
includes acquiring a cross-entropy loss value based on the classification information and the pre- labeled class (Section 4.3, paragraph 3, “This model is suited for the binary classification task of DAL and CAL. In this case it is a single demb = 300 ReLu layer. We minimize the weighted cross-entropy as well. We use the Adam optimizer with the same parameters as above (i.e., wherein under the broadest reasonable interpretation (BRI) classification model use pre-labeled class data)”)
Zhang does not explicitly teach:
wherein acquiring the loss information
Yoo further teaches:
wherein acquiring the loss information (Section 3.3, paragraph 2, “Given a training data point x, we obtain a target prediction through the target model as yˆ = Θtarget(x), and also a predicted loss [acquiring loss information] through the loss prediction module as ˆl = Θloss(h))
The motivation for claim 7 is the same motivation for claim 1.
Regarding claim 12:
Zhang teaches the method of claim 11.
Zhang further teaches:
wherein the first operation includes: acquiring, based on the classification model, target training dynamics information corresponding to training data included in the first dataset; (Section 3, “to use model-independent measures, from fitting the model on the seed data L [acquiring, based on the classification information] (i.e., wherein seed data L is interpreted as the first dataset), by using data maps Data maps help identify [acquiring] characteristics of instances within the broader trends of a dataset by leveraging their training dynamics (i.e., the behavior of a model during training, such as mean and standard deviation of confidence and correctness [target training dynamics information] with respect to the gold label)")
acquiring, based on the training dynamics prediction model, predictive training dynamics information corresponding to the training data included in the first dataset; and (Algorithm 1, “Algorithm 1: Cartography Active Learning 1 input: Labeled seed set L [first dataset], Unlabeled set U, Total budget K, Number of queries n, Correctness threshold tcor = 0.2; for i = 1, ..., n do 3 Ψ(L), Ψ(U) ← train main classifier θ on L, get representations of L and U; 4 µˆ, σˆ, φˆ ← get data map statistics of L with θ; [training dynamics information] 5 Pθ 0 ← train binary classifier θ 0 on Ψ(L) with yΨ(xˆi) = ( 1, if φˆ i > tcor 0, else [training dynamics prediction model] (i.e., wherein the classifier predicts, hence, ‘prediction model’))
Zhang does not explicitly teach:
acquiring, based on the target training dynamics information and the predictive training dynamics information, loss information.
Yoo teaches:
acquiring, based on the target training dynamics information and the predictive training dynamics information, loss information (Section 3.3, paragraph 2, “Given a training data point x, we obtain a target prediction through the target model as yˆ = Θtarget(x), and also a predicted loss [loss information] through the loss prediction module as ˆl = Θloss(h) [predictive training dynamics information]. With the target annotation y of x, the target loss can be computed as l = Ltarget(ˆy, y) to learn the target model . Since this loss l is a ground-truth target of h [target training dynamics information] for the loss prediction module, we can also compute the loss for the loss prediction module as Lloss( ˆl, l). Then, the final loss function to jointly learn both of the target model and the loss prediction module is defined as Ltarget(ˆy, y) + λ · Lloss( ˆl, l) (1) where λ is a scaling constant”)
Yoo and Zhang are both related to the same field of endeavor (i.e., active learning). In view of the teachings of Yoo it would have been obvious for a person of ordinary skill in the art to apply the teachings of Yoo to Zhang before the effective filing date of the claim invention in order to improve the efficiency of classification models (Yoo, Abstract, “The performance of deep neural networks improves with more annotated data. The problem is that the budget for annotation is limited. One solution to this is active learning, where a model asks human to annotate data that it perceived as uncertain. A variety of recent methods have been proposed to apply active learning to deep networks but most of them are either designed specific for their target tasks or computationally inefficient for large networks. In this paper, we propose a novel active learning method that is simple but task-agnostic, and works efficiently with the deep networks. We attach a small parametric module, named “loss prediction module,” to a target network, and learn it to predict target losses of unlabeled inputs. Then, this module can suggest data that the target model is likely to produce a wrong prediction.”)
Regarding claim 16:
Zhang teaches the method of claim 11.
Zhang further teaches:
wherein the first operation includes: acquiring, based on the classification model, classification information on training data included in the first dataset; and (Section 3, paragraph 9, “To select presumably informative instances, we train a binary classifier [classification model] on the seed set L and apply it to U [first dataset] to select the instances that are the closest to the decision boundary between ambiguous and hard-to-learn instances (i.e., wherein classification information is interpreted as the measures of distance to the decision)”)
acquiring, based on the classification information and the pre-labeled class on the training data included in the first dataset, (Section 3, paragraph 2, “Then we introduce CAL, which pro poses to learn a data map from the seed labeled data L and identifying regions of instances with a binary classifier…To identify these regions, we require: (1) a data map can be learned from limited data, and (2) a classifier to identify informative instances (i.e., wherein seed labeled data is interpreted as the first dataset that is pre-labeled, hence, with a class)”…“We start with a seed set size of 1,000 [first dataset] for AGNews and 500 for TREC, this means after the AL iterations we will have 2,500 labeled instances for AGNews and 2,000 for TREC. Our motivation here is to keep the AL simulation realistic. We assume enough annotation budget initially annotate 500-1,000 samples. Then, in every AL iteration annotate an additional 50 samples, which seems manageable for an annotator. Finally, we run 30 AL iterations to give a good overview of the performance of the acquisition functions over the iterations towards convergence”)
Zhang does not explicitly teach:
loss information, and wherein the classification information includes a result of calculating, for each of a plurality of classes, probability of data belonging to that class
Yoo teaches:
loss information, and wherein the classification information includes a result of calculating, for each of a plurality of classes, probability of data belonging to that class (Section 3.3, paragraph 2, “Given a training data point x, we obtain a target prediction through the target model as yˆ = Θtarget(x), and also a predicted loss [loss information] through the loss prediction module as ˆl = Θloss(h))… Section 2, “a naive way to define uncertainty is to use the posterior probability of a predicted class [26, 25], or the margin between posterior probabilities of a predicted class and the secondly predicted class [19, 43]. The entropy [47, 31, 19] of class posterior probabilities generalizes the former definitions. For SVMs, distances [52, 53, 27] to the decision boundaries can be used to define uncertainty. Another approach is the query-by-committee [49, 34, 18]. This method constructs a committee comprising multiple independent models, and measures disagreement among them to define uncertainty.”)
The motivation for claim 16 is the same motivation for claim 12.
Claim(s) 3 and 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang, as modified by Yoo, further in view of Wu et al., Non-Patent Literature “Learning Kullback-Leibler Divergence-Based Gaussian Model for Multivariate Time Series Classification”
Regarding claim 3:
Zhang, as modified by Yoo, teaches the method of claim 2.
Zhang does not explicitly teach:
wherein acquiring the loss information includes acquiring a Kullback-Leibler divergence value based on the target training dynamics information and the predictive training dynamics information
Yoo further teaches:
wherein acquiring the loss information includes ((Section 3.3, paragraph 2, “Given a training data point x, we obtain a target prediction through the target model as yˆ = Θtarget(x), and also a predicted loss [acquiring loss information] through the loss prediction module as ˆl = Θloss(h))
Wu teaches:
acquiring a Kullback-Leibler divergence value based on the target training dynamics information and the predictive training dynamics information (Section III, Part A, paragraph 2, “Assuming that the statistical models P1 and P2 represent two N-dimensional probability distribution functions, respectively, the Kullback-Leibler divergence between those two models is defined as Equations 1 and 2 in the case of discrete and continuous random variables, respectively: KL (P1||P2) = X x∈X P1 (x)log P1 (x) P2 (x) (1) KL (P1||P2) = Z x∈X P1 (x)log P1 (x) P2 (x) dx (2) The physical meaning of the above equations is to calculate the degree of difference between the statistical model and the given statistical model. The Kullback-Leibler divergence is non-negative and asymmetrical (i.e., wherein the target training dynamics information and the predictive training dynamics information is interpreted as P1 and P2 of the equation.)”)
Wu and Zhang are both related to the same field of endeavor (i.e., classification models). In view of the teachings of Wu it would have been obvious for a person of ordinary skill in the art to apply the teachings of Wu to Zhang before the effective filing date of the claim invention in order to improve the efficiency of classification models by measuring the probability distribution during training (Wu, Abstract, “Furthermore, the Kullback-Leibler divergence is used as the similarity measurement to implement the classification of unlabeled subsequences, because it can effectively measure the similarity between different distributions.”)
Regarding claim 13:
Zhang, as modified by Yoo, teaches the method of claim 12.
Zhang further teaches:
wherein training the classification model and the training dynamics prediction model includes (Section 4.3, “this work uses two models for the AL setup solving the classification tasks. Input: Labeled seed set L, Unlabeled set U, Total budget K, Number of queries n, Correctness threshold tcor = 0.2; 2 for i = 1, ..., n do 3 Ψ(L), Ψ(U) ← train main classifier θ on L [training a classification model]… Algorithm 1, “Pθ 0 ← train binary classifier θ 0 on Ψ(L) with yΨ(xˆi) = ( 1, if φˆ i > tcor 0, else [training dynamics prediction model] (i.e., wherein the classifier predicts, hence, ‘prediction model’))”)
Zhang, as modified by Yoo, does not explicitly teach:
acquiring a Kullback-Leibler divergence value based on the target training dynamics information and the predictive training dynamics information
Wu teaches:
acquiring a Kullback-Leibler divergence value based on the target training dynamics information and the predictive training dynamics information (Section III, Part A, paragraph 2, “Assuming that the statistical models P1 and P2 represent two N-dimensional probability distribution functions, respectively, the Kullback-Leibler divergence between those two models is defined as Equations 1 and 2 in the case of discrete and continuous random variables, respectively: KL (P1||P2) = X x∈X P1 (x)log P1 (x) P2 (x) (1) KL (P1||P2) = Z x∈X P1 (x)log P1 (x) P2 (x) dx (2) The physical meaning of the above equations is to calculate the degree of difference between the statistical model and the given statistical model. The Kullback-Leibler divergence is non-negative and asymmetrical (i.e., wherein the target training dynamics information and the predictive training dynamics information is interpreted as P1 and P2 of the equation.)”)
Wu and Zhang are both related to the same field of endeavor (i.e., classification models). In view of the teachings of Wu it would have been obvious for a person of ordinary skill in the art to apply the teachings of Wu to Zhang before the effective filing date of the claim invention in order to improve the efficiency of classification models by measuring the probability distribution during training (Wu, Abstract, “Furthermore, the Kullback-Leibler divergence is used as the similarity measurement to implement the classification of unlabeled subsequences, because it can effectively measure the similarity between different distributions.”)
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang, as modified by Yoo, further in view of Raschka et al., Non-Patent Literature “Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning.”
Regarding claim 8:
Zhang, as modified by Yoo, teaches the method of claim 6.
Zhang does not explicitly teach:
wherein acquiring the loss information includes: determining, based on the classification information, a class to which the training data included in the first dataset is most likely to belong; and checking whether the determined class matches the pre-labeled class
Yoo further teaches:
wherein acquiring the loss information includes (Section 3.3, paragraph 2, “Given a training data point x, we obtain a target prediction through the target model as yˆ = Θtarget(x), and also a predicted loss [acquiring loss information] through the loss prediction module as ˆl = Θloss(h))
Raschka teaches:
determining, based on the classification information, a class to which the training data included in the first dataset is most likely to belong; and checking whether the determined class matches the pre-labeled class (“After the learning algorithm fit a model in the previous step, the next question is: How "good" is the performance of the resulting model? This is where the independent test set comes into play. Since the learning algorithm has not "seen" this test set before, it should provide a relatively unbiased estimate of its performance on new, unseen data. Now, we take this test set and use the model to predict the class labels (i.e., wherein using the previously trained model to make predictions, hence, ‘first dataset used in the trained model’). Then, we take the predicted class labels and compare them to the "ground truth," the correct class labels, to estimate the models generalization accuracy or error (i.e., wherein the predicted