Detailed Action
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-10 are pending for examination. Claim 1 is independent.
Response to Unity of Invention
Applicant has elected Group 1 including claims 1-10 in the Response to Election/Restriction Filed 05/26/2026
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1
According to the first part of the analysis, in the instant case, claims 1-10 are directed to a method. Thus, each of the claims falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Regarding Claim 1:
2A Prong 1:
A method comprising:
using, (This step for determining probabilities for classes is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation).) and
labeling, (This step for labeling data is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation/judgment).)
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
receiving, by a computer system, an unlabeled data object to be labeled using a classification model that is trained to output a probability distribution across a plurality of classes that are treated by the classification model as mutually exclusive; (This step is directed to receiving information, which is understood to be insignificant extra-solution activity and data gathering. See MPEP 2106.05(g).)
by the computer system (This step is understood as mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f).)
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are insignificant extra solution activity in combination of generic computer functions that are implemented to perform the disclosed abstract idea above.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
receiving, by a computer system, an unlabeled data object to be labeled using a classification model that is trained to output a probability distribution across a plurality of classes that are treated by the classification model as mutually exclusive; (This step is directed to transmitting or receiving information, which is understood to be insignificant extra-solution activity and is well understood, routine and conventional activity of transmitting and receiving data as identified by the court (MPEP2106.05(d)(ll)(i))))
by the computer system (This step is understood as mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f).)
The additional elements as disclosed above in combination of the abstract idea
are not sufficient to amount to significantly more than the judicial exception as they are
well, understood, routine and conventional activity as disclosed in combination
of generic computer functions that are implemented to perform the disclosed abstract idea above.
Regarding Claim 2
2A Prong 1:
wherein using the classification model to determine the set of non-mutually-exclusively probabilities includes determining, for each class of the plurality of classes, a binary distribution, wherein the binary distribution includes a first probability that a given class applies to the unlabeled data object, and a second probability that the given class does not apply to the unlabeled data object. (This step for is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation/judgment).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 3
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
further comprising training the classification model to determine the set of non-mutually-exclusively probabilities by using training signal annealing. (Training a model is understood as mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f).))
Regarding Claim 4
2A Prong 1:
wherein using training signal annealing includes, after a set of training data objects have been analyzed at least once:
identify a lowest probability from a set of probabilities associated with the set of training data objects, wherein the set includes probabilities that are greater than a threshold; (This step for is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation/judgment).) and
determine, based on the lowest probability, whether to omit one or more training data objects of the set from a consistency loss operation. (This step for is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation/judgment).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 5
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
further comprising training the classification model to determine the set of non-mutually-exclusively probabilities by using confidence-based masking. (Training a model is understood as mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f).))
Regarding Claim 6
2A Prong 1:
further comprising wherein using confidence-based masking includes:
analyzing, using the classification model, a set of training data objects to determine a respective probability for each class; (This step for is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation/judgment).)
determining, for each training data object in the set, a respective average probability margin across the plurality of classes; (This step for is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation/judgment).) and
determining, based on the respective average probability margins, whether to omit one or more training data objects of the set from a consistency loss operation. (This step for is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation/judgment).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 7
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
further comprising training the classification model to determine the set of non-mutually-exclusively probabilities by using consistency loss. (Training a model is understood as mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f).))
Regarding Claim 8
2A Prong 1:
wherein using consistency loss includes, after a set of training data objects have been analyzed at least once:
generating a divergence value for respective ones of the plurality of classes that are associated with a set of training data objects; (This step for is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation/judgment).) and
determining a weighted average of the divergence values across all associated ones of the plurality of classes. (This step for is practically performable in the human mind and is understood to be a recitation of a mental process (i.e., evaluation/judgment).)
2A Prong 2 & 2B: The claim does not recite any additional elements.
Regarding Claim 9
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
further comprising training the classification model to determine the set of non-mutually-exclusively probabilities by using confidence scheduling. (Training a model is understood as mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f).))
Regarding Claim 10
2A Prong 1: The claim does not recite any Abstract idea.
2A Prong 2 & 2B:
wherein using confidence scheduling includes:
after a set of training data objects have been analyzed in a first training iteration, performing an additional training iteration on a subset of the set of training data objects, wherein the subset excludes training data objects that result in probabilities below a first threshold of a set of thresholds; and after the set of training data objects have been analyzed in the additional training iteration, performing a subsequent training iteration on a portion of the subset of training data objects, wherein the portion excludes training data objects that result in probabilities below a second threshold of the set of thresholds, wherein the second threshold is higher than the first threshold. (Training a model is understood as mere instructions to implement an abstract idea on a computer - see MPEP 2106.05(f).))
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Tschernezki et al. (US 11,651,602 B1, hereinafter "Tschernezki").
Regarding Claim 1
Tschernezki discloses: A method comprising:
receiving, by a computer system ([Col 7 lines 40-67 and Fig 1] disclose a computing system.), an unlabeled data object to be labeled using a classification model that is trained to output a probability distribution across a plurality of classes that are treated by the classification model as mutually exclusive; ([Col 15 lines 56-67, Col 16 lines 1-40, Col 17 lines 5-10, and Fig 3] describes receiving an image (e.g. unlabeled image data 302) that is input into a trained machine learning model (i.e. classification model) to output a vector with probability distributions over a set of classes. Examiner interprets the five classes described in Col 16 lines 33-37 and Fig 3 as mutually exclusive.)
using, by the computer system, the classification model in a manner that determines a set of non-mutually-exclusive probabilities that respective ones of the plurality of classes apply to the unlabeled data object; ([Col 16 lines 28-45, Col 17 lines 4-31, and Fig 3] describes using the trained machine learning model (i.e. classification) that determines an output vector with different probability distribution for different classes applied to the input image. Examiner interprets the five class probabilities described as non-mutually-exclusive probabilities.) and
labeling, by the computer system using the set of non-mutually-exclusively probabilities, the unlabeled data object. ([Col 7 lines 53-56, Col 16 lines 37-65, Col 17 lines 1-3, Col 18 lines 39-44, Fig 1(150), and Fig 3-4] describes classifying (i.e. labeling) using the probability distribution, the input image.)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tschernezki in view of Alvares-Cherman et al. ("Incorporating label dependency into the binary relevance framework for multi-label classification", hereinafter "Cherman").
Regarding Claim 2
Tschernezki discloses: The method of claim 1, wherein using the classification model to determine the set of non-mutually-exclusively probabilities includes determining, for each class of the plurality of classes, ([Col 16 lines 28-45, Col 17 lines 4-31, and Fig 3] describes using the trained machine learning model (i.e. classification) that determines an output vector with different probability distribution for different classes applied to the input image.)
Tschernezki does not explicitly disclose: determining, for each class of the plurality of classes, a binary distribution, wherein the binary distribution includes a first probability that a given class applies to the unlabeled data object, and a second probability that the given class does not apply to the unlabeled data object.
However, Cherman discloses in the same field of endeavor: determining, for each class of the plurality of classes, a binary distribution, wherein the binary distribution includes a first probability that a given class applies to the unlabeled data object, and a second probability that the given class does not apply to the unlabeled data object. ([Introduction, Section 2.1-2.2, and Fig 2-3] describes a binary classification for each yj, and each example in the training set is labeled as positive if yj is an element of the respective multi-label example, negative otherwise.)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of binary classification disclosed by Cherman into the method of multi-classification using a Model disclosed by Tschernezki to generate a binary distribution. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of binary classification disclosed by Cherman as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to transform a multi-label classification problem into several independent binary classification problems.
Claim(s) 3, 5-7, and 9-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tschernezki in view of Xie et al. ("Unsupervised Data Augmentation for Consistency Training", hereinafter "Xie").
Regarding Claim 3
Tschernezki discloses: The method of claim 1, further comprising training the classification model to determine the set of non-mutually-exclusively probabilities by ([Col 14 lines 5-67 and Fig 1-3] describe training the model.)
Tschernezki does not explicitly disclose: using training signal annealing.
However, Xie discloses in the same field of endeavor: using training signal annealing ([Appendix A.1-A.2 and page 16] disclose Training Signal Annealing.)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of Training Signal Annealing disclosed by Xie into the method of multi-classification using a Model disclosed by Tschernezki to perform training signal annealing. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of Training Signal Annealing disclosed by Xie as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination for being able to train a model and reduces model error rates.
Regarding Claim 5
Tschernezki in view of Xie disclose: The method of claim 1, further comprising training the classification model to determine the set of non-mutually-exclusively probabilities by using confidence-based masking. ([Section 2.4 and Appendix A.1-A.2], Xie describes confidence-based masking that mask out examples that the current model is not confident about.)
Regarding Claim 6
Tschernezki in view of Xie disclose: The method of claim 5, further comprising wherein using confidence-based masking includes: analyzing, using the classification model, a set of training data objects to determine a respective probability for each class; ([Col 16 lines 28-45, Col 17 lines 4-31, and Fig 3] Tschernezki, describes using the trained machine learning model (i.e. classification) that determines an output vector with different probability distribution for different classes applied to the input image.
determining, for each training data object in the set, a respective average probability margin across the plurality of classes; ([Col 13 lines 53 Col 15 lines 35-44 Col 16 lines 41-67, and Fig 3-4], Tschernezki describes a combined output (e.g. averaged score).) and
determining, based on the respective average probability margins, whether to omit one or more training data objects of the set from a consistency loss operation. ([Section 2.4 and Appendix A.1-A.2], Xie describes a consistency loss and excluding training data.)
Regarding Claim 7
Tschernezki in view of Xie disclose: The method of claim 1, further comprising training the classification model to determine the set of non-mutually-exclusively probabilities by using consistency loss. ([Section 2.4 and Appendix A.1-A.2], Xie describes a consistency loss term computed.)
Regarding Claim 9
Tschernezki in view of Xie disclose: The method of claim 1, further comprising training the classification model to determine the set of non-mutually-exclusively probabilities by using confidence scheduling. ([Appendix A.1-A.2 and page 16] , Xie disclose confidence-based masking and scheduling.)
Regarding Claim 10
Tschernezki in view of Xie disclose: The method of claim 9, wherein using confidence scheduling includes: after a set of training data objects have been analyzed in a first training iteration, performing an additional training iteration on a subset of the set of training data objects, wherein the subset excludes training data objects that result in probabilities below a first threshold of a set of thresholds; and after the set of training data objects have been analyzed in the additional training iteration, performing a subsequent training iteration on a portion of the subset of training data objects, wherein the portion excludes training data objects that result in probabilities below a second threshold of the set of thresholds, wherein the second threshold is higher than the first threshold. ([Section 2.4 and Appendix A.1-A.2], Xie describes Training Signal Annealing (TSA) which gradually releases “training signals” of the labeled examples as training progresses. utilizing a labeled example if the model’s confidence on that example is lower than a predefined threshold (i.e. first threshold) which increases according to a schedule (i.e. increased threshold is a second threshold).)
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tschernezki in view of Xie and Marwah et al. (US 20240144075 A1, hereinafter "Marwah").
Regarding Claim 4
Tschernezki in view of Xie disclose: The method of claim 3, wherein using training signal annealing includes, after a set of training data objects have been analyzed at least once:
identify a threshold; and determine, based on the ([Section 2.4 and Appendix A.1-A.2] describes confidence-based masking and further sharpening predictions. The confidence-based masking out examples that the current model is not confident about. Specifically, the consistency loss term is computed only on examples whose highest probability among classification categories is greater than a threshold.)
Tschernezki in view of Xie does not explicitly disclose: identify a lowest probability from a set of probabilities
However, Marwah discloses in the same field of endeavor: identify a lowest probability from a set of probabilities associated with the set of training data objects; and determine, based on the lowest probability, whether to omit one or more training data objects of the set from a consistency loss operation; ([Para 0075, 0061,Claim 15, Fig 4, and Fig 9] describe identifying data points having the lowest label quality measures and further removing them.)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of label quality measuring disclosed by Marwah into the method of Tschernezki in view of Xie to identify a lowest probability. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of label quality measuring disclosed by Marwah as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination to improve the overall quality of the set of data points by removing low quality data points.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Tschernezki in view of Xie and Dewan et al. (US 20110044534 A1, hereinafter "Dewan")..
Regarding Claim 8
Tschernezki in view of Xie disclose: The method of claim 7, wherein using consistency loss includes, after a set of training data objects have been analyzed at least once: generating a divergence value for respective ones of the plurality of classes that are associated with a set of training data objects ([Section 2.2.], Xie describes calculating a divergence metric for classes.); and
Tschernezki in view of Xie does not explicitly disclose: determining a weighted average of the divergence values across all associated ones of the plurality of classes.
However, Dewan discloses in the same field of endeavor: determining a weighted average of the divergence values across all associated ones of the plurality of classes. ([Para 0034, 0038-0043, and Fig 4] describe determining a weighted Kullback-Leibler divergence.)
It would have been obvious to a person of ordinary skill in art before the effective filling date of the invention to implement the function of KL divergence disclosed by Dewan into the method of Tschernezki in view of Xie to determine a weighted average of divergence. The modification would have been obvious because one of the ordinary skills of the art would be motivated to utilize the feature of KL divergence disclosed by Dewan as all the references are in the field of machine learning. A person of ordinary skill of the art would have been motivated to perform the combination to evaluate differences between model predictions.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Merler et al. (US 9928448 B1) describes calculating probability scores for a plurality of classes. Fisher (US 20120076406 A1) also describes a identifying a lowest rank above a threshold..
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TEWODROS E MENGISTU whose telephone number is (571)270-7714. The examiner can normally be reached Mon-Fri 9:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ABDULLAH KAWSAR can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TEWODROS E MENGISTU/Examiner, Art Unit 2127
Prosecution Insights