Notice of Pre-AIA or AIA Status The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Claim Rejections - 35 USC § 102 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention. Claim(s) 1 -2, 5, 10, 13-15, 18, and 22 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shinvu, Improving misclassification for one class in a multi-class classification task, 2021 With respect to claim 1 , Shinvu teaches 1. A computer implemented method comprising: providing a multi-attribute classifier trained to classify a plurality of attributes ” on p p . 1 -2 (“Here I am trying to use 3 convolution layer neural network to classify a set of images. . . Black rot: 1180 Esca: 1383healthy: 423 leaf blight: 1076”); (Examiner finds 3 layer neural network (NN) teaches the multi attribute classifier; Examiner finds “Black rot, Esca, healthy, and leaf blight” teach the attributes); “ evaluating a performance of the multi-attribute classifier for classifying each attribute of the plurality of attributes ” on p. 1 (“I have one class which is misclassified and I cannot understand what to do next. I have tried to reduce the value of dropout layer, but results got worst. [sic]”); (Examiner finds this passage teaches an evaluation that all but one of the attributes are misclassified); “ determining that the performance of the multi-attribute classifier for at least a particular attribute of the plurality of attributes falls below a defined standard ” on p. 1 (“I have one class which is misclassified and I cannot understand what to do next. I have tried to reduce the value of dropout layer, but results got worst. [sic]”);(Examiner finds “misclassified” teaches falling below a standard; Examiner finds the particular attribute is the one class (i.e. black rot)); p. 2 heat map (Examiner finds heat map /confusion matrix indicates black rot falls below threshold (i.e. because the shading is lighter)); “ and responsive to determining that the performance of the multi-attribute classifier for at least the particular attribute of the plurality of attributes falls below the defined standard, causing training and generating of a single attribute classifier for classifying the particular attribute ” on p. 3 (“Split into two networks; the first differentiates between Leaf Blight-Healthy-Black Rot/Esca, and the second differentiates between Black Rot-Esca”); (Examiner finds “the second differentiates between Black Rot-Esca” teaches training and generating the “single attribute classifier”); “ wherein the single attribute classifier is subsequently used in combination with the multi-attribute classifier for classifying the particular attribute of the plurality of attributes ” on p. 3 (“Split into two networks; the first differentiates between Leaf Blight-Healthy-Black Rot/Esca, and the second differentiates between Black Rot-Esca”); (Examiner finds “the second differentiates between Black Rot-Esca” teaches training and generating the “single attribute classifier”); (Examiner finds “split into two networks” teaches using each network in combination). Claim 14 and claim 22 are rejected for the same reason as claim 1 above. With respect to claim 2 , Shinvu teaches “ 2. The method of claim 1 wherein the defined standard is a threshold associated with training criteria for the multi-attribute classifier and evaluating the performance includes: applying the multi-attribute classifier to a test data set to determine a performance score for each said attribute ” on p. 1 Here I am trying to use 3 convolution layer neural network to classify a set of images (train data: (3249) , validation data: (487), test data: (326)) I have one class which is misclassified and I cannot understand what to do next. I have tried to reduce the value of dropout layer, but results got worst. p. 2, heat map /confusion matrix (Examiner finds the heat map at top of page 2 teaches a performance score for each attribute): “ and comparing the performance score for at least the particular attribute to the threshold for the multi-attribute classifier in classifying attributes ” on p. 1 Here I am trying to use 3 convolution layer neural network to classify a set of images (train data: (3249) , validation data: (487), test data: (326)) I have one class which is misclassified and I cannot understand what to do next. I have tried to reduce the value of dropout layer, but results got worst. p. 2, heat map (Examiner finds predicted black rot falls below threshold of other attributes based on shading, for example); (Examiner finds the shading teaches a comparison of performance between the attributes). Claim 1 5 is rejected for the same reason as claim 2 above. With respect to claim 5, Shinvu teaches “ 5. The method of claim 3, wherein the new data set is a subset of a larger data set, and the original training data set was generated by filtering that larger data set based on a criterion ” on p. 2 I had split the two datasets as follow[s]: (Examiner finds the original training data set (x_train, y_train) is filtered from the larger, initial data set by definition; that is, training data, test data, and validation data are all subsets of a larger initial data set by definition; new data set is any one of validation or test sets (x_valid, x_test, etc. ). Claim 18 rejected for this same reason. With respect to claim 10 Shinvu teaches “ 10. The method of claim 1, the method further comprising: providing a system comprising the multi-attribute classifier and the single attribute classifier; and in response to the particular attribute having the performance below the defined standard for the multi-attribute classifier, the system configured for applying the single attribute classifier instead of the multi-attribute classifier for classifying the particular attribute ” on p. 3 (emphasis added): Split into two networks; the first differentiates between L eaf Blight-Healthy-Black Rot/Esca, and the second differentiates between Black Rot-Esca. (In order to determine the classification of “Black Rot or Esca” the single NN classifier is used) . With respect to claim 13 , Shinvu teaches “ 13. The method of claim 1, wherein the multi-attribute classifier is a multi-class multi- output classifier wherein each said attribute of the plurality of attributes input to the multi- attribute classifier is associated with a separate classification output. “ on p. 3 Split into two networks; the first differentiates between Leaf Blight-Healthy-Black Rot/Esca, and the second differentiates between Black Rot-Esca. (multi-output is (1)Leaf Blight, (2) Healthy, or (3) Black Rot/Esca; (1),(2),and (3) are attributes of the NN multi classifier). Claim Rejections - 35 USC § 103 The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. Claim(s) 3 , 7, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shinvu as applied to claim 1 and claim 14 above and further in view of Fernandez-Navarro, A dynamic over-sampling procedure based on sensitivity for multi-class problems, 2011. With respect to claim 3 , it appears Shinvu fails to explicitly teach “ 3. The method of claim 1, wherein the single attribute classifier is caused to be trained with a new data set different from an original training data set for the multi-attribute classifier ” However, Fernandez-Navarro , A dynamic over-sampling procedure based on sensitivity for multi-class problems , teaches “ 3. The method of claim 1, wherein the single attribute classifier is caused to be trained with a new data set different from an original training data set for the multi-attribute classifier ” in the abstract (Examiner finds the new, different data set is the dataset that results from the oversampled minority (minimum sized) class; Examiner finds this minority class teaches the single attribute; Examiner further finds that the minority class in Fernandez-Navarro is at least one); p. 1826 left column first full paragraph (“. . .selects the minimum size class. . . “) Fernandez-Navarro , and Shinvu are analogous art because they are from the same field of endeavor as the claimed invention. It would have been obvious to one skilled in the art before the effective filing date of the invention to modify t he single attribute classifier in Shinvu “ to be trained with a new data set different from an original training data set for the multi-attribute classifier ” as suggested by Fernandez-Navarro , The motivation would have been to increase the accuracy of the single classifier. See Fernandez-Navarro , abstract. Claim 1 6 is rejected for the same reason as claim 3 above. With respect to claim 7 , it appears Shinvu fails to explicitly teaches“ 7. T he method of claim 3, further comprising: determining that the particular attribute is related to a particular category of object and causing the new data set to contain samples related to the particular attribute for categories of available objects other than the particular category of object in addition to samples related to the particular category of object. ” However, Fernandez-Navarro teaches “ determining that the particular attribute is related to a particular category of object and causing the new data set to contain samples related to the particular attribute for categories of available objects other than the particular category of object in addition to samples related to the particular category of object ” in the abstract (new minority data set has both attributes of available object and “other” objects). Shinvu and Fernandez-Navarro are analogous art because they are from the same field of endeavor as the claimed invention. It would have been obvious to one skilled in the art before the effective filing date of the invention to modify the particular attribute in Shinvu et al. to include “ determining that the particular attribute is related to a particular category of object and causing the new data set to contain samples related to the particular attribute for categories of available objects other than the particular category of object in addition to samples related to the particular category of object ” as taught by Fernandez-Navarro . The motivation would have been to increase the accuracy of the single classifier. See Fernandez-Navarro , abstract. Claim 20 is rejected for this same reason. Claim(s) 8, 9, 11, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shinvu as applied to claim 1 and claim 14 above and further in view of Rodriguez US 20230237369 A1. With respect to claim 8 , it appears Shinvu fails to explicitly teach “ The method of claim 1 wherein the defined standard is a precision constraint value .” However, Rodriguez US 20230237369 A1 teaches “ The method of claim 1 wherein the defined standard is a precision constraint value ” in para. 52 (“ Indeed, during their experiments (e.g., using a “testing” dataset and a separate “production” dataset), the present inventors implemented an embodiment described herein: after initial training, the machine learning classifier exhibited a recall rate of 32%, a precision rate of 29% , and an area-under-curve score of 0.78; however, after deletion of weakly-predictive feature categories and retraining, the machine learning classifier exhibited a recall rate of 96%, a precision rate of 49 %, and an area-under-curve score of 0.99 ”). Shinvu and Rodriguez are analogous art because they are from the same field of endeavor as the claimed invention. It would have been obvious to one skilled in the art before the effective filing date of the invention to modify the defined standard in Shinvu to include “w herein the defined standard is a precision constraint value . The motivation would have been to increase the accuracy of the model. See Rodriguez para. 52. Claim 21 is rejected for the same reason as claim 8 above. With respect to claim 9, it appears Shinvu fails to explicitly teach “ 9. The method of claim 1 wherein the defined standard includes a precision constraint value and a recall value; and wherein determining that the performance for the particular attribute falls below the defined standard includes: determining that the performance for the particular attribute meets the precision constraint value; and responsive to determining that the performance for the particular attribute meets the precision constraint value. However, Rodriguez teaches “ wherein the defined standard includes a precision constraint value and a recall value ” in para. 52 the present inventors implemented an embodiment described herein: after initial training, the machine learning classifier exhibited a recall rate of 32%, a precision rate of 29% , and an area-under-curve score of 0.78; however, after deletion of weakly-predictive feature categories and retraining, the machine learning classifier exhibited a recall rate of 96%, a precision rate of 49% , and an area-under-curve score of 0.99. Clearly, various embodiments described herein constitute concrete and tangible technical improvements in the field of machine learning classification, and thus such embodiments certainly qualify as useful and practical applications of computers. (Examiner finds the recall value is anything over 32% and the precision constraint value is anything above 29%). “ and wherein determining that the performance for the particular attribute falls below the defined standard includes: determining that the performance for the particular attribute meets the precision constraint value ” the present inventors implemented an embodiment described herein: after initial training, the machine learning classifier exhibited a recall rate of 32%, a precision rate of 29% , and an area-under-curve score of 0.78; however, after deletion of weakly-predictive feature categories and retraining, the machine learning classifier exhibited a recall rate of 96%, a precision rate of 49% , and an area-under-curve score of 0.99. Clearly, various embodiments described herein constitute concrete and tangible technical improvements in the field of machine learning classification, and thus such embodiments certainly qualify as useful and practical applications of computers. “ and responsive to determining that the performance for the particular attribute meets the precision constraint value. ” the present inventors implemented an embodiment described herein: after initial training, the machine learning classifier exhibited a recall rate of 32%, a precision rate of 29% , and an area-under-curve score of 0.78; however, after deletion of weakly-predictive feature categories and retraining, the machine learning classifier exhibited a recall rate of 96%, a precision rate of 49% , and an area-under-curve score of 0.99. Clearly, various embodiments described herein constitute concrete and tangible technical improvements in the field of machine learning classification, and thus such embodiments certainly qualify as useful and practical applications of computers. (Examiner finds the precision constraint value is anything above 29%). Rodriguez and Shinvu are analogous art because they are from the same field of endeavor as the claimed invention. It would have been obvious to one skilled in the art before the effective filing date of the invention to modify the defined standard in Shinvu to include “ wherein the defined standard includes a precision constraint value and a recall value; and wherein determining that the performance for the particular attribute falls below the defined standard includes: determining that the performance for the particular attribute meets the precision constraint value; and responsive to determining that the performance for the particular attribute meets the precision constraint value ” as taught by Rodriguez . The motivation would have been to increase the accuracy and improve the performance of the classifier. See Rodriguez para. 52. With respect to claim 11 , it appears Shinvu fails to explicitly te4ach “ 11. The method of claim 2, wherein training of the single attribute classifier additionally comprises evaluating performance for the single attribute classifier to determine whether meets the defined standard and responsive to evaluating that the performance fails to meet the defined standard causing re-training of the single attribute classifier. “ However, Rodriguez teaches “ 11. The method of claim 2, wherein training of the single attribute classifier additionally comprises evaluating performance for the single attribute classifier to determine whether meets the defined standard and responsive to evaluating that the performance fails to meet the defined standard causing re-training of the single attribute classifier ” in para. 52: [0052] Moreover, various embodiments of the subject innovation can integrate into a practical application various teachings described herein relating to automated training of machine learning classification for patient missed care opportunities or late arrivals. As explained above, existing techniques can cause machine learning classifiers to achieve quite low performance metrics. The present inventors realized that such low performance metrics are often caused because existing techniques train machine learning classifiers on irrelevant and/or weakly-predictive features (e.g., such machine learning classifiers can become distracted by irrelevant and/or weakly-predictive features). Accordingly, the present inventors devised various embodiments described herein. Specifically, various embodiments described herein can train a machine learning classifier on a set of annotated data candidates. Furthermore, after such training, various embodiments described herein can analyze (e.g., analytically and/or via artificial intelligence) the trained, updated, and/or optimized internal parameters of the machine learning classifier, so as to rank the feature categories that define the set of annotated data candidates in order of classification importance. In various cases, various embodiments described herein can delete from the set of annotated data candidates any feature categories whose ranks (e.g., whose classification importance scores) fail to satisfy any suitable threshold value . Accordingly, various embodiments described herein can retrain the machine learning classifier on the set of annotated data candidates after such deletion, which can cause the machine learning classifier to achieve significantly improved performance (e.g., the weakly-predictive features can be no longer present to distract and/or bog down the machine learning classifier). Indeed, during their experiments (e.g., using a “testing” dataset and a separate “production” dataset), the present inventors implemented an embodiment described herein: after initial training, the machine learning classifier exhibited a recall rate of 32%, a precision rate of 29%, and an area-under-curve score of 0.78; however, after deletion of weakly-predictive feature categories and retraining, the machine learning classifier exhibited a recall rate of 96%, a precision rate of 49%, and an area-under-curve score of 0.99. Clearly, various embodiments described herein constitute concrete and tangible technical improvements in the field of machine learning classification, and thus such embodiments certainly qualify as useful and practical applications of computers. Rodriguez and Shinvu are analogous art because they are from the same field of endeavor as the claimed invention. It would have been obvious to one skilled in the art before the effective filing date of the invention to modify the “ training of the single attribute classifier ” Shinvu to include “ wherein training of the single attribute classifier additionally comprises evaluating performance for the single attribute classifier to determine whether meets the defined standard and responsive to evaluating that the performance fails to meet the defined standard causing re-training of the single attribute classifier ”” as taught by Rodriguez . The motivation would have been to increase the accuracy and improve the performance of the classifier. See Rodriguez para. 52. Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shinvu . With respect to claim 12 , Shinvu fails to explicitly teach “ 12. The method of claim 1, further comprising: tracking performance of the multi-attribute classifier together with one or more single attribute classifiers to determine whether to generate additional single attribute classifiers based on the tracking, wherein each single attribute classifier is used for classifying a respective associated attribute instead of the multi-attribute classifier. However, Examiner finds “ tracking performance of the multi-attribute classifier together with one or more single attribute classifiers to determine whether to generate additional single attribute classifiers based on the tracking, wherein each single attribute classifier is used for classifying a respective associated attribute instead of the multi-attribute classifier” would have been obvious to one skilled in the art based on KSR rationale E. See MPEP 2143(I)(E). Examiner finds (1) before the effective filing date of the invention, there was a r ecognized design need to solve a problem . That is, there was a need to solve the problem of a multiclass classifier failing to classify a particular attribute. See Shinvu p. 1. Examiner further finds (2) before the effective filing date of the invention there were a finite number of identified, predictable potential solutions to the recognized need or problem . Examiner finds that the predicable potential solutions were that each of the classifiers in a neural network could fall below a particular threshold based on a number of factors including data drift. Examiner finds the finite number of solutions were to improve any classifier that fell below a certain threshold by retraining the model. See generally Shinvu pages 1-2. Examiner further finds (3) one of ordinary skill in the art could have pursued the known potential solutions with a reasonable expectation of success . That is, one skilled in the art would have pursued “ tracking performance of the multi-attribute classifier together with one or more single attribute classifiers to determine whether to generate additional single attribute classifiers based on the tracking, wherein each single attribute classifier is used for classifying a respective associated attribute instead of the multi-attribute classifier. ” This would have improved performance of any of the classifications in a neural network when new training data was available to the neural network and any one of the classifiers failed to classify new attribute of the new data. Allowable Subject Matter Claims 4, 6, 17, and 19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Reasons for Indicating Allowable Subject Matter The prior art of record fails to teach or suggest “ wherein the new data set is biased towards containing only samples related to the particular attribute” as recited in claim 4 and claim 17. Examiner finds it would not have been obvious to modify the prior art of record to include “ wherein the new data set is biased towards containing only samples related to the particular attribute .” It would frustrate the purpose of the cited prior art by overfitting the data. See What is Overfitting on pages 2-3 (emphasis added): Why does overfitting occur? You only get accurate predictions if the machine learning model generalizes to all types of data within its domain. Overfitting occurs when the model cannot generalize and fits too closely to the training dataset instead. Overfitting happens due to several reasons, such as: • The training data size is too small and does not contain enough data samples to accurately represent all possible input data values. • The training data contains large amounts of irrelevant information, called noisy data. • The model trains for too long on a single sample set of data. • The model complexity is high, so it learns the noise within the training data. Overfitting examples Consider a use case where a machine learning model has to analyze photos and identify the ones that contain dogs in them. If the machine learning model was trained on a data set that contained majority photos showing dogs outside in parks , it may may learn to use grass as a feature for classification, and may not recognize a dog inside a room. Another overfitting example is a machine learning algorithm that predicts a university student's academic performance and graduation outcome by analyzing several factors like family income, past academic performance, and academic qualifications of parents. However, the test data only includes candidates from a specific gender or ethnic group. In this case, overfitting causes the algorithm's prediction accuracy to drop for candidates with gender or ethnicity outside of the test dataset. Conclusion The following prior art is relevant to Applicant’s specification: US 20210406770 A1 [0049] According to an aspect, the quality metrics are a measure or an estimation of the error of the result. The quality metrics may comprise one of accuracy, precision, recall, F1 score or combinations thereof. These parameters ar e well-known parameters to determine errors in statistics . Precision defines false positive and recall defines false negative errors. Accuracy is a weighted arithmetic mean of precision and inverse precision (weighted by bias) as well as a weighted arithmetic mean of recall and inverse recall. The F1 score is the harmonic mean of the precision and recall, where an F1 score reaches its best value at 1. most of all positive results (average precision recall score) using a region under the curve. US 20210168638 A1 [ 0067] The performance of the model may be evaluated by any well-known evaluation metrics. For binary classification models, f1-score, AuC ROC score, precision and recall metrics are often used. The choice of evaluation metrics may depend on the problem formulation. US 20200372383 A1 [0047] Recall and precision are metrics derived from four basic numbers, i.e., true positive (TP), true negative (TN), false positive (FP) and false negative (FN) in binary classification, as shown in Equation 1 and Equation 2. TP, TN, FP and FN are well known from the concept of confusion matrix. Any inquiry concerning this communication or earlier communications from the examiner should be directed to FILLIN "Examiner name" \* MERGEFORMAT ALBERT M PHILLIPS, III whose telephone number is FILLIN "Phone number" \* MERGEFORMAT (571)270-3256 . The examiner can normally be reached FILLIN "Work Schedule?" \* MERGEFORMAT 10a-6:30pm EST M-F . Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, FILLIN "SPE Name?" \* MERGEFORMAT Ann J Lo can be reached at FILLIN "SPE Phone?" \* MERGEFORMAT (571) 272-9767 . The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /ALBERT M PHILLIPS, III/ Primary Examiner, Art Unit 2159