DETAILED ACTION
This action is in response to the application filed on 05/03/2023. Claims 1-20 are pending in the application and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 5-7; 8-9, 12-14; 15-16, 19-20 are rejected under 35 U.S.C. 102 as being anticipated by Ardila et al. (US 12271443 B1, hereinafter “Ardila”).
Regarding Claim 1,
Ardila discloses A method of retraining a target machine learning model trained by first data samples from first input data, the first data samples being classified as members of a first consistency class, the method comprising: (Ardila [Column 1 Lines 31-43]; “For example, if the data sample set has many more data samples for a first label than a second label, the machine learning model can generate predictions that are disproportionately skewed toward the first label. As another example, if the data sample set includes data samples with features that are highly correlated with and representative of the labels, the machine learning model can be overtrained and cannot generalize to predict labels for outlier data samples. As yet another example, if the data sample set includes data samples that share a feature but are differently and inconsistently labeled, the machine learning model cannot produce labels that accurately reflect the labeling inconsistencies for the feature.” wherein the trained machine learning model trained on data samples disproportionately skewed towards the overrepresented first label reads on a target machine learning model trained by first samples from first input data classified as a first consistency class)
selecting second data samples from second input data based on predictions from a surrogate membership classification model, wherein the second data samples are predicted by the surrogate membership classification model to be members of a second consistency class different from the first consistency class based on feature content of each data sample of the second input data; (Ardila [Column 1 Lines 44-59]; “In such cases, it is often desirable to supplement a data sample set with additional data samples and associated labels. A developer can review a set of unlabeled data samples to select and label some of the data samples for addition to the data sample set … Instead, data samples can be selected automatically from an unlabeled data sample set, such as a random selection from the set of unlabeled data samples, or based on time (e.g., selecting the oldest or newest unlabeled data samples first), for presentation to and labeling by a developer.”
Ardila [Column 10 Lines 25-34]; “Based on the analysis logic 202-1, the first computing engine 200-1 can generate one or more data sampling criteria 122-1 indicating properties of data samples that could be selected from the set of unlabeled data samples 116, which could improve label representation of the machine learning model 114. For example and without limitation, the one or more sampling criteria 122-1 could indicate an unlabeled data samples 116 for which the machine learning model 114 predicts a label 110 that is underrepresented in the data sample set 106.” wherein the data samples of the “underrepresented” distribution label selected from the unlabeled data set based on predicted data sampling objectives/criteria by the surrogate supplemental engine reads on selected second data samples based on predictions from a surrogate membership classification model; wherein the second data samples determined by the predicted data sampling criteria to be of an “underrepresented” class label reads on the second data members of a second consistency class different from the first consistency class
Ardila [Column 8 Lines 26-34]; “Alternatively or additionally, in some embodiments, at least part of the machine learning trainer 112 and/or the supplementing engine 118 is a configuration of a hardware circuit (e.g., configurations of the lookup tables within the logic blocks of one or more FPGAs). In some embodiments, the memory 104 includes additional components (e.g., machine learning libraries used by the machine learning trainer 112 and/or supplementing engine 118)” wherein the supplementing engine configured through machine learning libraries reads on a surrogate membership classification model for prediction of sampling objectives and their associated supplemental data of a second underrepresented label)
and retraining the target machine learning model using the second data samples, based on the selecting operation (Ardila [Column 9 Lines 30-53]; “In some embodiments, the supplementing engine 118 initially determines a sampling objective 120 for the data sample set 106 associated with the machine learning model 114 based on an analysis of the machine learning model. In particular. the supplementing engine 118 can determine the sampling objective 120 based on predictions of the machine learning model 114 for at least one data sample 108 of the data sample set 106. The predictions can include, for example, the labels 110 associated with one or more data samples 108 of the data sample set 106; bounding boxes around identified features in data samples 108; and/or prediction confidence scores of the predictions. In some embodiments, the supplementing engine 118 determines the sampling objective 120 based on a set of class probability distributions of associations between the labels 110 and at least one data sample 108. In various embodiments, the predictions can be generated during the determining of the sampling objective 120; stored during an initial training of the machine learning model 114; or received (e.g., from a user) for a machine learning model 114 to be trained or retrained. Based on the predictions, the supplementing engine 118 can determine that the machine learning model 114 produces predictions for a first label 110 that are disproportionate with the predictions of a second label 110.” wherein the retraining of the initial machine learning model using a supplemented data sample set derived from the supplementing engine’s predicted sampling objectives reads on retraining of the initial model using selected supplemental second data samples)
Regarding Claim 2,
Ardila teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Ardila already discloses wherein the selecting operation comprises: training the surrogate membership classification model using a focused training dataset selected from the first input data and the second input data (Ardila [Column 8 Lines 15-34]; “As a third such example and without limitation, some disclosed embodiments include different implementations of the machine learning trainer 112 and/or the supplementing engine 118. In some embodiments, at least part of the machine learning trainer 112 and/or the supplementing engine 118 is embodied as a program in a high-level programming language (e.g., C, Java, or Python), including a compiled product thereof. Alternatively or additionally, in some embodiments, at least part of the machine learning trainer 112 and/or the supplementing engine 118 is embodied in hardware-level instructions (e.g., a firmware that the processor 102 loads and executes). Alternatively or additionally, in some embodiments, at least part of the machine learning trainer 112 and/or the supplementing engine 118 is a configuration of a hardware circuit (e.g., configurations of the lookup tables within the logic blocks of one or more FPGAs). In some embodiments, the memory 104 includes additional components (e.g., machine learning libraries used by the machine learning trainer 112 and/or supplementing engine 118)” wherein the supplementing engine and machine learning trainer configured through machine learning libraries reads on prediction-based surrogate membership classification model
Ardila [Column 9 Lines 30-58]; “In some embodiments, the supplementing engine 118 initially determines a sampling objective 120 for the data sample set 106 associated with the machine learning model 114 based on an analysis of the machine learning model. In particular. the supplementing engine 118 can determine the sampling objective 120 based on predictions of the machine learning model 114 for at least one data sample 108 of the data sample set 106. The predictions can include, for example, the labels 110 associated with one or more data samples 108 of the data sample set 106; bounding boxes around identified features in data samples 108; and/or prediction confidence scores of the predictions. In some embodiments, the supplementing engine 118 determines the sampling objective 120 based on a set of class probability distributions of associations between the labels 110 and at least one data sample 108. In various embodiments, the predictions can be generated during the determining of the sampling objective 120; stored during an initial training of the machine learning model 114; or received (e.g., from a user) for a machine learning model 114 to be trained or retrained. Based on the predictions, the supplementing engine 118 can determine that the machine learning model 114 produces predictions for a first label 110 that are disproportionate with the predictions of a second label 110. The supplementing engine 118 can determine that the machine learning model 114 produces class probability for which the highest probabilities of various labels 110 are often very close, possibly resulting in inconsistent labeling by the machine learning model 114”
Ardila [Column 9 Line 66]; “The supplementing engine 118 selects a sampling objective from a sampling objective set. More particularly, the supplementing engine 118 includes a set of computing engines 200. For each sampling objective included in the sampling objective set, the supplementing engine 118 generates corresponding data sampling criteria via a different computing engine 200 in the set of computing engines 200. Each computing engine 200 applies an analysis logic 202 to the machine learning model 114 and/or data sample set 106 in order to determine the relevance of the sampling objective 120 of the computing engine 200.” wherein the supplementing engine’s optimization of sampling objectives based on the data sample set comprising first and second labeled information thus reads on a focused training group selected from the entirety of the first and second input data)
Regarding Claim 5,
Ardila teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Ardila already discloses wherein the first input data is input to the target machine learning model before the second input data ([Column 2 Line 64]; “At least one technical advantage of the disclosed techniques is the data sample set is supplemented to include additional data samples that can improve the performance of machine learning models training on the supplemented data sample set, such as (without limitation) improving label balance, improving robustness, and/or improving consistency. Machine learning models that are trained or retrained on the supplemented data sample set can exhibit improved performance such as labeling accuracy, precision, and/or recall. Further, supplementing the data sample set with data samples based on an indicated set of objectives for improving machine learning models can increase the likelihood of successful training while reducing cost and complexity. Finally, allowing a developer to indicate objectives for improved machine learning models, and selecting unlabeled data samples for labeling based on the indicated objectives, can increase the performance of machine learning models trained on the supplemented data sample set even if the developer does not understand the causes of poor performance” wherein the supplemented data sample set (data samples associated with second input data) used as supplemental additional secondary samples to the original first input sample data set thus inherently reads on the first input data being input to the target machine learning model before the second input data)
Regarding Claim 6,
Ardila teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Ardila already discloses wherein the selecting operation comprises: generating, by the surrogate membership classification model, a consistency score for each data sample of the second input data; and identifying the second data samples based on whether the consistency score of each data sample of the second input data satisfies a membership condition (Ardila [Column 5 Line 51]; “As a first example, and without limitation, for a sampling objective 120 of improving label balance, the supplementing engine 118 determines that the data sample set 106 includes the number of data samples 108 associated with a first label 110 is disproportionate compared with the number of data samples 108 associated with a second label 110. For instance, an imbalanced data sample set 106 could include 1,000 data samples 108 associated with a first label 110 and only 50 data samples 108 associated with a second label 110. An imbalanced data sample set 106 could include a wide variety of data samples 108 associated with a first label 110 (e.g., images of 100 different types of cars associated with a “car” label 110), but a limited variety of data samples 108 associated with a second label 110 (e.g., only images of a school bus associated with a “bus” label 110). An imbalanced data sample set 106 could result in a trained machine learning model 114 with acceptable performance metrics for a first label 110 (e.g., an F1 score of 0.9 for a first label 110), and a poor performance metrics for a second label 110 (e.g., an F1 score of 0.4 for a second label 110). In such cases, the data sample set 106 overrepresents the first label 110 and underrepresents the second label 110. For instance, a data sample set of images of vehicles includes many images of cars, but few images of bicycles. In such cases, the supplementing engine 118 selects a data sampling criterion 122 for data samples 108 that could be associated with a second label 110.” wherein the sampling objective dependent on the threshold acceptable F1 score (consistency score for second data samples generated by classification model) performed for data samples of the second limited variety of data samples associated with the second label consistency class reads on a consistency score of each data sample satisfying a membership condition)
Regarding Claim 7,
Ardila teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Ardila already discloses wherein the first consistency class and the second consistency class are mutually exclusive (Ardila [Figure 4];
PNG
media_image1.png
495
702
media_image1.png
Greyscale
wherein the first and second labels associated with first and second consistency classes (overrepresented “Day” label and underrepresented “Night” label) are embodiments of the first consistency class and second consistency class being mutually exclusive labels (as Night and Day are mutually exclusive) thus reading on the first and second consistency classes being discretely mutually exclusive)
Claims 8-9 and 13 recite a system to perform the method of Claims 1-2 and 6. Thus, Claims 8-9 and 13 are rejected for reasons set forth in the rejection of Claims 1-2 and 6.
Regarding Claim 12,
Ardila teaches the method of Claim 8 (and thus the rejection of Claim 8 is incorporated). Ardila already discloses wherein the second data samples are weighted based on a consistency score relative to the first data samples prior to retraining of the target machine learning model (Ardila [Column 3 Line 66]; “The machine learning model 114 can be an artificial neural network including a series of layers of neurons. In various embodiments, the neurons of each layer are at least partly connected to, and receive input from, an input source and/or one or more neurons of a previous layer. Each neuron can multiply each input by a weight; process a sum of the weighted inputs using an activation function; and provide an output of the activation function as the output of the artificial neural network and/or as input to a next layer of the artificial neural network. In various embodiments, the machine learning model 114 can include features such as convolutional filters that are applied by each layer to subsets of the input; memory structures, such as long short-term memory (LSTM) units or gated recurrent units (GRU); one or more encoder and/or decoder layers; or the like … If the associated label 110 and the predicted label 110 do not match, then the machine learning trainer 112 adjusts the internal weights of the neurons of the machine learning model 114. The machine learning trainer 112 repeats this weight adjustment process over the course of training until the output of the machine learning model 114 is sufficiently close to or matches with the label 110 associated with the data sample 108. In various embodiments, in various embodiments, during training, the machine learning trainer 112 monitors a performance metric, such as a loss function, that indicates the correspondence between the associated labels 110 and the predicted labels 110 for the data samples 108 of the data sample set 106. The machine learning trainer 112 trains the machine learning model 114 through one or more epochs until the performance metric indicates that the correspondence of the associated labels 110 and the predicted labels 110 is within an acceptable range of accuracy. The trained machine learning model 114 is capable of making predictions 130 of labels 110 for unlabeled data samples 116 in a manner that is consistent with the associations of the data sample set 106.” wherein weighted inputs of the neural network thus implicitly reads on the second data samples of the second data input being weighted based on their respective label’s distribution relative to the distribution of all other labels (including the first data samples) thus reading on second data samples weighted relative to first data samples in the initial training of the target machine learning model)
Regarding Claim 14,
Ardila teaches the method of Claim 8 (and thus the rejection of Claim 8 is incorporated). Ardila already discloses wherein the retraining data selector is further configured to trigger retraining based on data samples of the second consistency class satisfying a retraining condition (Ardila [Column 5 Line 23]; “The supplementing engine 118 is a program stored in the memory 104 and executed by the processor 102 to facilitate the selection of unlabeled data samples 116 to be labeled and included in the supplemented data sample set 128. In particular, the supplementing engine 118 determines one or more data sampling criteria 122 based on the one or more sampling objectives 120. The one or more data sampling criteria 122 indicate properties of data samples 116 that could supplement the data sample set 106, which could improve the performance of the machine learning model 114 if trained or retrained on the supplemented data sample set 128 if supplemented according to the one or more sampling objectives 120. For example and without limitation, the properties of the data samples 108 could be based on the content of the data samples 116, such as a lightness or brightness level of an image. The properties of the data samples 116 could be based on data features determined by the machine learning model 114, such as a classification, or a feature vector generated by one or more layers of a convolutional neural network (CNN). The properties of the data samples 116 could be based on metadata, such as a date, time of day, or geocoordinate included in image EXIF metadata, or a source of the data samples 116. The properties of the data samples 116 could be based on a comparison of each unlabeled data sample 116 with one or more of the labeled data samples 108, such as a similarity measurement between the unlabeled data sample 116 and the labeled data samples 108 that are associated with a label 110.
(15) As a first example, and without limitation, for a sampling objective 120 of improving label balance, the supplementing engine 118 determines that the data sample set 106 includes the number of data samples 108 associated with a first label 110 is disproportionate compared with the number of data samples 108 associated with a second label 110. For instance, an imbalanced data sample set 106 could include 1,000 data samples 108 associated with a first label 110 and only 50 data samples 108 associated with a second label 110. An imbalanced data sample set 106 could include a wide variety of data samples 108 associated with a first label 110 (e.g., images of 100 different types of cars associated with a “car” label 110), but a limited variety of data samples 108 associated with a second label 110 (e.g., only images of a school bus associated with a “bus” label 110). An imbalanced data sample set 106 could result in a trained machine learning model 114 with acceptable performance metrics for a first label 110 (e.g., an F1 score of 0.9 for a first label 110), and a poor performance metrics for a second label 110 (e.g., an F1 score of 0.4 for a second label 110). In such cases, the data sample set 106 overrepresents the first label 110 and underrepresents the second label 110. For instance, a data sample set of images of vehicles includes many images of cars, but few images of bicycles. In such cases, the supplementing engine 118 selects a data sampling criterion 122 for data samples 108 that could be associated with a second label 110” wherein the data sampling criterion for retraining comprising, in part, an acceptable performance threshold associated with the classification model’s F1 score thus reads on triggered retraining based on data samples of the second consistency class satisfying a retraining condition)
Claims 15-16 and 19-20 recite a computer program product comprising a processor readable storage media and stored program instructions to perform the method of Claims 1-2 and 6-7. Thus, Claims 15-16 and 19-20 are rejected for reasons set forth in the rejection of Claims 1-2 and 6-7.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3-4; 10-11; 17-18 are rejected under 35 U.S.C. 102 as being unpatentable over Ardila et al. (US 12271443 B1, hereinafter “Ardila”) in view of Silveira Junior et al. (US 20230342612 A1, hereinafter “Silveira Junior”).
Regarding Claim 3,
Ardila teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). Ardila fails to explicitly disclose but Silveira Junior discloses wherein the selecting operation comprises: generating a feature importance matrix for the first input data and the second input data relative to the target machine learning model (Silveira Junior [Figure 1];
PNG
media_image2.png
284
399
media_image2.png
Greyscale
Silveira Junior [0017]; “Embodiments of the invention estimate the relative importance of features within a machine learning model (e.g., an artificial neural network (ANN) such as a multi-layer perceptron (MLP)) while accounting for input set distribution P(X) and outcome distribution P(Y|X), where X is the input set and Y is the set of possible outcomes in a space. Embodiments of the invention may also consider changes in these distributions over time.
A header matrix may be an Identity matrix that is prepended to a machine learning model and becomes part of the first layer of the machine learning model. In one example, an identity matrix is a square matrix wherein the main diagonal is comprised only of “1”s and the other elements are only zeros. The number of columns is the same as the number of rows. The header matrix is configured to collect variations (e.g., gradients) related to each feature or variable during a backpropagation phase of training epochs. This allows the relative importance of the features to be inferred or estimated. In one example, the gradients can be accumulated over multiple epochs using an accumulation matrix and the relative features can be inferred from the accumulation matrix.” wherein the input data set and its associated modified identity matrix representative of feature importance reads on first and second data inputs (part of the input data set) associated with a feature importance matrix)
It would have been obvious to modify the method of Ardila retraining a model initially trained from first input data using a surrogate machine learning model selecting samples from second input data to generate a feature importance matrix for the first and second data. One would have been motivated to do so because “Being able to determine how feature importance dynamically varies in time can be useful in the mechanism of machine learning model drift detection/correction, for example. Regardless of which model drift detection is used, once a “drift” is identified, “feature importance” can be used to assess the drift severity and help decision making” (Silveira Junior [0019]).
Regarding Claim 4,
Ardila/Silveira Junior teaches the method of Claim 1 (and thus the rejection of Claim 1 is incorporated). The combination already discloses wherein the selecting operation comprises: training the surrogate membership classification model using a focused training dataset selected from the first input data and the second input data (Ardila [Column 8 Lines 15-34]; “As a third such example and without limitation, some disclosed embodiments include different implementations of the machine learning trainer 112 and/or the supplementing engine 118. In some embodiments, at least part of the machine learning trainer 112 and/or the supplementing engine 118 is embodied as a program in a high-level programming language (e.g., C, Java, or Python), including a compiled product thereof. Alternatively or additionally, in some embodiments, at least part of the machine learning trainer 112 and/or the supplementing engine 118 is embodied in hardware-level instructions (e.g., a firmware that the processor 102 loads and executes). Alternatively or additionally, in some embodiments, at least part of the machine learning trainer 112 and/or the supplementing engine 118 is a configuration of a hardware circuit (e.g., configurations of the lookup tables within the logic blocks of one or more FPGAs). In some embodiments, the memory 104 includes additional components (e.g., machine learning libraries used by the machine learning trainer 112 and/or supplementing engine 118)” wherein the supplementing engine and machine learning trainer configured through machine learning libraries reads on prediction-based surrogate membership classification model
Ardila [Column 9 Lines 30-58]; “In some embodiments, the supplementing engine 118 initially determines a sampling objective 120 for the data sample set 106 associated with the machine learning model 114 based on an analysis of the machine learning model. In particular. the supplementing engine 118 can determine the sampling objective 120 based on predictions of the machine learning model 114 for at least one data sample 108 of the data sample set 106. The predictions can include, for example, the labels 110 associated with one or more data samples 108 of the data sample set 106; bounding boxes around identified features in data samples 108; and/or prediction confidence scores of the predictions. In some embodiments, the supplementing engine 118 determines the sampling objective 120 based on a set of class probability distributions of associations between the labels 110 and at least one data sample 108. In various embodiments, the predictions can be generated during the determining of the sampling objective 120; stored during an initial training of the machine learning model 114; or received (e.g., from a user) for a machine learning model 114 to be trained or retrained. Based on the predictions, the supplementing engine 118 can determine that the machine learning model 114 produces predictions for a first label 110 that are disproportionate with the predictions of a second label 110. The supplementing engine 118 can determine that the machine learning model 114 produces class probability for which the highest probabilities of various labels 110 are often very close, possibly resulting in inconsistent labeling by the machine learning model 114”
Ardila [Column 9 Line 66]; “The supplementing engine 118 selects a sampling objective from a sampling objective set. More particularly, the supplementing engine 118 includes a set of computing engines 200. For each sampling objective included in the sampling objective set, the supplementing engine 118 generates corresponding data sampling criteria via a different computing engine 200 in the set of computing engines 200. Each computing engine 200 applies an analysis logic 202 to the machine learning model 114 and/or data sample set 106 in order to determine the relevance of the sampling objective 120 of the computing engine 200.” wherein the supplementing engine’s optimization of sampling objectives based on the data sample set comprising first and second labeled information thus reads on a focused training group selected from the entirety of the first and second input data))
and the focused training dataset is selected based on the feature importance matrix (Silveira Junior [Figure 1];
PNG
media_image2.png
284
399
media_image2.png
Greyscale
Silveira Junior [0017]; “Embodiments of the invention estimate the relative importance of features within a machine learning model (e.g., an artificial neural network (ANN) such as a multi-layer perceptron (MLP)) while accounting for input set distribution P(X) and outcome distribution P(Y|X), where X is the input set and Y is the set of possible outcomes in a space. Embodiments of the invention may also consider changes in these distributions over time.
A header matrix may be an Identity matrix that is prepended to a machine learning model and becomes part of the first layer of the machine learning model. In one example, an identity matrix is a square matrix wherein the main diagonal is comprised only of “1”s and the other elements are only zeros. The number of columns is the same as the number of rows. The header matrix is configured to collect variations (e.g., gradients) related to each feature or variable during a backpropagation phase of training epochs. This allows the relative importance of the features to be inferred or estimated. In one example, the gradients can be accumulated over multiple epochs using an accumulation matrix and the relative features can be inferred from the accumulation matrix.” wherein the input data set and its associated modified identity matrix representative of feature importance reads on first and second data inputs (part of the input data set) associated with a feature importance matrix
Silveira Junior [0100]; “The method of embodiment 1, 2, 3, 4, 5, 6, 7, and/or 8, further comprising performing feature selection, machine learning introspection, dimensionality reduction, and/or drift detection based on the relative importance of each of the features.” wherein the dimensionality reduction performed on the dataset in response to the determined relative importance of the features thus reads on a focused dataset selected based on the feature importance matrix for use in future training)
Claims 10-11 recite a system to perform the method of Claims 3-4. Thus, Claims 10-11 are rejected for reasons set forth in the rejection of Claims 1-2 and 6.
Claims 17-18 recite a computer program product comprising a processor readable storage media and stored program instructions to perform the method of Claims 3-4. Thus, Claims 17-18 are rejected for reasons set forth in the rejection of Claims 3-4.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
“Model Drift Detection Techniques” (US 12586566 B1) which discloses classification distribution consistency classes associated with model drift.
“Dataset Distillation” [2020] (Wang et al.) which discloses relearning of an initial machine learning model through learning of second samples for initial machine learning model input.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN J KIM whose telephone number is (571) 272-0523.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached on (571) 272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN J KIM/Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights