DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 10-14 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Prendki (US 2022/0138561) and Stergioudis (US 20220230095).
Regarding claim 1, Prendki discloses a computer-implemented method for active learning, comprising:
using a model to estimate a utility distribution of a raw data pool based on a raw data subset obtained from the raw data pool; determining a selection criterion based on the utility distribution (“The next step is (Smart) Data Sampling 102 (FIG. 1A, FIG. 2). FIG. 4 illustrates an example of data sampling or sample generation. A training dataset 402, which typically is content trimmed, is processed at 404 to result in creating and storing a plurality of samples 406, 408, 410. This process involves selecting multiple subsets of the data and generating samples that may be used to train separate versions of the model” at paragraph 0063, line 1);
performing a data selection process based on the selection criterion, wherein the data selection process comprises using the model to calculate a utility score associated with a raw image from the raw data pool, using the selection criterion to selectively provide the raw image to an oracle to obtain a selected image corresponding to the raw image, and incorporating the selected image into a training data pool, wherein the utility score associated with the raw image provided to the oracle meets the selection criterion (“The next step is Data Scoring/Ranking 106 (FIG. 1A, FIG. 2). In the next phase, the system now goes through an advanced analysis of the metadata that was generated” at paragraph 0075, line 1; “The effect is to control data valuation in element 106 to generate a score representing the utility of data in predictive performance from a smaller but cleaner data set, thus reinforcing the learned results of metadata generation” at paragraph 0075, last sentence); and
using the training data pool to train a target model (“The next step of the procedure is metamodel training and filter building 110. When deciding on a threshold, the system actually decides a cutoff to separate the data in two sets: “helpful data” (high scores/ranks) and “useless/harmful data” (low scores). By assigning a “high quality” label to the former, and a “low quality” label to the latter, the system actually created a labeled training set to train a binary classifier meant to predict data quality on future training sets” at paragraph 0099).
While Prendki discusses that their invention is able to be used with proxy models (“However, embodiments also are useful with “proxy” models, which solve the same problem or about the same task, to build such filters” at paragraph 0033, line 5), Prendki does not explicitly disclose using a proxy model in the method steps.
Stergioudis teaches a computer-implemented method for active learning, comprising:
using a proxy model (“The generated surrogate model can facilitate or allow for the improvement of active learning. The surrogate model can be a surrogate, proxy or student model of the third-party model” at paragraph 0030, line 3) to estimate a distribution of a raw data pool based on a raw data subset obtained from the raw data pool (“The unlabeled data collector 204 can perform pre-processing on the data prior to storing the data in the unlabeled data structure 218. For example, the unlabeled data collector 204 can identify data samples that satisfy a condition prior to storing the data samples in the unlabeled data structure 218” at paragraph 0074, line 1);
determining a selection criterion (“the surrogate model can provide multiple candidate outputs along with confidence scores for each candidate output” at paragraph 0030, line 15);
performing a data selection process based on the selection criterion, wherein the data selection process comprises using the proxy model to calculate a score associated with a raw image from the raw data pool, using the selection criterion to selectively provide the raw image to an oracle to obtain a selected image corresponding to the raw image, and incorporating the selected image into a training data pool, wherein the score associated with the raw image provided to the oracle meets the selection criterion (“At ACT 406, the surrogate model 222 can make predictions on the unlabeled data set 218. The data processing system can use the surrogate model 222 to make a predictions on the unlabeled data set 218. The prediction can include classes for data samples along with a probability or confidence score of the data sample corresponding to the class. At ACT 408, the data processing system can use a querying strategy to select samples 410 from the unlabeled data set 218. The data processing system can select the samples 410 based on the predictions output from the surrogate model 222 and the querying strategy” at paragraph 0118; “At ACT 422, the data processing system can output the selected samples 410 to an oracle 424. The oracle 424 can refer to a user of the data processing system or other data source that can provide hard targets or ground-truth data for the selected samples. The oracle 424 can facilitate an active learning process. At ACT 426, the data processing system can identify the hard targets 428. For example, the data processing system can receive the hard targets 428 from a user interface or other data source” at paragraph 0120); and
using the training data pool to train a target model (“At ACT 438, the data processing system can provide the labeled data set 436 to the 3P model 238 to cause the 3P system 228 to update the 3P model 238” at paragraph 0122, line 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a proxy model as taught by Stergioudis for the data selection of Prendki to “improve active learning by efficiently selecting data samples from the unlabeled pool of data” (Prendki at paragraph 0030, line 1).
Regarding claim 11, Prendki discloses a computer system for active learning, comprising:
a processing unit (“In particular embodiments, computer system 1000 includes a processor 1002” at paragraph 0127, line 1); and
a storage unit, configured to store a raw data pool, a training data pool, and an active learning program, wherein the active learning program comprises instructions that, when executed by the processing unit (“In particular embodiments, memory 1004 includes main memory for storing instructions for processor 1002 to execute or data for processor 1002 to operate on” at paragraph 0129, line 1), cause the computer system to execute steps including:
using a model to estimate a utility distribution of a raw data pool based on a raw data subset obtained from the raw data pool; determining a selection criterion based on the utility distribution (“The next step is (Smart) Data Sampling 102 (FIG. 1A, FIG. 2). FIG. 4 illustrates an example of data sampling or sample generation. A training dataset 402, which typically is content trimmed, is processed at 404 to result in creating and storing a plurality of samples 406, 408, 410. This process involves selecting multiple subsets of the data and generating samples that may be used to train separate versions of the model” at paragraph 0063, line 1);
performing a data selection process based on the selection criterion, wherein the data selection process comprises using the model to calculate a utility score associated with a raw image from the raw data pool, using the selection criterion to selectively provide the raw image to an oracle to obtain a selected image corresponding to the raw image, and incorporating the selected image into a training data pool, wherein the utility score associated with the raw image provided to the oracle meets the selection criterion (“The next step is Data Scoring/Ranking 106 (FIG. 1A, FIG. 2). In the next phase, the system now goes through an advanced analysis of the metadata that was generated” at paragraph 0075, line 1; “The effect is to control data valuation in element 106 to generate a score representing the utility of data in predictive performance from a smaller but cleaner data set, thus reinforcing the learned results of metadata generation” at paragraph 0075, last sentence); and
using the training data pool to train a target model (“The next step of the procedure is metamodel training and filter building 110. When deciding on a threshold, the system actually decides a cutoff to separate the data in two sets: “helpful data” (high scores/ranks) and “useless/harmful data” (low scores). By assigning a “high quality” label to the former, and a “low quality” label to the latter, the system actually created a labeled training set to train a binary classifier meant to predict data quality on future training sets” at paragraph 0099).
While Prendki discusses that their invention is able to be used with proxy models (“However, embodiments also are useful with “proxy” models, which solve the same problem or about the same task, to build such filters” at paragraph 0033, line 5), Prendki does not explicitly disclose using a proxy model in the method steps.
Stergioudis teaches a computer system for active learning, comprising:
a processing unit (“As shown in FIG. 1A, computer 100 may include one or more processors 105” at paragraph 0035, line 1); and
a storage unit, configured to store a raw data pool, a training data pool, and an active learning program, wherein the active learning program comprises instructions that, when executed by the processing unit (“Non-volatile memory 120 stores operating system 135, one or more applications 140, and data 145 such that, for example, computer instructions of operating system 135 and/or applications 140 are executed by processor(s) 105 out of volatile memory 110” at paragraph 0035, fourth to last sentence), cause the computer system to execute steps including:
using a proxy model (“The generated surrogate model can facilitate or allow for the improvement of active learning. The surrogate model can be a surrogate, proxy or student model of the third-party model” at paragraph 0030, line 3) to estimate a distribution of a raw data pool based on a raw data subset obtained from the raw data pool (“The unlabeled data collector 204 can perform pre-processing on the data prior to storing the data in the unlabeled data structure 218. For example, the unlabeled data collector 204 can identify data samples that satisfy a condition prior to storing the data samples in the unlabeled data structure 218” at paragraph 0074, line 1);
determining a selection criterion (“the surrogate model can provide multiple candidate outputs along with confidence scores for each candidate output” at paragraph 0030, line 15);
performing a data selection process based on the selection criterion, wherein the data selection process comprises using the proxy model to calculate a score associated with a raw image from the raw data pool, using the selection criterion to selectively provide the raw image to an oracle to obtain a selected image corresponding to the raw image, and incorporating the selected image into a training data pool, wherein the score associated with the raw image provided to the oracle meets the selection criterion (“At ACT 406, the surrogate model 222 can make predictions on the unlabeled data set 218. The data processing system can use the surrogate model 222 to make a predictions on the unlabeled data set 218. The prediction can include classes for data samples along with a probability or confidence score of the data sample corresponding to the class. At ACT 408, the data processing system can use a querying strategy to select samples 410 from the unlabeled data set 218. The data processing system can select the samples 410 based on the predictions output from the surrogate model 222 and the querying strategy” at paragraph 0118; “At ACT 422, the data processing system can output the selected samples 410 to an oracle 424. The oracle 424 can refer to a user of the data processing system or other data source that can provide hard targets or ground-truth data for the selected samples. The oracle 424 can facilitate an active learning process. At ACT 426, the data processing system can identify the hard targets 428. For example, the data processing system can receive the hard targets 428 from a user interface or other data source” at paragraph 0120); and
using the training data pool to train a target model (“At ACT 438, the data processing system can provide the labeled data set 436 to the 3P model 238 to cause the 3P system 228 to update the 3P model 238” at paragraph 0122, line 1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize a proxy model as taught by Stergioudis for the data selection of Prendki to “improve active learning by efficiently selecting data samples from the unlabeled pool of data” (Stergioudis at paragraph 0030, line 1).
Regarding claims 2 and 12, the Prendki and Stergioudis combination discloses a method and system wherein the step of using the proxy model to estimate the utility distribution of the raw data pool further comprises inputting each raw image in the raw data subset into the proxy model to calculate the utility score associated with that raw image, and estimating the utility distribution of the raw data pool based on the utility scores associated with the raw images in the raw data subset (“At ACT 406, the surrogate model 222 can make predictions on the unlabeled data set 218. The data processing system can use the surrogate model 222 to make a predictions on the unlabeled data set 218. The prediction can include classes for data samples along with a probability or confidence score of the data sample corresponding to the class. At ACT 408, the data processing system can use a querying strategy to select samples 410 from the unlabeled data set 218. The data processing system can select the samples 410 based on the predictions output from the surrogate model 222 and the querying strategy” Stergioudis at paragraph 0118; “The effect is to control data valuation in element 106 to generate a score representing the utility of data in predictive performance from a smaller but cleaner data set, thus reinforcing the learned results of metadata generation” Predki at paragraph 0075, last sentence); and
wherein the step of determining the selection criterion further comprises deriving a cumulative distribution from the utility distribution, and using the cumulative distribution to determine the selection criterion, wherein the selection criterion is associated with a first specified proportion (“The next step is Threshold/Optimization 108 (FIG. 1A, FIG. 2). Thanks to the previous step, the system now has scored/ordered the training set initially provided by the customer, according to the predictive value of the data. A higher score or ranking means this data contains more “valuable information” for the model to learn from, and (training) data with a lower score has “less” information” Predki at paragraph 0091, line 1; “selecting the samples in a way to maximize diversity (i.e.: the overlap between two samples remains small)” Predki at paragraph 0066, line 2).
Regarding claims 3 and 13, the Prendki and Stergioudis combination discloses a method and system wherein the data selection process comprises a first selection procedure that is performed for at least two iterations based on the selection criterion (“The system may execute on a first, full-size sample in a first iteration, to generate a filter that is used to reduce the size of training sets in subsequent training iterations” Prendki at paragraph 0019, line 4), and wherein each iteration of the first selection procedure comprises:
sampling the raw image from the raw data pool (“The next step is (Smart) Data Sampling 102 (FIG. 1A, FIG. 2). FIG. 4 illustrates an example of data sampling or sample generation. A training dataset 402, which typically is content trimmed, is processed at 404 to result in creating and storing a plurality of samples 406, 408, 410. This process involves selecting multiple subsets of the data and generating samples that may be used to train separate versions of the model” Prendki at paragraph 0063, line 1);
inputting the raw image into the proxy model to calculate the utility score associated with the raw image (“The effect is to control data valuation in element 106 to generate a score representing the utility of data in predictive performance from a smaller but cleaner data set, thus reinforcing the learned results of metadata generation” Prendki at paragraph 0075, last sentence; “At ACT 406, the surrogate model 222 can make predictions on the unlabeled data set 218. The data processing system can use the surrogate model 222 to make a predictions on the unlabeled data set 218. The prediction can include classes for data samples along with a probability or confidence score of the data sample corresponding to the class. At ACT 408, the data processing system can use a querying strategy to select samples 410 from the unlabeled data set 218. The data processing system can select the samples 410 based on the predictions output from the surrogate model 222 and the querying strategy” Stergioudis at paragraph 0118); and
checking if the utility score associated with the raw image meets the selection criterion, and provide the raw image to the oracle to obtain the corresponding selected image and incorporate the selected image into the training data pool if so (“The next step is Threshold/Optimization 108 (FIG. 1A, FIG. 2). Thanks to the previous step, the system now has scored/ordered the training set initially provided by the customer, according to the predictive value of the data. A higher score or ranking means this data contains more “valuable information” for the model to learn from, and (training) data with a lower score has “less” information” Predki at paragraph 0091, line 1; “At ACT 422, the data processing system can output the selected samples 410 to an oracle 424. The oracle 424 can refer to a user of the data processing system or other data source that can provide hard targets or ground-truth data for the selected samples. The oracle 424 can facilitate an active learning process. At ACT 426, the data processing system can identify the hard targets 428. For example, the data processing system can receive the hard targets 428 from a user interface or other data source” Stergioudis at paragraph 0120);
wherein the steps of sampling the raw image, calculating the utility score for the raw image, and checking if the utility score associated with the raw image meets the selection criterion are repeated until number of the selected images incorporated into the training data pool in that iteration of first data selection process reaches a specified budget (“These elements in effect implement a meta-learning approach in which data valuation is integrated into the training process of a predictive model with the results of improving both predictive performance and subsequent iterations of data scoring, ranking, or valuation” Predki at paragraph 0122, last sentence); and
wherein before entering next iteration of the first selection procedure, the proxy model is retrained using the training data pool (“At ACT 440, the data processing system can also use the labeled data set 436 to update the surrogate model 222 using knowledge distillation such that the surrogate model 222 can mimic aspects of the 3P model 238” Stergioudis at paragraph 0122, line 3).
Regarding claims 4 and 14, the Prendki and Stergioudis combination discloses a method and system wherein the data selection process further comprises using the cumulative distribution to determine an exclusion criterion associated with a second specified proportion (“The “threshold” can either reflect the maximum amount of the data that is desired to be used when training future versions of the model, or the limit (value) under which data seems to become useless (flat learning curve) or harmful (decreasing learning curve)” Predki at paragraph 0098), and wherein each iteration of the first selection procedure further comprises:
checking if the utility score associated with the raw image that does not meet the selection criterion meets the exclusion criterion, excluding the raw image if so, and incorporating the raw image into a candidate dataset as a candidate image if not (“Furthermore, threshold optimization can improve resilience to noisy samples by excluding lower-value samples” Predki at paragraph 0097, last sentence; “However, what this disclosure achieves is to make the learning process much faster by injecting the most valuable data first, in order to faster reach the point where the information contained in the remaining of the data is redundant with the rest, or useless (or even harmful to the model).” Predki at paragraph 0091, last sentence; given that the lower value data towards the end of the total possible training set does not improve the model learning compared to a threshold, it is excluded from consideration).
Regarding claims 10 and 20, Predki discloses a method and system wherein the raw data subset is obtained from the raw data pool through simple random sampling (“Step 2 of the process above generally comprises tagging or scoring data as useful. In one implementation, Step 2 of the process above may be implemented using a brute-force approach. In this approach, S samples of size N are randomly sampled from training data (with replacement)” Predki at paragraph 0035, line 1).
Claim(s) 6 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Prendki and Stergioudis as applied to claims 2 and 12 above, and further in view of Xiong et al. (US 20170024642).
The Prendki and Stergioudis combination discloses the elements of claims 2 and 12 as described above.
The Prendki and Stergioudis combination does not explicitly disclose that the proxy model comprises a dropout layer which randomly omits each neuron with a specified dropout probability; and wherein the utility score is calculated based on multiple outputs across multiple forward passes of the proxy model.
Xiong et al. teaches a method and system in the same field of endeavor of neural network training wherein the model comprises a dropout layer which randomly omits each neuron with a specified dropout probability; and wherein the utility score is calculated based on multiple outputs across multiple forward passes of the model (“In one aspect, different neural networks in the plurality of neural networks differ only in that during the forward pass, feature detectors are selectively disabled randomly, pseudorandomly or using a fixed or predetermined pattern, in the fashion of the Dropout procedure, and the selection of feature detectors to be deactivated is not the same in different neural networks. An example of a suitable fixed or predetermined pattern is an irrational number such as Pi. For example, hidden units or input units may be disabled randomly with a predetermined probability (the dropout probability), or pseudo-randomly with the predetermined probability” at paragraph 0021, line 1; “Therefore, in this example, for each training case, each hidden layer feature detector is randomly omitted from the network with a probability of 0.5 and each input layer feature detector is randomly omitted from the network with a probability 0.2, so each hidden or input feature detector cannot rely on other hidden or input feature detectors being present’ at paragraph 0031, second to last sentence).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to utilize the dropout training as taught by Xiong et al. for the proxy model of the Prendki and Stergioudis combination as it “has the effect of creating an exponentially large ensemble of neural networks with different structures, but with shared parameters, which may provide improved computational efficiency at training time and at testing time” (Xiong et al. at paragraph 0018, line 1).
Allowable Subject Matter
Claims 5, 7-9, 15 and 17-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter: the prior art does not teach or disclose based on the ranking of the utility scores associated with the candidate images, selecting a specified number of candidate images with lowest utility scores to provide to the oracle to obtain the corresponding selected images, and incorporating the selected images into the training data pool as required by claims 5 and 15; that the utility score is calculated based on a combination of semantic certainty, spatial certainty, and occurrence certainty associated with the multiple outputs, wherein the semantic certainty represents the proxy model's confidence in predicting a class of an object, the spatial certainty represents the proxy model's confidence in predicting a spatial extent of the object, and the occurrence certainty represents a frequency of the object's occurrence in the multiple forward passes of the proxy model as required by claims 7 and 17.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KATRINA R FUJITA whose telephone number is (571)270-1574. The examiner can normally be reached Monday - Friday 9:30-5:30 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 5712723638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KATRINA R FUJITA/Primary Examiner, Art Unit 2672
Prosecution Insights