DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the Application filed on 01/20/2026. Claims 1-20 are pending in the case. This action is Final.
Applicant Response
3. In Applicant’s response dated 01/20/2026, Applicant amended Claims 1-3, 9-12, 17 and 20 and argued against all objections and rejections previously set forth in the Office Action dated 01/20/2026.
Claim Rejections - 35 USC § 101
4. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
5. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed towards an abstract idea, without significantly more.
Step 1
According to the first part of the analysis, in the instant case, claim is directed to a computer implemented method, which is a process and falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Regarding Claim 1, 10 and 17,
At step 2A, prong 1, Does the claim recite a judicial exception?
Claim 1 further recites the steps of:
determining a plurality of observations in a feature space for the plurality of data records, wherein the plurality of observations are associated with data points in the feature space for corresponding ones of the plurality of data records (This step recites mathematical concept of vector and data point transformation which falls into the “Mathematical Concepts” grouping of abstract ideas.)
determining distances between each of the plurality of observations in the feature space based on the data points (This step recites mathematical computation that involve clustering, distances and distribution which is mathematical operation, which falls into the “Mathematical Concepts” grouping of abstract ideas.)
estimating a distribution of the plurality of observations based on the distances (This step recites mathematical computation that involve clustering, distances and distribution which is mathematical operation, which falls into the “Mathematical Concepts” grouping of abstract ideas.)
calculating, for each of the plurality of data records, a diversity score of each of the plurality of data records based on a distribution of each of the plurality of data records in a feature space associated with the ML model (This step recites mathematical computation that involve clustering, distances and distribution which is mathematical operation, which falls into the “Mathematical Concepts” grouping of abstract ideas.)
calculating, for each of the plurality of data records, a model attribution score of each of the plurality of data records associated with outputs of the ML mode (This step recites mathematical computation that involve clustering, distances and distribution which is mathematical operation, which falls into the “Mathematical Concepts” grouping of abstract ideas.)
sampling, based on the diversity scores and the model attribution scores, the plurality of data records from the training data based on the plurality of observations, wherein the sampling includes selecting a portion of the plurality of data records based on the distances between each of corresponding ones of the plurality of observations being greater than a minimum distance between a set of the plurality of observations (This step recites mathematical computation that involve weight selection and scoring which is mathematical operation, which falls into the “Mathematical Concepts” grouping of abstract ideas.)
generating, based on the sampling, a sampled training data set that enables the ML model to be trained (This step involves information organization using ML model is mathematical operation, which falls into the “Mathematical process” grouping of abstract ideas.),
The claim recites mathematical concepts (calculating diversity score and a model attribution score) and data organization (sampling and generating sampled data). Accordingly, the claims recite an abstract idea.
Step 2A prong 2: Does the claim recite additional elements? Do those additional elements, individually and in combination, integrate the judicial exception into a practical application?
Further, the claim does not recite any additional element which could integrate this abstract idea into a practical application, because the additional elements recited of consist of:
“… a non-transitory memory; and one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising (Claim 1) and a processor that executes instructions; a non-transitory computer-readable medium having instructions executable by the processor…” (Claim17) (Generic computer components on which to implement the math abstract idea (see MPEP 2106.05(f));
receiving training data for a machine learning (ML) model comprising a plurality of data records (This step is data gathering and merely receiving training data is insignificant extra solution because the data is collected for mathematical analysis)
The additional elements are recited at a high level of generality and do not amount to significantly more than the abstract idea (MPEP 2106.05(f)). The claim uses a computer to perform a math and does not improve the function of the computer or other technology. Accordingly, the claim does not integrate the abstract idea into practical application.
Thus, the claim is directed towards the abstract idea.
Step 2B: Do the additional elements, considered individually and in combination, amount to significantly more than the judicial exception?
“… a non-transitory memory; and one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising (Claim 1) and a processor that executes instructions; a non-transitory computer-readable medium having instructions executable by the processor…” (Claim17) (Generic computer components on which to implement the math abstract idea (see MPEP 2106.05(f));
receiving training data for a machine learning (ML) model comprising a plurality of data records (This step is data gathering and merely receiving training data is insignificant extra solution because the data is collected for mathematical analysis)
The additional elements, alone and in combination, fail to integrate the abstract idea into a practical application or add “significantly more.” Thus, the claims are not patent eligible. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. Neither can insignificant extra-solution activity. All these additional elements as generically claimed are thus considered well-understood, routine, and conventional. Therefore, these limitations, taken alone or in combination, do not integrate the abstract idea into a practical application or recite significantly more that the abstract idea.
Thus, these independent claims are not patent eligible.
The dependent claims respectively recite a judicial exception in limitations of: “wherein the distance comprise a vector distance between different ones of the plurality observations in the feature space from data features utilized by the ML model with the training data.” (claims 2), wherein the data point comprises a coordinate placement of each of the plurality of data records in the feature space based on feature data associated with the ML model for each of the plurality of data records, and wherein the diversity score is increased when the vector distance between coordinate placements of the different ones of the plurality of observations is increased.”, (claim3), “wherein the coordinate placement is determined based on one of a kernel density, a gaussian mixture, or a clustering algorithm.”, (claims 4), wherein the calculating the model attribution score is based on a level of confidence of an accuracy of each of the outputs associated with each of the plurality of data records by the ML model.”(claim 5), “wherein the operations further comprise: training the ML model using the sampled training data set.”(claims 6), “wherein the operations further comprise: retraining the ML model from a previous ML model configuration using the sampled training data set.”,(claims 7), “determining a first weight to apply to the diversity score and a second weight to apply to the model attribution score; and applying the first weight to the diversity score and the second weight the model attribution score prior to the sampling.”(Claim 8), “herein the sampled training data set enables the ML model to determine one of a policy selection determination, a risk and fraud analysis, or a marketing mode.” (claim 9), and “wherein prior to the accessing, the method further comprises: estimating the distributions of the plurality of data records over the feature space for features associated with the plurality of data records; and determining the diversity scores based on the estimating.” (Claims 11) “wherein the determining distances between the distributions in the feature space.”,(claims 12), “calculating a likelihood of one of the plurality of data records to be observed during training of the ML model based on the distributions, wherein the diversity scores are further based on the calculated likelihood.”(Claim 13), “: iterating the calculating of the likelihood over the plurality of data records using the distributions.”(claim 14), “wherein prior to the accessing, the method further comprises: calculating the confidences in the output of the ML model for the plurality of data records based on certainties that the plurality of data records are correctly classified by the ML model.” (Claims 15). “Wherein the calculating the confidences is performed using a previous iteration of the ML model.” (Claim 16), “wherein the ML model is previously trained using a set of sampled data from at least a portion of the plurality of data records.” (claim 18),
“Wherein the retraining comprises reconfiguring at least one of a weight or a value of one or more nodes of the ML model based on the sampled training data set.”, (claim 19), wherein the diversity scores are based on a distribution in a feature space of the plurality of data records, and wherein and the model attribution scores are based on a confidence of a predictive output for each of the plurality of data records by the ML model.” (Claim20).
These additional limitations (in claims 2-9, 10-16 and 18-20) also constitute concepts performed Mathematical concept or mathematical operation groupings of abstract ideas.
This judicial exception is not integrated into a practical application. Additional elements “computer readable medium comprising: computer program code (in claims 2-9, 10-16 and 18-20) all amount to no more than adding insignificant extra-solution activity/specifications related to data gathering, data input, or data transmittal. These additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The dependent claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of non-transitory computer readable medium comprising: computer program code are again insignificant extra-solution activity steps that cannot provide an inventive concept. All these additional elements as generically claimed are considered well-understood, routine, and conventional.
Therefore, these limitations, taken alone or in combination, do not integrate the abstract idea into a practical application or recite significantly more that the abstract idea. Thus, all the dependent claims are also not patent eligible.
Examiner Comments
8. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 103
9. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
10. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Anand (Pub. No. US 20220351055 B1, Pub. Date 2022-11-03) in view of Calapodescu (Pub. No. US 20160307113 A1, Pub. Date 2016-10-20.)
Anand teaches a system comprising: a non-transitory memory; and one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations (see Anand: Fig.17, [0169], “computer 1702, the computer 1702 including a processing unit 1704, a system memory 1706 and a system bus 1708. The system bus 1708 couples’ system components including, but not limited to, the system memory 1706 to the processing unit 1704.”), comprising:
receiving training data for a machine learning (ML) model comprising a plurality of data records (see Anand: Fig.16, [0152], “act 1602 can include accessing, by a device (e.g., 114) operatively coupled to a processor, a first set of data candidates (e.g., 106) and a second set of data candidates (e.g., 108), wherein a machine learning model (e.g., 104) is trained on the first set of data candidates.”)
determining a plurality of observations in a feature space for the plurality of data records (see Anand: Fig.16, [0154], “act 1606 can include generating, by the device (e.g., 118), a first set of compressed data points (e.g., 502) by applying a dimensionality reduction technique to the first set of latent activations, and generating, by the device (e.g., 118), a second set of compressed data points (e.g., 504) by applying the dimensionality reduction technique to the second set of latent activations.”) wherein the plurality of observations are associated with data points in the feature space for corresponding ones of the plurality of data records (see Anand: Fig.16, [0153], “a first set of latent activations (e.g., 202) generated by the machine learning model based on the first set of data candidates, and obtaining, by the device (e.g., 116), a second set of latent activations (e.g., 204) generated by the machine learning model based on the second set of data candidates”)
determining distances between each of the plurality of observations in the feature space based on the data points (see Anand: Fig.14, [0138], “act 1410 can include, for each compressed training data point that belongs to the selected cluster, computing, by the device (e.g., 120), the Euclidean distance between the compressed training data point and the center of the selected cluster. When this is performed for each compressed training data point that belongs to the selected cluster, this is can result in a set of Euclidean distances that are associated with the selected cluster.”)
estimating a distribution of the plurality of observations based on the distances (see Anand: Fig.14, [0140], “act 1414 can include computing, by the device (e.g., 120), a standard deviation distance value for the selected cluster, which standard deviation distance value can be denoted as σ, based on the set of Euclidean distances associated with the selected cluster. In various aspects, the computer-implemented method 1400 can proceed back to act 1404.”)
calculating, for each of the plurality of data records a diversity score of each of the plurality of data records (see Anand: Fig.16, [0155], “act 1608 can include computing, by the device (e.g., 120), a diversity score (e.g., 702) based on the first set of compressed data points and the second set of compressed data points.”) based on the distribution (see Anand: Fig.14, [0141], “the computer-implemented method 1400 can iterate through acts 1404-1414, until a μ and a σ are computed for each cluster of compressed training data points.”)
Anand does not teach system wherein:
calculating, for each of the plurality of data records, a model attribution score of each of the plurality of data records associated with outputs of the ML model;
sampling, based on the diversity scores and the model attribution scores, the plurality of data records from the training data based on the plurality of observations, wherein the sampling includes selecting a portion of the plurality of data records based on the distances between each of corresponding ones of the plurality of observations being greater than a minimum distance between a set of the plurality of observations; and
generating, based on the sampling, a sampled training data set that enables the ML model to be trained.
However, Calapodescu teaches the system wherein:
calculating, for each of the plurality of data records, a model attribution score of each of the plurality of data records associated with outputs of the ML model (see Calapodescu: Fig.3, [0078], At S300, using the current classifier model 56, the entropy H(d) for all documents remaining in the pool is computed, e.g., by the entropy computation component 36. The entropy H(d) of each document in the pool set relative to the classifier model 56 can be computed.”, examiner notes that entropy H(d) is a model based score associated with outputs of the ML models)
sampling, based on the diversity scores and the model attribution scores, the plurality of data records from the training data based on the plurality of observations (see Calapodescu: Fig.1, [0007], “quantifying to what extent a candidate sample is new with respect to the samples already selected in a batch during its construction. In practice, a hybrid criterion is often used, which aggregates the uncertainty value (or expected added value) with the diversity measure. The MMR (Maximum Marginal Relevance) principle is an example of such a hybrid criterion”), wherein the sampling includes selecting a portion of the plurality of data records based on the distances between each of corresponding ones of the plurality of observations being greater than a minimum distance between a set of the plurality of observations (see Calapodescu: Fig.1, [0062], “In selecting the family of hash functions to be used (e.g., based on a training set of objects), the (d.sub.1,d.sub.2,p.sub.1,p.sub.2)—sensitive criteria a) considers only those objects with a high probability of collision (low distance/high similarity between them) and requires selection of a family of hash functions which provide a high probability that these will be assigned to the same bucket, while the (d.sub.1, d.sub.2, p.sub.1,p.sub.2)—sensitive criteria b) considers only those objects with a low probability of collision (high distance/low similarity between them) and requires a family of hash functions which provide a low probability that these will be assigned to the same bucket. Both criteria are met in the family of hash functions which are selected for use in the method.”); and
generating, based on the sampling, a sampled training data set that enables the ML model to be trained (see Calapodescu: Fig.1, [0106], “The classifier model is then retrained using all (or at least some) of the labeled training objects in the set 70 (S116). Specifically, a classification function is learned which best fits the labels and representations 50 of the objects in the training set. As will be appreciated, rather than using the same representations that are used for generation of the signatures, another type of multidimensional vectorial representation of the objects can be used.”)
Because both Anand and Calapodescu are in the same/similar field of endeavor of selecting machine learning training data, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Anand to include the a system that calculate a model attribution(uncertainty) score and diversity score of the plurality of data records to generate a sampled training data set that enables the ML model to be trained as taught by Calapodescu. One would have been motivated to make such a combination in provide smaller, more efficient training datasets and a lag-free and efficient machine learning model training system. (see Calapodescu [0005])
Regarding Claim 2,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 1. Anand further teaches the system wherein:
the distance comprises a vector distance between different ones of the plurality of observations in the feature space from data features utilized by the ML model with the training data (see Anand: Fig.15, [0145], “ act 1506 can include computing, by the device (e.g., 120), the Euclidean distance between the selected compressed test data point and the nearest cluster of compressed training data points (e.g., the cluster whose center is closest and/or nearest in terms of Euclidean distance to the selected compressed test data point).”)
Regarding Claim 3,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 2. Anand further teaches the system wherein:
the data point comprises a coordinate placement for the corresponding one of the plurality of data records based on feature data associated with the ML model for each of the plurality of data records (see Anand: Fig.9, [0109], “the graph 900 shows a non-limiting example where the set of compressed test data points 504 nicely fit into and/or otherwise conform to the clusters of the set of compressed training data points 502. In other words, the green dots in FIG. 9 are all located very close to the red dot clusters. Accordingly, the diversity score 702 that corresponds to the graph 900 can be small in magnitude. Based on the graph 900, the operator can determine that the machine learning model 104 can be accurately deployed on the set of test data candidates 108, that the machine learning model 104 cannot be trained on the set of test data candidates 108 without overfitting, and/or that automatic annotation can be accurately applied to the set of test data candidates 108.”), and wherein the diversity score is increased when the vector distance between coordinate placements of the different ones of the plurality of data records is increased (see Anand: Fig.15, [0107], “The set of compressed training data points 502 and the set of compressed test data points 504 were then generated as described herein, using t-SNE as the dimensionality reduction technique, and where each compressed data point was a two-element vector. The graphs 900-1100 were then plotted.”)
Regarding Claim 4,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 3. Anand further teaches the system wherein:
the coordinate placement is determined based on one of a kernel density, a gaussian mixture, or a clustering algorithm (see Anand: Fig.11, [0111], “the set of compressed training data points 502 is illustrated in red, and the set of compressed test data points 504 is illustrated in green. As shown, the set of compressed training data points 502 are grouped into five distinct clusters 902-910, which correspond to the five distinct classes (e.g., Tibia, Femoral, Sagittal Relevant, Coronal Relevant, Irrelevant). As can be easily seen, the graph 1100 shows a non-limiting example where the set of compressed test data points 504 do not nicely fit into and/or otherwise conform to the clusters of the set of compressed training data points 502. In other words, the green dots in FIG. 11 are mostly located away from the red dot clusters.”)
Regarding Claim 5,
As shown above, Anand and Erenrich and teaches all the limitations of claim 1. Anand further teaches the system wherein:
the calculating the model attribution score is based on a level of confidence of an accuracy of each of the outputs associated with each of the plurality of data records by the ML model (see Anand: Fig.4B, [0083], “a decoder may reconstruct the sequence of layers from the last hidden state representation. Upon comparison of the reconstructed layer surface data with a layer surface data of the layer being printed, the encoder-decoder based model may identify a deviation between the reconstructed layer surface data and the layer surface data of the layer being printed. Such a deviation is indicated as a predicted anomaly score.”);
Regarding Claim 6,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 1. Anand further teaches the system wherein:
the operations further comprise: training the ML model using the sampled training data set (see Anand: Fig.16, [0152], “act 1602 can include accessing, by a device (e.g., 114) operatively coupled to a processor, a first set of data candidates (e.g., 106) and a second set of data candidates (e.g., 108), wherein a machine learning model (e.g., 104) is trained on the first set of data candidates.”)
Regarding Claim 7,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 1. Anand further teaches the system wherein:
retraining the ML model from a previous ML model configuration using the sampled training data set (see Anand: Fig.12, [0121], “iterate through acts 1206-1214 until every training data candidate has been analyzed (e.g., until a hidden activation map has been inserted into the set of training activation maps for each training data candidate). At this point, the computer-implemented method 1200 can proceed to act 1216.”)
Regarding Claim 8,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 1. Erenrich further teaches the system wherein:
determining a first weight to apply to the diversity score and a second weight to apply to the model attribution score (see Calapodescu: Fig.2, [0008], “where H(d) is the entropy score derived from the current classifier model estimated probabilities P(c|d), the weight β can be learned on a calibration set, and sim(d.sub.i,d.sub.i) can be the cosine distance calculated on a bag-of-words representation of the documents d.sub.i and d.sub.j. At each iteration, all documents in the pool have their MMR score computed and the document with highest score is added to the batch.”); and
applying the first weight to the diversity score and the second weight the model attribution score prior to the sampling (see Calapodescu: Fig.2, [0008], “where H(d) is the entropy score derived from the current classifier model estimated probabilities P(c|d), the weight β can be learned on a calibration set, and sim(d.sub.i,d.sub.i) can be the cosine distance calculated on a bag-of-words representation of the documents d.sub.i and d.sub.j. At each iteration, all documents in the pool have their MMR score computed and the document with highest score is added to the batch.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Anand to include a system that determining and applying prior to sampling a first weight to apply to the diversity score and a second weight to apply to the model attribution as taught by Calapodescu. One would have been motivated to make such a combination in provide smaller, more efficient training datasets and a lag-free and efficient machine learning model training system.
Regarding Claim 9,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 1. Anand further teaches the system wherein:
the sampled training data set enables the ML model to determine one of a policy selection determination, a risk and fraud analysis, or a marketing model (see Calapodescu: Fig.2, [0049], “the trained classifier model 56 may be used, by the classification component 42, to label a new object 60, such as some or all the remaining objects in the pool, or a new object not initially in the pool. At S126, the label is output.).
See motivation to combine Anand and Calapodescu Claim 1 above.
Regarding Claim independent 10,
Claim 10 is directed to a method claim and has similar/same claim limitation as claim 1 and is rejected under same rationale.
Regarding Claim 11,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 10. Anand further teaches the system wherein:
determining the diversity scores based on the estimating (see Anand: Fig.16, [0155], “act 1608 can include computing, by the device (e.g., 120), a diversity score (e.g., 702) based on the first set of compressed data points and the second set of compressed data points.”)
Regarding Claim 12,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 11. Anand further teaches the system wherein:
determining the distances is further based on a vector computation between vector d corresponding to the data points (see Anand: Fig.14, [140], “act 1414 can include computing, by the device (e.g., 120), a standard deviation distance value for the selected cluster, which standard deviation distance value can be denoted as σ, based on the set of Euclidean distances associated with the selected cluster. In various aspects, the computer-implemented method 1400 can proceed back to act 1404.”)
Regarding Claim 13,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 11. Anand further teaches the system wherein:
calculating a likelihood of one of the plurality of data records to be observed during training of the ML model based on the distributions (see Anand: Fig.14, [0137], “act 1408 can include computing, by the device (e.g., 120), the center of the selected cluster. For example, the center of a given cluster of compressed training data points can be equal to the average of all the compressed training data points that belong to that given cluster.”), wherein the diversity scores are further based on the calculated likelihood (see Anand: Fig.16, [act 1608 can include computing, by the device (e.g., 120), a diversity score (e.g., 702) based on the first set of compressed data points and the second set of compressed data points.”)
Regarding Claim 14,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 11. Anand further teaches the system wherein:
iterating the calculating of the likelihood over the plurality of data records using the distributions (see Anand: Fig.14, [0139], “act 1412 can include computing, by the device (e.g., 120), an average distance value for the selected cluster, which average distance value can be denoted as μ, based on the set of Euclidean distances associated with the selected cluster.”)
Regarding Claim 15,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 11. Erenrich further teaches the system wherein:
calculating the confidences in the output of the ML model for the plurality of data records based on certainties that the plurality of data records are correctly classified by the ML model (see Erenrich: Fig.8, [0079], “The P(c|d) vales may be retrieved by the classification component 42. There may be any number of classes c, such as 2, 3 or more, depending on the type of classifier model. For a binary classifier that is uncertain as to which of two classes to assign to an object, the probability for each class may be about 0.5, resulting in an entropy close to 1. Where the classifier is more certain, the entropy will be less than 1.”)
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Anand to include a system that calculating the confidences in the output of the ML model are correctly classified by the ML model as taught by Calapodescu. One would have been motivated to make such a combination in provide smaller, more efficient training datasets and a lag-free and efficient machine learning model training system.
Regarding Claim 16,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 11. Erenrich further teaches the system wherein:
the calculating the confidences is performed using a previous iteration of the ML model (see Anand: Fig.13, [0121], “the computer-implemented method 1200 can iterate through acts 1206-1214 until every training data candidate has been analyzed (e.g., until a hidden activation map has been inserted into the set of training activation maps for each training data candidate). At this point, the computer-implemented method 1200 can proceed to act 1216.”)
Regarding independent Claim 17,
Claim 17 is directed to a non-transitory machine-readable medium claim and has similar/same claim limitation as claim 1 and is rejected under same rationale.
Regarding Claim 18,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 11. Anand further teaches the system wherein:
the ML model is previously trained using a set of sampled data from at least a portion of the plurality of data records (see Anand: Fig.12, [0116], “act 1206 can include determining, by the device (e.g., 116), whether each training data candidate in the set of training data candidates has been analyzed by the device. If not, the computer-implemented method 1200 can proceed to act 1208. If so, the computer-implemented method 1200 can proceed to act 1216.”)
Regarding Claim 19,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 17. Anand further teaches the non-transitory machine-readable medium wherein:
the retraining comprises reconfiguring at least one of a weight or a value of one or more nodes of the ML model based on the sampled training data set (see Anand: Fig.13, [0131], “, the computer-implemented method 1300 can iterate through acts 1306-1314 until every test data candidate has been analyzed (e.g., until a hidden activation map has been inserted into the set of test activation maps for each test data candidate). At this point, the computer-implemented method 1300 can proceed to act 1316.”)
Regarding Claim 20,
As shown above, Anand and Calapodescu and teaches all the limitations of claim 17. Anand and Calapodescu further teaches the non-transitory machine-readable medium wherein:
the model attribution scores are based on a confidence of a predictive output for each of the plurality of data records by the ML model (see Calapodescu: Fig.2, [0078], “using the current classifier model 56, the entropy H(d) for all documents remaining in the pool is computed, e.g., by the entropy computation component 36. The entropy H(d) of each document in the pool set relative to the classifier model 56 can be computed.”)
Because both Anand and Calapodescu are in the same/similar field of endeavor of selecting machine learning training data, accordingly, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to modify the teaching of Anand to include model attribution scores are based on a confidence of a predictive output for each of the plurality of data records by the ML model as taught by Calapodescu. One would have been motivated to make such a combination in provide smaller, more efficient training datasets and a lag-free and efficient machine learning model training system.
Response to Arguments
Claim Rejections - 35 U.S.C. § 101,
Regarding the 35 U.S.C. 101 rejection for being directed non-statutory subject matter has been updated and sustained based on applicant amendments and. Therefore, the 35 U.S.C. 101 rejection has been sustained.
Claim Rejections - 35 U.S.C. § 103,
Applicant’s arguments with respect to claim amendments have been considered but are moot considering the new combination of references being used in the current rejection. The new combination of references was necessitated by Applicant’s claim amendments. Therefore, the claims are rejected under the new combination of references as indicated above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PGPUB
NUMBER:
INVENTOR-INFORMATION:
TITLE / DESCRIPTION
US 20190102692 A1
KWANT; Richard
Title: METHOD, APPARATUS, AND SYSTEM FOR QUANTIFYING A DIVERSITY IN A MACHINE LEARNING TRAINING DATA SET
Description: An approach is provided for quantifying a diversity of a machine learning training data set. The approach involves creating a matrix data structure storing a plurality of feature data records describing the observations in the training data set.
US 20210064624 A1
Carbune; Victor
Title: USING LIVE DATA STREAMS AND/OR SEARCH QUERIES TO DETERMINE INFORMATION ABOUT DEVELOPING EVENTS
Description:
Techniques and a framework are described herein
for gathering information about developing events
from multiple live data streams and pushing new
pieces of information to interested individuals as
those pieces of information are learned.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZELALEM W SHALU whose telephone number is (571)272-3003. The examiner can normally be reached M- F 0800am- 0500pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Zelalem Shalu/Examiner, Art Unit 2145
/CESAR B PAULA/Supervisory Patent Examiner, Art Unit 2145
Prosecution Insights