DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on October 28, is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 5-8 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 5:
Claim 5 recites the limitation "and using the first and second sets of inferences for detecting an overfitting status of the target ML model. " in lines 5-6. In the claim, for the term “the first and second sets of inferences”, a first set of inferences is not present prior to this recitation. As such, there is insufficient antecedent basis for this limitation in the claim.
Claims 6-8 inherit the deficiencies of claim 5 and are therefore also rejected as being indefinite.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 3-8 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process) without significantly more.
Claim 3:
Regarding claim 3, in step 1 of the 101-analysis set forth in MPEP 2106, the claim recites “A method comprising”, and method or process is one of the four statutory categories of invention.
In step 2A prong 1 of the 101-analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a process that, under the broadest reasonable interpretation, covers a mental process but for recitation of generic computer components:
“…identifying an overfitting status of the trained target ML model;” (this is a mental process, a person could mentally evaluate identifying an overfitting status of a trained model, see MPEP § 2106.04(a)(2)(III)),
If claim limitations, under the broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process and mathematical concept grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In step 2A prong 2 of the 101-analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
“A method comprising: obtaining training history of a trained target ML model;” (Obtaining training history of a ML model is considered insignificant extra-solution activity of mere data gathering – see MPEP § 2106.05(g)),
“obtaining validation losses from the obtained training history;” (Obtaining validation losses from obtained training history is considered insignificant extra-solution activity of mere data gathering – see MPEP § 2106.05(g)),
“and using one or more trained classifiers with the obtained validation losses inputting thereto for…” (Using trained classifiers with validation losses and inputting is considered mere instructions to apply an exception using generic computer – see MPEP § 2106.05(f)),
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In step 2B of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements ii and iii recites insignificant extra-solution activity of mere data gathering, which are well understood routines and conventional activity, see receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362, and additional element iv recites mere instructions to apply an exception using generic computer, which is not indicative of significantly more.
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim 4:
Regarding claim 4, it is dependent upon claim 3, and thereby incorporates the limitations of, and corresponding analysis to claim 3. Further, claim 4 recites the following additional elements:
“The method of claim 3 further comprising: interpolating the obtained validation losses.” (this is a mental process, a person can mentally interpolate validation losses, see MPEP § 2106.04(a)(2)(III)),
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 5:
Regarding claim 5, in step 1 of the 101-analysis set forth in MPEP 2106, the claim recites “A method for performing during training of a target ML model,”, and method or process is one of the four statutory categories of invention.
In step 2A prong 1 of the 101-analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a process that, under the broadest reasonable interpretation, covers a mental process but for recitation of generic computer components:
“generating a second set of inferences;” (this is a mental process, a person could mentally evaluate generating a set of inferences, see MPEP § 2106.04(a)(2)(III)),
“and using the first and second sets of inferences for detecting an overfitting status of the target ML model.” (this is a mental process, a person could mentally detect an overfitting status of a model using a first and second set inferences, see MPEP § 2106.04(a)(2)(III)),
If claim limitations, under the broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In step 2A prong 2 of the 101-analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
“A method for performing during training of a target ML model,” (Training of a target ML model is considered mere instructions to apply an exception using generic computer – see MPEP § 2106.05(f)),
“the method comprising: obtaining training history of the target ML model;” (Obtaining training history of a ML model is considered insignificant extra-solution activity of mere data gathering – see MPEP § 2106.05(g)),
“using one or more trained classifiers with at least a portion of the training history inputting thereto for…” (Using trained classifiers with training history and inputting is considered mere instructions to apply an exception using generic computer – see MPEP § 2106.05(f)),
Since the claim as a whole, looking at the additional elements individually and in combination, does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In step 2B of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements iii and v recites mere instructions to apply an exception using generic computer, and additional element iv recites insignificant extra-solution activity of mere data gathering, which are well understood routines and conventional activity, see receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362, which is not indicative of significantly more.
Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Claim 6:
Regarding claim 6, it is dependent upon claim 5, and thereby incorporates the limitations of, and corresponding analysis to claim 5. Further, claim 6 recites the following additional elements:
“The method of claim 5 further comprising: obtaining the at least portion of the training history using a rolling window.” (In step 2A, prong 2, this is considered insignificant extra-solution activity of mere data gathering – see MPEP § 2106.05(g)). (In step 2B, this is also considered insignificant extra-solution activity of mere data gathering, which are well understood routines and conventional activity, see receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 – see MPEP § 2106.05(g)).
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 7:
Regarding claim 7, it is dependent upon claim 5, and thereby incorporates the limitations of, and corresponding analysis to claim 5. Further, claim 7 recites the following additional elements:
“The method of claim 5 further comprising: stopping the training of the target ML model if the overfitting status indicating occurrence of overfitting.” (In step 2A, prong 2, this is considered mere instructions to apply an exception using generic computer – see MPEP § 2106.05(f)). (In step 2B, this is also considered mere instructions to apply an exception using generic computer – see MPEP § 2106.05(f)).
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim 8:
Regarding claim 8, it is dependent upon claim 7, and thereby incorporates the limitations of, and corresponding analysis to claim 7. Further, claim 8 recites the following additional elements:
“The method of claim 7, wherein the training history comprises validation losses;” (In step 2A, prong 2, this is considered as mere field of use or technological environment in which to apply a judicial exception, see MPEP § 2106.05(h)). (In step 2B, this is also considered mere field of use or technological environment in which to apply a judicial exception - see MPEP § 2106.05(h)).
“and wherein the method further comprises: outputting an epoch having a lowest validation loss.” (In step 2A, prong 2, this is considered insignificant extra-solution activity of mere data gathering – see MPEP § 2106.05(g)). (In step 2B, this is also considered insignificant extra-solution activity of mere data gathering, which are well understood routines and conventional activity, see receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 – see MPEP § 2106.05(g)).
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-3, and 9 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Baker J. et al, (US. Patent Application Publication 20220383111 A1) effectively filed on September 9, 2020, (hereafter Baker).
Claim 1:
Regarding claim 1, Baker teaches “A method comprising: obtaining training-history data points and corresponding labels of one or more trained artificial-intelligence (AI) models, each label indicating an overfitting status of the corresponding training-history data point;”
See Baker in paragraph [0049] describing, “Block 407 is a data store that stores a history of auxiliary data for each datum for each round of processing of that datum by block 405 or block 406. The history may comprise data recorded for one or more previous iterations of training and may comprise a history of training on that datum for each of a plurality of modules and for a plurality of contexts in which each module is trained wherein the context may comprise the module being trained as part of a larger network.” Further, see Baker in paragraph [0043] describing, “Block 401 is a pool of data streams of training data. In some embodiments, each datum of training data comprises a vector of numerical values representing the values input into the input layer of the parent network. Each datum of training data further comprises some representation of the desired output for the datum, which, in the case of a classification problem, may be a label from a finite set of labels and in the case of a regression problem may be represented by a scalar or vector of numerical values. In either case, the desired output may be represented by an error-loss function by which computer system 200 may compute the cost of the deviation of a particular output from the desired output computed on a given datum in a representation such that computer system 200 can compute the derivative of the error-loss function with respect to each output value.” Further, see Baker in paragraph [0052] describing, “Block 413 comprises one or more modules that have been modified in order to make a comparative performance test. For example, a change in the architecture may be made and tested to see if there is a significant improvement in performance or an improvement in performance/cost. As another example, a datum may be tested by training a module both on a data stream including the datum and on a data stream not including the datum. This comparison testing may be used to diagnose overfitting or even to diagnose a datum that has been mislabeled in the training data.” Further, see Baker in paragraph [0092] describing, “In various embodiments, learning coach 520 may conduct various kinds of tests or experiments. For example, learning coach 520 may compare the performance of a selected system configuration on a selected set of development test data while learning coach 520 varies the selection of training data. Learning coach 520 may also compare the performance of a trained model when learning coach 520 varies the development test set. From this information, learning coach 520 may estimate the bias and variance of the models and diagnose problems of underfitting and overfitting and try various remedies as illustrated in FIG. 11 or various existing remedies for overfitting or underfitting that are well-known to those skilled in the art of machine learning and neural network training.” Here, Baker establishes retrieving history of training from auxiliary datum that contains corresponding labels. The datum is used for diagnosing overfitting, the labels correspond to datum can be seen to indicate a status that corresponds to the training history.
Further, Baker teaches “and training one or more classifiers using the obtained training-history data points and the corresponding labels.”
See Baker in paragraph [0102] describing, “ In a detector or classifier, the computation may be skipped if the output value is sufficiently extreme in either direction, that is, if the module is sure of its decision of detection or non-detection (e.g., has a confidence value greater than a threshold that an object was detected or have a confidence value greater than a threshold that the object was not detected).” Here, Baker establishes a detector as a classifier. Further, see Baker in paragraph [0109] describing, “In an example embodiment, a front-end detector module may first be trained as a complete stand-alone network. In some embodiments, there may be labeled training data for the feature and the feature detector module is initially trained on this labeled data, so the output of the feature detector is easily interpretable.” Here, Baker establishes training of a detector or classifier trained on labeled training data which can be seen as training data with corresponding labels. Further, see Baker in paragraph [0111] describing, “In some embodiments, the parent network illustrated in FIG. 7 and all the modules may be trained by the process illustrated in FIG. 1. In some embodiments, a first classifier back-end module may be trained in the context of a parent network such as illustrated in FIG. 7. Then a stand-alone second classifier back-end module may be designed to optimize its interpretability. The second, more interpretable, classifier back-end module may then be trained by imitation learning as illustrated in FIG. 8. Finally, the second classifier back-end module may be substituted into the parent network in place of the first classifier back-end module for further training.” Further, see Baker in paragraph [0043] describing, “Block 401 is a pool of data streams of training data. In some embodiments, each datum of training data comprises a vector of numerical values representing the values input into the input layer of the parent network. Each datum of training data further comprises some representation of the desired output for the datum, which, in the case of a classification problem, may be a label from a finite set of labels and in the case of a regression problem may be represented by a scalar or vector of numerical values. In either case, the desired output may be represented by an error-loss function by which computer system 200 may compute the cost of the deviation of a particular output from the desired output computed on a given datum in a representation such that computer system 200 can compute the derivative of the error-loss function with respect to each output value.” Here, Baker establishes training of classifier in the context of a parent network, this parent network has obtained training datum with labels as shown.
Claim 2:
Regarding claim 2, Baker teaches the limitations of claim 1.
Further, Baker teaches “The method of claim 1, wherein the one or more classifiers comprise one or more time-series classifiers.”
See Baker in paragraph [0111] describing, “In some embodiments, the parent network illustrated in FIG. 7 and all the modules may be trained by the process illustrated in FIG. 1. In some embodiments, a first classifier back-end module may be trained in the context of a parent network such as illustrated in FIG. 7. Then a stand-alone second classifier back-end module may be designed to optimize its interpretability. The second, more interpretable, classifier back-end module may then be trained by imitation learning as illustrated in FIG. 8. Finally, the second classifier back-end module may be substituted into the parent network in place of the first classifier back-end module for further training.” Further, see Baker in paragraph [0129] describing, “ Back-end module 1031 represents a recurrent neural network, which may be represented by computer system 200 as a feed forward network comprising multiple modified copies of the recurrent network with each recurrent connection replaced by a connection that goes to the next copy of the recurrent network. Such unfolding of a recurrent neural network is well known to those skilled in the art of training recurrent neural networks. Such an unfolded recurrent network may be used to model a sequence of data such as a time series of measurements, a text document, an audio recording or a video.” Here, Baker establishes the back end module as a classifier and the back-end module used to module time series, making it be seen as a time-series classifier.
Claim 3:
Regarding claim 3, Baker teaches “A method comprising: obtaining training history of a trained target ML model;”
See Baker in paragraph [0049] describing, “Block 407 is a data store that stores a history of auxiliary data for each datum for each round of processing of that datum by block 405 or block 406. The history may comprise data recorded for one or more previous iterations of training and may comprise a history of training on that datum for each of a plurality of modules and for a plurality of contexts in which each module is trained wherein the context may comprise the module being trained as part of a larger network.”
Further, Baker teaches “obtaining validation losses from the obtained training history;”
See Baker in paragraph [0043] describing, “Block 401 is a pool of data streams of training data. In some embodiments, each datum of training data comprises a vector of numerical values representing the values input into the input layer of the parent network. Each datum of training data further comprises some representation of the desired output for the datum, which, in the case of a classification problem, may be a label from a finite set of labels and in the case of a regression problem may be represented by a scalar or vector of numerical values. In either case, the desired output may be represented by an error-loss function by which computer system 200 may compute the cost of the deviation of a particular output from the desired output computed on a given datum in a representation such that computer system 200 can compute the derivative of the error-loss function with respect to each output value.”
Further, Baker teaches “and using one or more trained classifiers with the obtained validation losses inputting thereto for identifying an overfitting status of the trained target ML model.”
See Baker in paragraph [0024] describing, “As used herein, the terms “step” or “block” in the block diagrams and flowcharts refers to a step of a computer-implemented process executed by a computer system, which may be implemented as a machine learning system or an assembly of machine learning systems. Accordingly, each step or block can be embodied as a set of computer executable instructions stored in the memory of a computer system that, when executed by a processor of the computer system, cause the computer system to perform the described function(s). Each block can be implemented as either a machine learning system or as a nonmachine learning system, according to the function described in association with each particular block. Furthermore, each block can refer to one of multiple steps of a process embodied by computer-implemented instructions executed by a computer system (which may include, in whole or in part, a machine learning system) or an individual computer system (which may include, e.g., a machine learning system) executing the described step, which is in turn connected with other computer systems (which may include, e.g., additional machine learning systems) for executing the overarching process described in connection with each figure or figures.” Further, see Baker in paragraph [0028] describing, “Another aspect of the invention is that computer system 200 may process each datum many times, retaining and using auxiliary information from previous passes of processing. During training by stochastic gradient descent, for example, each datum may be processed for each of millions of epochs of training. During development testing, especially development testing automated with the assistant of a learning coach, computer system 200 may perform repeated testing during the training process. For example, in testing for overfitting on a module-by-module basis, computer system 200 may conduct thousands of tests.” Here, Baker establishes testing for overfitting which can be seen as identifying an overfitting status, in a computer system established to be a machine learning system which can be the trained target ML model. Further, see Baker in paragraph [0043] describing, “Block 401 is a pool of data streams of training data. In some embodiments, each datum of training data comprises a vector of numerical values representing the values input into the input layer of the parent network. Each datum of training data further comprises some representation of the desired output for the datum, which, in the case of a classification problem, may be a label from a finite set of labels and in the case of a regression problem may be represented by a scalar or vector of numerical values. In either case, the desired output may be represented by an error-loss function by which computer system 200 may compute the cost of the deviation of a particular output from the desired output computed on a given datum in a representation such that computer system 200 can compute the derivative of the error-loss function with respect to each output value.” Here, Baker establishes using a computer system which is established to be a machine learning system capable of doing a classification problem making it be seen as a classifier that computes a loss function for training history.
Claim 9:
Regarding claim 9, Baker teaches “A device comprising: a processor coupled to a memory, the processor being configured to execute computer-readable instructions to cause the device to: obtain training-history data points and corresponding labels of one or more trained artificial- intelligence (AI) models, each label indicating an overfitting status of the corresponding training- history data point;”
See Baker in paragraph [0024] describing, “The following description has set forth aspects of computer-implemented devices and/or processes via the use of block diagrams, flowcharts, and/or examples, which may contain one or more functions and/or operations. As used herein, the terms “step” or “block” in the block diagrams and flowcharts refers to a step of a computer-implemented process executed by a computer system, which may be implemented as a machine learning system or an assembly of machine learning systems. Accordingly, each step or block can be embodied as a set of computer executable instructions stored in the memory of a computer system that, when executed by a processor of the computer system, cause the computer system to perform the described function(s).” Further, see Baker in paragraph [0043] describing, “Block 401 is a pool of data streams of training data. In some embodiments, each datum of training data comprises a vector of numerical values representing the values input into the input layer of the parent network. Each datum of training data further comprises some representation of the desired output for the datum, which, in the case of a classification problem, may be a label from a finite set of labels and in the case of a regression problem may be represented by a scalar or vector of numerical values. In either case, the desired output may be represented by an error-loss function by which computer system 200 may compute the cost of the deviation of a particular output from the desired output computed on a given datum in a representation such that computer system 200 can compute the derivative of the error-loss function with respect to each output value.” Further, see Baker in paragraph [0052] describing, “Block 413 comprises one or more modules that have been modified in order to make a comparative performance test. For example, a change in the architecture may be made and tested to see if there is a significant improvement in performance or an improvement in performance/cost. As another example, a datum may be tested by training a module both on a data stream including the datum and on a data stream not including the datum. This comparison testing may be used to diagnose overfitting or even to diagnose a datum that has been mislabeled in the training data.” Further, see Baker in paragraph [0092] describing, “In various embodiments, learning coach 520 may conduct various kinds of tests or experiments. For example, learning coach 520 may compare the performance of a selected system configuration on a selected set of development test data while learning coach 520 varies the selection of training data. Learning coach 520 may also compare the performance of a trained model when learning coach 520 varies the development test set. From this information, learning coach 520 may estimate the bias and variance of the models and diagnose problems of underfitting and overfitting and try various remedies as illustrated in FIG. 11 or various existing remedies for overfitting or underfitting that are well-known to those skilled in the art of machine learning and neural network training.” Here, Baker establishes retrieving history of training from auxiliary datum that contains corresponding labels. The datum is used for diagnosing overfitting, the labels correspond to datum can be seen to indicate a status that corresponds to the training history.
Further, Baker teaches “and train one or more classifiers using the obtained training-history data points and the corresponding labels.”
See Baker in paragraph [0102] describing, “ In a detector or classifier, the computation may be skipped if the output value is sufficiently extreme in either direction, that is, if the module is sure of its decision of detection or non-detection (e.g., has a confidence value greater than a threshold that an object was detected or have a confidence value greater than a threshold that the object was not detected).” Here, Baker establishes a detector as a classifier. Further, see Baker in paragraph [0109] describing, “In an example embodiment, a front-end detector module may first be trained as a complete stand-alone network. In some embodiments, there may be labeled training data for the feature and the feature detector module is initially trained on this labeled data, so the output of the feature detector is easily interpretable.” Here, Baker establishes training of a detector or classifier trained on labeled training data which can be seen as training data with corresponding labels. Further, see Baker in paragraph [0111] describing, “In some embodiments, the parent network illustrated in FIG. 7 and all the modules may be trained by the process illustrated in FIG. 1. In some embodiments, a first classifier back-end module may be trained in the context of a parent network such as illustrated in FIG. 7. Then a stand-alone second classifier back-end module may be designed to optimize its interpretability. The second, more interpretable, classifier back-end module may then be trained by imitation learning as illustrated in FIG. 8. Finally, the second classifier back-end module may be substituted into the parent network in place of the first classifier back-end module for further training.” Further, see Baker in paragraph [0043] describing, “Block 401 is a pool of data streams of training data. In some embodiments, each datum of training data comprises a vector of numerical values representing the values input into the input layer of the parent network. Each datum of training data further comprises some representation of the desired output for the datum, which, in the case of a classification problem, may be a label from a finite set of labels and in the case of a regression problem may be represented by a scalar or vector of numerical values. In either case, the desired output may be represented by an error-loss function by which computer system 200 may compute the cost of the deviation of a particular output from the desired output computed on a given datum in a representation such that computer system 200 can compute the derivative of the error-loss function with respect to each output value.” Here, Baker establishes training of classifier in the context of a parent network, this parent network has obtained training datum with labels as shown.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 4 is rejected under 35 U.S.C. 103 as being unpatentable over Baker et al., in view of Power A. et al, "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets” available at https://arxiv.org/abs/2201.02177, effectively published on January 6, 2022, (hereafter Power).
Claim 4:
Regarding claim 4, Baker teaches the limitations of claim 3.
Baker does not appear to explicitly teach “The method of claim 3 further comprising: interpolating the obtained validation losses.”,
However in the same field of art, Power teaches “The method of claim 3 further comprising: interpolating the obtained validation losses.”
See Power in section A.3 Related Work on page 9 describing, “They find that the classical U-shaped validation loss curve is followed in some settings (including neural network training) by a second descent of loss that starts around the minimal capacity that is needed to interpolate any training data. We observe a second descent in validation loss (though not accuracy) as a function of the amount of training in some of our experiments, and it happens past the point of interpolating the training data.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of Baker with the teachings of Power by using Baker’s teachings of training classifiers and modules using training history, and incorporate with Power’s teaching of interpolating validation losses.
One of ordinary skill in the art would be motivated to do so because by integrating Power’s frameworks into the methods of Baker, which are both in the same field of art, one of ordinary skill in the art would bring a method of a transformer that can “show that neural networks are capable of generalizing to the empty slots in a variety of binary op tables” (Power, page 2 section 1 Introduction), “show that, long after severely overfitting, validation accuracy sometimes suddenly begins to increase from chance level toward perfect generalization. We call this phenomenon ‘grokking’.” (Power, page 2 section 1 Introduction), “compare various optimization details to measure their impact on data efficiency. We find that weight decay is particularly effective at improving generalization on the tasks we study” (Power, page 2 section 1 Introduction) and “visualize the symbol embeddings learned by these networks and find that they sometimes uncover recognizable structure of the mathematical objects represented by the symbols” (Power, page 2 section 1 Introduction).
Claim(s) 5-7 are rejected under 35 U.S.C. 103 as being unpatentable over Baker et al., in view of Fröhlich H. et al, (US. Patent Application Publication 20240112070 A1) effectively filed on September 30, 2022, (hereafter Fröhlich).
Claim 5:
Regarding claim 5, Baker teaches “A method for performing during training of a target ML model, the method comprising: obtaining training history of the target ML model;”
See Baker in paragraph [0049] describing, “Block 407 is a data store that stores a history of auxiliary data for each datum for each round of processing of that datum by block 405 or block 406. The history may comprise data recorded for one or more previous iterations of training and may comprise a history of training on that datum for each of a plurality of modules and for a plurality of contexts in which each module is trained wherein the context may comprise the module being trained as part of a larger network.”
Baker does not appear to explicitly teach “using one or more trained classifiers with at least a portion of the training history inputting thereto for generating a second set of inferences; and using the first and second sets of inferences for detecting an overfitting status of the target ML model.”,
However in the same field of art, Fröhlich teaches “using one or more trained classifiers with at least a portion of the training history inputting thereto for generating a second set of inferences;”
See Fröhlich in paragraphs [0019-0027] describing, “In brief, some embodiments may comprise the following operations: 1. define a time window of interest where the drift mode will be searched; 2. train a reduced reference model R0 at time t0 with a small sample d0 of the main dataset, that is, the training dataset for the main model upon which the model R0 is based, up to the overfitting point—and store the reference model R0; 3. then, for a predetermined number of steps n: a. at time ti, when new samples di are available, train a new version of the reduced reference model Ri, until the new version overfits the new data samples di—note that |di|=|d0| samples may be expected to be acquired between ti−1 and ti; b. compare respective inferences generated by R0 and Ri, and store the difference between those values as δ (an absolute delta value); 4. compose a series of the n deltas available, so as to define a drift mode curve; and 5. apply a suitable algorithm to determine the appropriate drift detection model—this algorithm may leverage similarity metrics, statistics and/or machine learning to match the drift mode curve to drift mode curve templates.” Further, see Fröhlich in paragraph [0010] describing, “Some embodiments may provide for the construction of the drift model regardless of whether the model serves a supervised or unsupervised problem, for both classification and regression problems, since the drift mode curve(s) for a model may be constructed using only the absolute difference in inference of the sub-models that are in use.” Further, see Fröhlich in paragraph [0038] describing, “With continued reference to FIG. 2 , inferencing with the sub-models RRM, and RM_t+1, may be performed 210 using the dataset collected between time t and t+1. Particularly, each of the sub-models may be used to make an inference/prediction/forecast based on that dataset. The outputs, or values, generated by each of the sub-models using the dataset may then be stored.” Here, Fröhlich establishes a model that serves for classification problems, making it be seen as a classifier. This model then is said to use generated inferences from a model that uses a training dataset. The model that generates the inference is a sub model and each of the sub models make an inference that is stored in which the inference stored can be the second set of inferences.
Further, Fröhlich teaches “and using the first and second sets of inferences for detecting an overfitting status of the target ML model.”
See Fröhlich in paragraphs [0019-0027] describing, “In brief, some embodiments may comprise the following operations: 1. define a time window of interest where the drift mode will be searched; 2. train a reduced reference model R0 at time t0 with a small sample d0 of the main dataset, that is, the training dataset for the main model upon which the model R0 is based, up to the overfitting point—and store the reference model R0; 3. then, for a predetermined number of steps n: a. at time ti, when new samples di are available, train a new version of the reduced reference model Ri, until the new version overfits the new data samples di—note that |di|=|d0| samples may be expected to be acquired between ti−1 and ti; b. compare respective inferences generated by R0 and Ri, and store the difference between those values as δ (an absolute delta value); 4. compose a series of the n deltas available, so as to define a drift mode curve; and 5. apply a suitable algorithm to determine the appropriate drift detection model—this algorithm may leverage similarity metrics, statistics and/or machine learning to match the drift mode curve to drift mode curve templates.” Further, see Fröhlich in paragraph [0027] describing, “In some embodiments, the reduced reference models R may be used only for drift mode building and not for true inference and other decision-making processes. That is, the reduced reference models R may be model trained up to the point of overfitting the applicable dataset as this is, in theory, the best possible performance of that model R, which may be likely to be unachievable in operation due to loss of generalization implied by the reduced reference model R. Embodiments may consider that the overfitted models provide a coherent comparison basis for their inference results quality.” Further, see Fröhlich in paragraph [0042] describing, “With continued reference to the example operations disclosed in FIG. 2 , attention is directed to further details concerning some examples of a reference reduced model RRM. In some embodiments, an overfitted reduced reference model R0 may be trained at the start of this interval, at a timestamp to. This model is referred to as a “reference” model because all other sub-models, or reference reduced models, may be compared to this one.” Here, Fröhlich establishes a reference model that can be seen as the target ML model that is trained up to the point of overfitting implying a detecting of overfitting. This reference model is established to be compared to sub-models which are established in previous limitations to produce inferences that are stored and used for training, seen as sets of inferences.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of Baker with the teachings of Fröhlich by using Baker’s teachings of training classifiers and modules using training history, and incorporate with Fröhlich’s teachings of defining a time window of a model and generating inferences.
One of ordinary skill in the art would be motivated to do so because by integrating Fröhlich’s frameworks into the methods of Baker, which are both in the same field of art, one of ordinary skill in the art would bring “an embodiment [that] may enable diagnoses of the performance of a machine learning model.” (Fröhlich, paragraph [0013]) and “define a time window of interest where the drift mode will be searched;” (Fröhlich, paragraph [0019]).
Claim 6:
Regarding claim 6, Baker in view of Fröhlich teaches the limitations of claim 5.
Baker does not appear to explicitly teach “The method of claim 5 further comprising: obtaining the at least portion of the training history using a rolling window.”,
However in the same field of art, Fröhlich teaches “The method of claim 5 further comprising: obtaining the at least portion of the training history using a rolling window.”
See Fröhlich in paragraphs [0019-0027] describing, “In brief, some embodiments may comprise the following operations: 1. define a time window of interest where the drift mode will be searched; 2. train a reduced reference model R0 at time t0 with a small sample d0 of the main dataset, that is, the training dataset for the main model upon which the model R0 is based, up to the overfitting point—and store the reference model R0; 3. then, for a predetermined number of steps n: a. at time ti, when new samples di are available, train a new version of the reduced reference model Ri, until the new version overfits the new data samples di—note that |di|=|d0| samples may be expected to be acquired between ti−1 and ti; b. compare respective inferences generated by R0 and Ri, and store the difference between those values as δ (an absolute delta value); 4. compose a series of the n deltas available, so as to define a drift mode curve; and 5. apply a suitable algorithm to determine the appropriate drift detection model—this algorithm may leverage similarity metrics, statistics and/or machine learning to match the drift mode curve to drift mode curve templates.” Here, Fröhlich establishes a time window being used in a process for obtaining a training dataset or training history, and the time window can be seen as a rolling window in this process. Further, see Fröhlich in paragraph [0062] describing, “A method, comprising: defining a time window during which a drift mode of a machine learning model will be evaluated, and the time window begins at t0; training, beginning at the time t0, a reduced reference model R0 with a data sample d0 that is a subset of a training dataset that was used to train the machine learning model; for ‘n’ iterations: at a time ti, when new data samples di are available, train a new version Ri of the reduced reference model R0; after Ri is trained, collect data vi; and compare respective inference values generated by the reduced reference model R0 and Ri, using the data samples d0 and vi, and store an absolute difference between the inference values;” Here, Fröhlich further establishes a time window as a rolling window, as known in the art can comprise of a time window that begins at a time and at later times in the process new data becomes available, at each iteration or interval a new version of the model is trained using the available data and this process repeats over time which is consistent with the definition of a rolling window in the art.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of Baker with the teachings of Fröhlich by using Baker’s teachings of training classifiers and modules using training history, and incorporate with Fröhlich’s teachings of defining a time window of a model and generating inferences.
One of ordinary skill in the art would be motivated to do so because by integrating Fröhlich’s frameworks into the methods of Baker, which are both in the same field of art, one of ordinary skill in the art would bring “an embodiment [that] may enable diagnoses of the performance of a machine learning model.” (Fröhlich, paragraph [0013]) and “define a time window of interest where the drift mode will be searched;” (Fröhlich, paragraph [0019]).
Claim 7:
Regarding claim 7, Baker in view of Fröhlich teaches the limitations of claim 5.
Further, Baker teaches “The method of claim 5 further comprising: stopping the training of the target ML model if the overfitting status indicating occurrence of overfitting.”
See Baker in paragraph [0050] describing, “For example, development testing may be used to make a decision for early stopping in the training of a module. Module-by-module development testing enables computer system 200 to stop training a module that may be overfitting its training data while continuing to train other modules.” Further, see Baker in paragraph [0003] describing, “The present invention, in one general aspect, is directed to a machine-learning computer system that breaks a neural network into a plurality of modules and tracks the training process module-by-module and datum-by-datum, recording auxiliary information during one epoch of the training process for retrieval during a later epoch.”
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over Baker et al., in view of Fröhlich H. et al, and further in view of Voelker A. et al, (US. Patent Application Publication 20210133568 A1) effectively filed on October 30, 2020, (hereafter Voelker).
Claim 8:
Regarding claim 8, Baker in view of Fröhlich teaches the limitations of claim 7.
Further, Baker teaches “The method of claim 7, wherein the training history comprises validation losses;”
See Baker in paragraph [0043] describing, “Block 401 is a pool of data streams of training data. In some embodiments, each datum of training data comprises a vector of numerical values representing the values input into the input layer of the parent network. Each datum of training data further comprises some representation of the desired output for the datum, which, in the case of a classification problem, may be a label from a finite set of labels and in the case of a regression problem may be represented by a scalar or vector of numerical values. In either case, the desired output may be represented by an error-loss function by which computer system 200 may compute the cost of the deviation of a particular output from the desired output computed on a given datum in a representation such that computer system 200 can compute the derivative of the error-loss function with respect to each output value.”
Further, Baker teaches “and wherein the method further comprises: outputting an epoch...”
See Baker in paragraph [0043] describing, “Determining whether to skip training of the first network module with the first datum in the Nth epoch can be based on: an estimate that a change in output of the first network module will be less than a threshold value for the first datum relative to an amount of change in other network modules of the parent neural network; whether the first network module has a stable extreme activation for the first datum for epochs prior to the Nth epoch; whether the parent neural network has a stable output for the first datum for epochs prior to the Nth epoch; and/or a magnitude of a partial derivative of an error-loss function for the parent neural network with respect to an output of the first network module.” Here, Baker establishes an output of an epoch based on a stable output/magnitude of an error-loss function.
Neither Baker or Fröhlich appear to explicitly teach “…having a lowest validation loss.”,
However, Voelker in an analogous system teaches “…having a lowest validation loss.”
See Voelker in paragraph [0125] describing, “To interpolate between ANN and SNN regimes, we set ω differently from one training epoch to the next, in an analogous manner to scheduling learning rates. Since ω is exponential in bit-precision, we vary ω on a logarithmic scale across the interval [ωh, ωt], where ωh is set to achieve rapid convergence during the initial stages of training, and ωl depends on the hardware and application. Once ω=ω1, training is continued until validation error stops improving, and only the model with the lowest validation loss during this fine-tuning phase is saved.”
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the base reference of Baker and Fröhlich with the teachings of Voelker by using Baker’s teachings of training classifiers and modules using training history and Fröhlich’s teachings of defining a time window of a model and generating inferences, and incorporate with Voelker’s teachings of interpolating between an ANN and SNN while finding the lowest validation loss.
One of ordinary skill in the art would be motivated to do so because by integrating Voelker’s frameworks into the methods of Baker and Fröhlich, which are both in the same field of art, one of ordinary skill in the art would bring “ methods for efficiently implementing multi-bit spiking neural networks on digital hardware by avoiding the computation of MAC operations whenever a neuron does not emit any spikes on a given timestep;” (Fröhlich, paragraph [0012]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HASSAN R SESAY whose telephone number is (571)272-8493. The examiner can normally be reached Monday-Friday 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HASSAN RAMADAN SESAY/Examiner, Art Unit 2146
/USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights