Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
The numbering of claims is not in accordance with 37 CFR 1.126 which requires the original numbering of the claims to be preserved throughout the prosecution. When claims are canceled, the remaining claims must not be renumbered. When new claims are presented, they must be numbered consecutively beginning with the number next following the highest numbered claims previously presented (whether entered or not).
Misnumbered claims. Applicant canceled claims 5-9 but claim 5 is still present. It is believed claims 6-9 were meant to be canceled.
Claim 12 is objected to under 37 CFR 1.75(c) as being in improper form because a multiple dependent claims 10 or 11. See MPEP § 608.01(n). The claim is actually proper but Applicant’s remarks and amendments appeared to want to remove all multiple dependent claims. Examiner is merely pointing out a remaining multiple dependent claim.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-5, 10-13, and 15-26 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Independent claim 1 recites, in part, “identifying a set of failure modes and determining for each failure mode a likelihood of that failure mode causing failure to meet the performance threshold...” It is not clear how the failure modes are the cause of failure to meet the performance threshold. From the claim language it seems that the algorithm is identified as not meeting a performance threshold and then a failure mode is explored. But the failure modes themselves were not the initial reason for failure. Why would the failure modes now have a likelihood of causing a failure to meet the performance threshold? Clarification is needed. Independent claim 16 recites similarly. The dependent claims are rejected based on their dependence to the independent claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites “[a] computer-implemented method of determining whether a task can be completed by machine learning, the method comprising: obtaining test data for the task; using the test data; for determining, for a plurality of machine learning algorithms, whether any of the machine learning algorithms is able to perform the task to meet a performance threshold; if none of the machine learning algorithms performs the task to the performance threshold, identifying a set of failure modes and determining for each failure mode a likelihood of that failure mode causing failure to meet the performance threshold; and providing an output indicating relative likelihoods of each failure mode of the set causing failure to meet the performance threshold.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “obtaining test data for the task,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
The limitation of “using the test data for determining, for a plurality of machine learning algorithms, whether any of the machine learning algorithms is able to perform the task to meet a performance threshold,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
The limitation of “if none of the machine learning algorithms performs the task to the performance threshold, identifying a set of failure modes and determining for each failure mode a likelihood of that failure mode causing failure to meet the performance threshold,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
That is, other than reciting “computer” nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “computer” language, “obtaining… determining… [and] identifying” in the context of this claim encompasses the user manually with pen and paper obtaining, determining, and identifying failures. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 (Additional Elements):
This judicial exception is not integrated into a practical application. In particular, the claim only recites element computer to enact obtaining, determining, and identifying. The computer in all steps is recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component.
The claim recites “providing an output indicating relative likelihoods of each failure mode of the set causing failure to meet the performance threshold.” Extra-solution activity.
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
As discussed above with respect to integration of the abstract idea into a practical application, the additional element of a computer to enact the obtaining, determining, and identifying steps amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 2 recites “[t]he method of claim 1, further comprising if one of the machine learning algorithms performs the task to the performance threshold, selecting that machine learning algorithm to perform the task.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “if one of the machine learning algorithms performs the task to the performance threshold, selecting that machine learning algorithm to perform the task,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 3 recites “[t]he method of claim 1, wherein after obtaining the test data, the method further comprises preparing the test data so that it is suitable for use by each of the machine learning algorithms of the plurality of machine learning algorithms.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “after obtaining the test data, the method further comprises preparing the test data so that it is suitable for use by each of the machine learning algorithms of the plurality of machine learning algorithms,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 4 recites “[t]he method of claim1, wherein the step of determining for a machine learning algorithm determines training and testing the machine learning algorithm at least once to determine whether an instance of the trained machine learning algorithm meets the performance threshold.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “wherein the step of determining for a machine learning algorithm determines training and testing the machine learning algorithm at least once to determine whether an instance of the trained machine learning algorithm meets the performance threshold,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 5 recites “[t]he method of claim 4, wherein the step of determining for a machine learning algorithm comprises a k-fold cross validation, wherein training and testing the machine learning algorithm occurs k times for k different divisions of the test data into training data and evaluation data to provide k instances of the trained machine learning algorithm, wherein for the k-fold cross validation, results for each of the k instances of the trained machine learning algorithm are averaged to determine whether the performance threshold is met.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “wherein the step of determining for a machine learning algorithm comprises a k-fold cross validation, wherein training and testing the machine learning algorithm occurs k times for k different divisions of the test data into training data and evaluation data to provide k instances of the trained machine learning algorithm, wherein for the k-fold cross validation, results for each of the k instances of the trained machine learning algorithm are averaged to determine whether the performance threshold is met,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 10 recites “[t]he method of claim1, wherein the step of determining for each failure mode a likelihood of that failure mode causing failure to meet the performance threshold comprises generating a plurality of data sets for each failure mode.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “wherein the step of determining for each failure mode a likelihood of that failure mode causing failure to meet the performance threshold comprises generating a plurality of data sets for each failure mode,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 11 recites “[t]he method of claim 10, wherein the generation of data sets for each failure mode is varied for each data set to provide a different likelihood of that failure mode applying to each data set.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “wherein the generation of data sets for each failure mode is varied for each data set to provide a different likelihood of that failure mode applying to each data set,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 12 recites “[t]he method of claim 10 or claim 11, wherein the step of determining for each failure mode a likelihood of that failure mode causing failure to meet the performance threshold comprises, for each failure mode, of determining how each data set for that failure mode performs for that machine learning algorithm and comparing the performance of the test data to each of the data sets.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “wherein the step of determining for each failure mode a likelihood of that failure mode causing failure to meet the performance threshold comprises, for each failure mode, of determining how each data set for that failure mode performs for that machine learning algorithm and comparing the performance of the test data to each of the data sets,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 13 recites “[t]he method of claim 12, wherein comparing the performance of the test data to each of the data sets comprises establishing where the test data lies on a linear interpolation between a least effective data set and a threshold data set that substantially performs at the performance threshold wherein the position of the test data on the linear interpolation is equated to a likelihood that the associated failure mode is responsible for machine learning failure.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “wherein comparing the performance of the test data to each of the data sets comprises establishing where the test data lies on a linear interpolation between a least effective data set and a threshold data set that substantially performs at the performance threshold wherein the position of the test data on the linear interpolation is equated to a likelihood that the associated failure mode is responsible for machine learning failure,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim 15 recites “[t]he method of claim 1 wherein the output is a listing of the failure modes in an order of likelihood that each failure mode is responsible for machine learning failure.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “method of claim 1,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim recites “wherein the output is a listing of the failure modes in an order of likelihood that each failure mode is responsible for machine learning failure.” Extra-solution activity.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claims 16-20, 21-24, and 25 are rejected based on similar rationale given to claims 1-5, 10-13 and 15, respectively.
Claim 26 recites “[t]he computing system of claim 25, wherein the output providing element is also adapted to provide a remediation strategy for providing a machine learnable data set for the task.”
Step 2A Prong 1 (Abstract Idea):
The limitation of “computing system of claim 25,” as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. Mental Process, Mathematics.
Step 2A Prong 2 (Additional Elements):
The claim recites “wherein the output providing element is also adapted to provide a remediation strategy for providing a machine learnable data set for the task.” Extra-solution activity.
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Step 2B (Inventive concept):
Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2 and 16-17 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Ghanta et al. US 2020/0034665 (hereinafter “Ghanta”).
Regarding claim 1, Ghanta teaches: A computer-implemented method of determining whether a task can be completed by machine learning, the method comprising: obtaining test data for the task; [FIG. 5 and 0106: “primary training module 302 trains 502 a first machine learning model for a first machine learning algorithm using a training data set 503.”]
using the test data; for determining, for a plurality of machine learning algorithms, whether any of the machine learning algorithms is able to perform the task to meet a performance threshold; [FIG. 5 and 0106: “the primary validation module 304 validates 504 the first machine learning algorithm/model using a validation data set 505a.” Also see 0108: “the secondary validation module 308 determines 512 whether the second machine learning algorithm/model is suitable for the assessing the predictive performance of the first machine learning algorithm/model for the inference data set.” Note, a first and a second machine learning algorithm, i.e., plurality of machine learning models.]
if none of the machine learning algorithms performs the task to the performance threshold, identifying a set of failure modes and determining for each failure mode a likelihood of that failure mode causing failure to meet the performance threshold; and [FIG. 5 and 0109: “the analysis module 310 determines 514 whether the predicted suitability of the first machine learning algorithm/model satisfies a predetermined suitability threshold. If so, the method 500 ends. Otherwise, the action module 312 triggers one or more actions associated with the first machine learning algorithm.”]
providing an output indicating relative likelihoods of each failure mode of the set causing failure to meet the performance threshold. [FIG. 5 and 0109: “may recommend 520 different machine learning algorithms for analyzing the inference data set, may update 522 suitability thresholds”]
Regarding claim 2, Ghanta teaches: The method of claim 1, further comprising if one of the machine learning algorithms performs the task to the performance threshold, selecting that machine learning algorithm to perform the task. [FIG. 5 and 0109: “the analysis module 310 determines 514 whether the predicted suitability of the first machine learning algorithm/model satisfies a predetermined suitability threshold. If so, the method 500 ends.”]
Claims 16-17 are rejected based on the same citations and rationale given to claims 1-2.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN D GIBSON whose telephone number is (571)431-0699. The examiner can normally be reached Monday - Friday 8:00 A.M.-4:00 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, BRYCE P BONZO can be reached at (571)-272-3655. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN D GIBSON/Primary Examiner, Art Unit 2113
Prosecution Insights