Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is
directed to an abstract idea without significantly more.
Step 1: The claims recite a system, method, and non-transitory computer-readable
medium, each of which are one of the four categories of eligible subject matter.
Claims 1, 10, and 19
Step 2A Prong 1: The claims recite the following limitations:
execute both new and original version algorithms together against un-labeled data, and selected samples, where the algorithm outcome differences are raised for expert review (Mental Process); and compare the new and original model algorithm performances by evaluating the counts where each of the new or original algorithms are correct and when their outputs are similar to determine if a new model should replace an original model (Mental Process).
Under the broadest reasonable interpretation of the claim language, analyzing differences between different versions of an algorithm and comparing versions based on performance are mental processes because a human mind can practically perform the processes with the aid of a pencil and paper. Accordingly, the claims recite an abstract idea.
Step 2A Prong 2: The judicial exceptions are not integrated into a practical
application. The claims recite the following additional elements:
update an original algorithm to a new algorithm based on expert analysis of the outcome of the original algorithm.
In view of P0023 of the specification of the instant application, updating an original algorithm to a new algorithm is generally linking the abstract ideas to the technological environment of machine learning, as discussed in MPEP 2106.05(h). The processors, memory, and instructions configuring the processors are generic computing components recited at a high level as a means to apply the judicial exception, as discussed in MPEP 2106.05(f). Receiving datasets and combining values are mere data
gathering, which is an insignificant extra-solution activity as discussed in MPEP 2106.05(g). The claims are directed towards an abstract idea.
Step 2B: The claims do not include additional elements that are sufficient to
amount to significantly more than the judicial exception. Updating an original algorithm to a new algorithm is generally linking the abstract ideas to the technological environment of machine learning, as discussed in MPEP 2106.05(h). The processors, memory, and instructions configuring the processors are generic computing components recited at a high level as a means to apply the judicial exception, as discussed in MPEP 2106.05(f). Receiving datasets and combining values are mere data
gathering, which is an insignificant extra-solution activity as discussed in MPEP 2106.05(g). The claims are not patent eligible.
Dependent Claims:
Claims 2-3, 8, 11-12, and 17: These claims recite further abstract ideas (mental processes) and thus are ineligible.
Claims 3-7, 9, 12-16, and 18: These claims recite further mere data gathering and generally linking the abstract ideas to the technological environment of machine learning and as explained above these do not provide a practical application or inventive concept and thus are ineligible.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 7 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph.
Claim 7 recites the limitation "the annotations" in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shirakawa Pub. No.: US 20210158213 A1), hereafter Shirakawa.
Regarding claims 1, 10, and 19, Shirakawa teaches update an original algorithm to a new algorithm based on expert analysis of the outcome of the original algorithm (In step S614, the continuous learning cycle execution unit 318 causes the learning model learning unit 316 to relearn the learning model (generation of a new learning model) based on analysis performed in step S613, P0091-P0093); and execute both new and original version algorithms together against un-labeled data, and selected samples, where the algorithm outcome differences are raised for expert review (In step S615, the continuous learning cycle execution unit 318 compares the prediction accuracy of the new learning model evaluated in step S614 with the prediction accuracy of the learning model 322, which is in operation, evaluated in step S612, P0094); and compare the new and original model algorithm performances by evaluating the counts where each of the new or original algorithms are correct and when their outputs are similar to determine if a new model should replace an original model (In step S615, prediction accuracy of both the existing learning model and the new learning model are compared. If the new learning model is more accurate, it replaces the current model in step S616, P0094-P0096).
Regarding claims 2, 11, and 20, Shirakawa teaches the limitations of claims 1, 10, and 19 as outlined above. Shirakawa further teaches wherein improved performance of an event detection algorithm is verified by offering an expert to annotate events, wherein outputs of a trained algorithm and the original algorithm differ, and wherein a decision of taking a modified algorithm into use is based on accuracy results associated with the annotated events (continuous learning cycle execution unit 318 compares the prediction accuracy of both the current model and the new model. The prediction accuracy is based on an evaluation of feedback data. The evaluation of feedback data is interpreted as annotated events. If the new model is determined to have improved prediction accuracy, the existing model is modified to reflect improvements of the new model, P0090-P0091, P0095).
Regarding claims 3 and 12, Shirakawa teaches the limitations of claims 1 and 10 as outlined above. Shirakawa further teaches wherein the expert review can be from at least one of a human or system, with additional information data in use which collected with invasive methods or data collected later in time (continuous learning cycle execution unit 318 performs the evaluation by controlling learning model evaluation unit 315, P0040, P0089. More feedback data may be collected in future steps, P0097-P0099).
Regarding claims 4 and 13, Shirakawa teaches the limitations of claims 1 and 10 as outlined above. Shirakawa further teaches wherein events are raised for re-annotation in an on-line process for data already collected but without verified annotations (Feedback data used for training purposes and feedback data used for evaluation are different, but at this step both feedback data have been collected, P0092-P0093).
Regarding claims 5 and 14, Shirakawa teaches the limitations of claims 1 and 10 as outlined above. Shirakawa further teaches wherein within a comparison, n (a number of) annotations are raised for re-annotation and a trained model needs to outperform an original model with events in scope (The new model must have better prediction accuracy than the existing learning model based on evaluations regarding feedback data, P0090-P0095).
Regarding claims 6 and 15, Shirakawa teaches the limitations of claims 1 and 10 as outlined above. Shirakawa further teaches wherein methods to update an algorithm include at least one of: training methods utilizing existing databases, or methods utilizing new sample(s) only (During the continuous learning cycle, new feedback data is received after discarding feedback data of the current learning cycle for each learning cycle, P0064).
Regarding claims 7 and 16, Shirakawa teaches the limitations of claims 1 and 10 as outlined above. Shirakawa further teaches wherein the annotations are weighted based on their estimated criticality or importance in a model selection / determination / decision process (Evaluation results are assigned a corresponding correlation coefficient that represents how likely prediction accuracy is deteriorating. If it is determined that the prediction accuracy is deteriorating, continuous learning cycle execution unit proceeds to initiate another cycle of continuous learning. P0087, P0099-P0100).
Regarding claims 8 and 17, Shirakawa teaches the limitations of claims 1 and 10 as outlined above. Shirakawa further teaches wherein acceptance criteria for either model is pre-defined with limits developed by an expert in continuous learning models (User who is operating learning model management system 101 may decide what threshold or criteria must be met for a model to be accepted or further trained, P0069, P0079-P0081).
Regarding claims 9 and 18, Shirakawa teaches the limitations of claims 8 and 17 as outlined above. Shirakawa further teaches wherein if the acceptance criteria is not satisfied due to low sample sizes below a defined threshold, the comparison is continued by feeding new data samples to the models (When criteria is not met, training continues and the model’s prediction accuracy is reevaluated until the criteria or threshold are met, P0081).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20230087777 A1 (Sha et al) teaches a method including model updates based on evaluation of a newer model.
US 20230206091 A1 (Pourmohammad et al) teaches a method including updating machine learning models based on human expert analysis of a current and new model.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHAN MOUNDI whose telephone number is (703)756-1547. The examiner can normally be reached 8:30 A.M. - 5 P.M..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Ell can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.M./ Examiner, Art Unit 2141
/MATTHEW ELL/ Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights