DETAILED ACTION
The following is a Non-Final Office action in response to communications received 2/13/26. Claims 1, 11, and 20 have been amended. Therefore, claims 1-24 are pending and addressed below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-24 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim(s) 1, 11, and 20 recite(s) the limitation(s) of “monitoring performance of a generated model while the generated model is being used for classification on live data received by the generated model, the live data including a plurality of data subgroups, the monitoring including determining a first performance value of the generated model at a first point in time and determining a second performance value of the generated model at a second point in time for the plurality of data subgroups associated with the generated model” and “identifying one or more low-performing subgroups where a performance value of the low-performing subgroup is below a predetermined threshold”. This/These limitation(s), as drafted, is(are) a process (processes) that, under its (their) broadest reasonable interpretation, cover(s) performance of the limitation(s) in the mind but for the recitation of generic computer components. That is, other than reciting “at least one processor” and “memory” in claim 11 and “a non-transitory computer program product”, “at least one processor”, and “at least one computing system” in claim 20, nothing in the claim elements precludes the steps from practically being performed in the mind.
The examiner also notes that “monitoring performance of a generated model while the generated model is being used for classification on live data received by the generated model, the live data including a plurality of data subgroups, the monitoring including determining a first performance value of the generated model at a first point in time and determining a second performance value of the generated model at a second point in time for the plurality of data subgroups associated with the generated model” involves subjective choices of how performance is monitored (observed), what factors and criteria are used to determine what is performance and a performance value, the type of performance value used (the examiner notes it could be numerical (1-10, 1-100, etc.), subjective types (critical and non-critical, high, medium, and low, clear, warning, and problem, etc.), color (red, yellow, green), or any other representation that can be thought of in the human mind, and the correspondence between monitored data and a “performance value”, the thresholds between different performance values, the factors, criteria, types, and correspondence of classifications, and includes the concepts of observation, evaluation, judgment, and opinion, and
“identifying one or more low-performing subgroups where a performance value of the low-performing subgroup is below a predetermined threshold” involves subjective choices of what constitutes “low” performing, if there a medium, average, high, severe, critical, or more types of performance with their own corresponding thresholds, and the correspondence of criteria and factors to a threshold for “low”, and includes the concepts of evaluation, judgment, and opinion. The mere nominal recitation of generic processing components does not take the claim limitation(s) out of the mental processes grouping. Thus, the claim(s) recite(s) a mental process, concepts that may be performed in the human mind, in this case being observation, evaluation, judgment, and opinion in claim 1, 11, and 20.
Additionally, the claims recite the limitation(s) “monitoring performance of a generated model while the generated model is being used for classification on live data including a plurality of data subgroups, the monitoring including determining a first performance value of the generated model at a first point in time and determining a second performance value of the generated model at a second point in time for the plurality of data subgroups associated with the generated model”, “rendering, within a graphical user interface, a plot including a first axis and a second axis, the first axis including a characterization of a first performance metric and the second axis including a characterization of a second performance metric”, and “rendering, within the graphical user interface and the plot, a first graphical object at a first location characterizing the first performance value and a second graphical object at a second location characterizing the second performance value”, which describe(s) the concept of “collecting information, analyzing it, and displaying certain results of the collection and analysis” (collecting live data (collecting)), classifying/determining performance values (analyzing), rendering within a GUI (displaying)), which corresponds to concepts identified as abstract ideas by the courts, such as in Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016). The concepts described in claim(s) 1, 11, and 20 are not meaningfully different than those concepts found by the courts to be abstract ideas.
This judicial exception is not integrated into a practical application because the additional elements recited including “rendering, within a graphical user interface, a plot including a first axis and a second axis, the first axis including a characterization of a first performance metric and the second axis including a characterization of a second performance metric”, “rendering, within the graphical user interface and the plot, a first graphical object at a first location characterizing the first performance value and a second graphical object at a second location characterizing the second performance value”, “determining the plurality of data subgroups associated with the generated model”, “terminating performance of the generated model for the identified low-performing subgroup”, and “improving the performance of the generated model by: generating a split-model for the identified low-performing subgroup; or removing the identified low-performing subgroup from the plurality of data subgroups” in claims 1, 11, and 20 are recited at a high level of generality, i.e., as generic processor performing a generic computer function of displaying a plot of data points. Generic processor limitations are no more than mere instructions to apply the exception using a generic computer component.
The examiner notes that “terminating performance of the generated model for the identified low-performing subgroup” and “removing the identified low-performing subgroup from the plurality of data subgroups” do not improve the functioning of the computer itself. The examiner also notes that simply stating that the following steps ‘improve the performance of the generated model’ is insufficient to indicate an improvement to the functioning of the computer. The improvement would have to flow from the claim limitations themselves and in this case neither “generating a split-model for the identified low-performing subgroup” or “removing the identified low-performing subgroup from the plurality of data subgroups” appear improve the performance of the generated model, let alone the functioning of the computer. Simply generating a model or removing a set of data would not improve the performance of either the generated model or the computer itself. The computer’s performance is not improved, the production time, production costs, efficiencies, accuracy, speed, and waste of the program or model do not correspond to an improvement of the computer itself. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the additional elements fail to improve the functionality of the computer itself.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea. Generic computer components recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system. Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology or effects a transformation or reduction of a particular article to a different state or thing. Their collective functions merely provide conventional computer implementation. Furthermore, the applicant’s own specification details the generic nature of the computing components, which also precludes them from presenting anything significantly more ([0090], fig. 35).
Claim(s) 2-10, 12-19, and 21-24 do(es) not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements when considered both individually and as an ordered combination do not amount to significantly more than the abstract idea. Generic computer components recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system. Thus, taken alone, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation.
Claims 2 and 12 involve a mental process in the use of a random model, given that “randomness” necessarily includes at least weights, selection of random generating function, and/or a seed and rendering of constant accuracy and cost and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claims 3 and 13 simply list possible parts of the performance metrics and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claims 4 and 14 simply render a line between the two objects and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claims 5 and 15 involve a mental process in the determining of a third performance value and also simply rendering the value and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claims 6 and 16 involve a mental process in the form of subjective choices used to choose a shape and/or color (from a subjective set of shapes and colors) and which characteristic corresponds to each (also includes a subjective choice) and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claims 7 and 17 involve mental processes in the form of classification on live data and generation of a third and fourth performance value as detailed above and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claims 8 and 18 involve mental processes in the subjective choice of which subgroups to associate with the model and the determination of the performance value for each sub group and also simply renders an object for each subgroup and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claims 9 and 19 simply detail the size of the graphical object for each subgroup and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claim 10 involves mental processes in the form of subjective choices of which subgroups to associate with the model, the determination of the performance value for each sub group, the determination that the generated model can be improved based on the performance value of each subgroup, and the prompt to the user given that it’s based on the subjective “performance”, and also simply renders the prompt and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claims 21 and 22 involve mental processes in the form of subjective choices of which subgroups are chosen to associate with the generated model, the identification of data corresponding to one or more models having a performance “outside” of and “expected outcomes region” (choice of the threshold or “expected outcomes region” and what is “outside” it, as well as the choices involved in setting up a correspondence between the data and the models, and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claim 23 and 24 involve mental processes in what are included in the subgroups and which products, countries, and/or verticals are selected to consist of the “set” and do(es) not provide a practical application and also do(es) not provide significantly more in that the computer system is not improved or even affected.
Claims 1-24 is(are) therefore not drawn to eligible subject matter as they are directed to an abstract idea without significantly more.
Response to Arguments
Applicant's arguments filed 2/13/26 have been fully considered but they are not persuasive.
In response to applicant’s argument (see p. 9-11 of remarks) that the claimed subject matter is not directed to an abstract idea and provide an improvement to the functioning of the computer and the field of model development, the examiner respectfully disagrees.
The examiner notes that “terminating performance of the generated model for the identified low-performing subgroup” and “removing the identified low-performing subgroup from the plurality of data subgroups” do not improve the functioning of the computer itself and “model development” is not a technical field. The examiner also notes that simply stating that the following steps ‘improve the performance of the generated model’ is insufficient to indicate an improvement to the functioning of the computer. The improvement would have to flow from the claim limitations themselves and in this case neither “generating a split-model for the identified low-performing subgroup” or “removing the identified low-performing subgroup from the plurality of data subgroups” appear improve the performance of the generated model, let alone the functioning of the computer. Simply generating a model or removing a set of data would not improve the performance of either the generated model or the computer itself. The computer’s performance is not improved, the production time, production costs, efficiencies, accuracy, speed, and waste of the program or model do not correspond to an improvement of the computer itself. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the additional elements fail to improve the functionality of the computer itself.
The examiner notes that “monitoring performance of a generated model while the generated model is being used for classification on live data received by the generated model, the live data including a plurality of data subgroups, the monitoring including determining a first performance value of the generated model at a first point in time and determining a second performance value of the generated model at a second point in time for the plurality of data subgroups associated with the generated model” involves subjective choices of how performance is monitored (observed), what factors and criteria are used to determine what is performance and a performance value, the type of performance value used (the examiner notes it could be numerical (1-10, 1-100, etc.), subjective types (critical and non-critical, high, medium, and low, clear, warning, and problem, etc.), color (red, yellow, green), or any other representation that can be thought of in the human mind, and the correspondence between monitored data and a “performance value”, the thresholds between different performance values, the factors, criteria, types, and correspondence of classifications, and includes the concepts of observation, evaluation, judgment, and opinion, and
“identifying one or more low-performing subgroups where a performance value of the low-performing subgroup is below a predetermined threshold” involves subjective choices of what constitutes “low” performing, if there a medium, average, high, severe, critical, or more types of performance with their own corresponding thresholds, and the correspondence of criteria and factors to a threshold for “low”, and includes the concepts of evaluation, judgment, and opinion. The mere nominal recitation of generic processing components does not take the claim limitation(s) out of the mental processes grouping. Thus, the claim(s) recite(s) a mental process, concepts that may be performed in the human mind, in this case being observation, evaluation, judgment, and opinion in claims 1, 11, and 20.
In response to applicant’s argument (see p. 11-12 of remarks) that the judicial exception is integrated into a practical application, the examiner respectfully disagrees.
The examiner notes that “terminating performance of the generated model for the identified low-performing subgroup” and “removing the identified low-performing subgroup from the plurality of data subgroups” do not improve the functioning of the computer itself. The examiner also notes that simply stating that the following steps ‘improve the performance of the generated model’ is insufficient to indicate an improvement to the functioning of the computer. The improvement would have to flow from the claim limitations themselves and in this case neither “generating a split-model for the identified low-performing subgroup” or “removing the identified low-performing subgroup from the plurality of data subgroups” appear improve the performance of the generated model, let alone the functioning of the computer. Simply generating a model or removing a set of data would not improve the performance of either the generated model or the computer itself. Once the model is generated or the data subgroup removed, the computer’s functionality it not changed. Going from not having a generated model to having a generated model, for instance, does not change the functioning of the computer. The computer’s performance is not improved, the production time, production costs, efficiencies, accuracy, speed, and waste of the program or model do not correspond to an improvement of the computer itself. Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the additional elements fail to improve the functionality of the computer itself. The examiner also notes that the test for an abstract idea, a practical application, and significantly more are not the same as a claim for which art was not found.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOSHUA P LOTTICH whose telephone number is (571)270-3738. The examiner can normally be reached Mon - Fri, 9:00am - 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bryce Bonzo can be reached at 5712723655. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOSHUA P LOTTICH/ Primary Examiner, Art Unit 2113