DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
The action is in response to the application filed on 6/29/23. Claims 1-20 are pending and have been examined.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/27/2023 is in compliance with the provisions of 37 CFR 1.97, 1.98, and MPEP § 609. It has been placed in the application file, and the information referred to therein has been considered as to the merits.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 14-16 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claim 14:
Claim 14 recites the limitation “The computer program product of claim 9” in line 1. However, there is no computer program product in Claim 9. For the purposes of examination on the merits of Claim 14, Claim 14 is being treated as dependent on Claim 10 instead of Claim 9.
Regarding Claims 15-16:
Claims 15-16 are rejected as being dependent on a rejected base claim without curing any of the deficiencies.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Step 1:
The claim recites a system which is one of the four statutory categories of patentable subject matter.
Step 2A prong 1:
The claim recites an abstract idea aggregate the one or more component level optimal error points which amounts to a mental process as it can be performed in a human mind.
The claim recites an additional abstract idea determine, in response to the aggregated one or more component level optimal error points, an optimal error point, wherein the optimal error point is associated with the composite artificial intelligence model which amounts to a mental process as it can be performed in a human mind.
The claim recites an additional abstract idea compare the optimal error point with the acceptance criteria which amounts to a mental process as it can be performed in a human mind.
The claim recites an additional abstract idea accept, in response to the optimal error point meeting the acceptance criteria, the optimal error point which amounts to a mental process as it can be performed in a human mind.
Step 2A prong 2:
The additional element of using a processing device is a generic computer component used to implement the abstract idea, therefore does not integrate the abstract idea into practical application MPEP 2106.05(f).
The additional element of using a non-transitory storage device containing instructions when executed by the processing device, causes the processing device to perform the steps of: is a generic computer component used to implement the abstract idea, therefore does not integrate the abstract idea into practical application MPEP 2106.05(f).
The additional element of receive, from a composite artificial intelligence model, wherein the composite artificial intelligence model comprises one or more component artificial intelligence models, one or more component level optimal error points, and wherein the one or more component level optimal error points are associated with the one or more component artificial intelligence models does not integrate the abstract idea into practical application because receiving data is considered an insignificant extra solution activity of “mere data gathering” MPEP 2106.05(g).
The additional element of receive an acceptance criteria does not integrate the abstract idea into practical application because receiving data is considered an insignificant extra solution activity of “mere data gathering” MPEP 2106.05(g).
Step 2B:
The additional element of using a processing device is a generic computer component used to implement the abstract idea, therefore does not amount to significantly more MPEP 2106.05(f).
The additional element of using a non-transitory storage device containing instructions when executed by the processing device, causes the processing device to perform the steps of: is a generic computer component used to implement the abstract idea, therefore does not amount to significantly more MPEP 2106.05(f).
The additional element of receive, from a composite artificial intelligence model, wherein the composite artificial intelligence model comprises one or more component artificial intelligence models, one or more component level optimal error points, and wherein the one or more component level optimal error points are associated with the one or more component artificial intelligence models does not amount to significantly more because the additional element is an insignificant extra solution activity and further is a well understood routine and conventional activity. See MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
The additional element of receive an acceptance criteria does not amount to significantly more because the additional element is an insignificant extra solution activity and further is a well understood routine and conventional activity. See MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Therefore, the claim is ineligible.
Regarding Claim 2:
Claim 2 which incorporates the rejection of Claim 1, recites further abstract ideas create a decision spectrum, define the acceptance criteria associated with the decision spectrum, determine, in response to the one or more received component metrics, a component score, and compare the component score with the acceptance criteria; and determine whether the component score meets the acceptance criteria which are mental processes that can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element receive one or more component metrics, wherein one or more component metrics are associated with the one or more component artificial intelligence models which is an extra solution activity of mere data gathering MPEP 2106.05(g) and further is a well understood routine and conventional activity MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)). The claim is ineligible.
Regarding Claim 3:
Claim 3 incorporates the rejection of Claim 1. The claim further recites a description of the acceptance criteria from the receive an acceptance criteria step and is ineligible for the same reasons as set forth in Claim 1. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. The claim is ineligible.
Regarding Claim 4:
Claim 4 which incorporates the rejection of Claim 1, recites a further abstract idea package, in response to the optimal error point being accepted, the optimal error point into the composite artificial intelligence model which is a mental process as it can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element implement the composite artificial intelligence model into a production environment which amounts to mere instructions to apply the abstract idea MPEP 2106.05(f). The claim is ineligible.
Regarding Claim 5:
Claim 5 incorporates the rejection of Claim 1. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element executing the instructions further causes the processing device to retrain, in response to the optimal error point being outside of the acceptance criteria, the composite artificial intelligence model which is generally linked to the abstract idea MPEP 2106.05(h). The claim is ineligible.
Regarding Claim 6:
Claim 6 which incorporates the rejection of Claim 5 recites a further abstract idea reconfiguring one or more hyperparameters associated with one or more component artificial intelligence models which is a mental process as it can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element receiving additional training data associated with the one or more component artificial intelligence models which is an extra solution activity of mere data gathering MPEP 2106.05(g) and further is a well understood routine and conventional activity MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)). The claim is ineligible.
Regarding Claim 7:
Claim 7 which incorporates the rejection of Claim 6 recites further abstract ideas comparing the historical data with the optimal error point, determining, in response to comparing the historical data and the optimal error point, one or more unoptimized component artificial intelligence models, and reconfiguring the one or more hyperparameters associated with the one or more unoptimized component artificial intelligence models which are mental processes as they can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element receiving historical data associated with the one or more component artificial intelligence models which is an extra solution activity of mere data gathering MPEP 2106.05(g) and further is a well understood routine and conventional activity MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765). The claim is ineligible.
Regarding Claim 8:
Claim 8 which incorporates the rejection of Claim 1, recites a further abstract idea the acceptance criteria is updated to an updated acceptance criteria in response to… which is a mental process as it can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element …receiving additional training data associated with the one or more component artificial intelligence models which is an extra solution activity of mere data gathering MPEP 2106.05(g) and further is a well understood routine and conventional activity MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765). The claim is ineligible. The claim is ineligible.
Regarding Claim 9:
Claim 9 which incorporates the rejection of Claim 1 recites further abstract ideas determining an amount of resources required to implement the composite artificial intelligence model and conserving one or more resources associated with the composite artificial intelligence model which are mental processes as they can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. The claim is ineligible.
Regarding Claim 10:
Step 1:
The claim recites a computer program product which is one of the four statutory categories of patentable subject matter.
Step 2A prong 1:
The claim recites an abstract idea aggregate the one or more component level optimal error points which amounts to a mental process as it can be performed in a human mind.
The claim recites an additional abstract idea determine, in response to the aggregated one or more component level optimal error points, an optimal error point, wherein the optimal error point is associated with the composite artificial intelligence model which amounts to a mental process as it can be performed in a human mind.
The claim recites an additional abstract idea compare the optimal error point with the acceptance criteria which amounts to a mental process as it can be performed in a human mind.
The claim recites an additional abstract idea accept, in response to the optimal error point meeting the acceptance criteria, the optimal error point which amounts to a mental process as it can be performed in a human mind.
Step 2A prong 2:
The additional element of using a non-transitory computer-readable medium comprising code causing an apparatus to: is a generic computer component used to implement the abstract idea, therefore does not integrate the abstract idea into practical application MPEP 2106.05(f).
The additional element of receive, from a composite artificial intelligence model, wherein the composite artificial intelligence model comprises one or more component artificial intelligence models, one or more component level optimal error points, and wherein the one or more component level optimal error points are associated with the one or more component artificial intelligence models does not integrate the abstract idea into practical application because receiving data is considered an insignificant extra solution activity of “mere data gathering” MPEP 2106.05(g).
The additional element of receive an acceptance criteria does not integrate the abstract idea into practical application because receiving data is considered an insignificant extra solution activity of “mere data gathering” MPEP 2106.05(g).
Step 2B:
The additional element of using a non-transitory computer-readable medium comprising code causing an apparatus to: is a generic computer component used to implement the abstract idea, therefore does not amount significantly more MPEP 2106.05(f).
The additional element of receive, from a composite artificial intelligence model, wherein the composite artificial intelligence model comprises one or more component artificial intelligence models, one or more component level optimal error points, and wherein the one or more component level optimal error points are associated with the one or more component artificial intelligence models does not amount to significantly more because the additional element is an insignificant extra solution activity and further is a well understood routine and conventional activity. See MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
The additional element of receive an acceptance criteria does not amount to significantly more because the additional element is an insignificant extra solution activity and further is a well understood routine and conventional activity. See MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Therefore, the claim is ineligible.
Regarding Claim 11:
Claim 11 which incorporates the rejection of Claim 10, recites further abstract ideas create a decision spectrum, define the acceptance criteria associated with the decision spectrum, determine, in response to the one or more received component metrics, a component score, and compare the component score with the acceptance criteria; and determine whether the component score meets the acceptance criteria which are mental processes that can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element receive one or more component metrics, wherein one or more component metrics are associated with the one or more component artificial intelligence models which is an extra solution activity of mere data gathering MPEP 2106.05(g) and further is a well understood routine and conventional activity MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)). The claim is ineligible.
Regarding Claim 12:
Claim 12 incorporates the rejection of Claim 10. The claim further recites a description of the acceptance criteria from the receive an acceptance criteria step and is ineligible for the same reasons as set forth in Claim 10. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. The claim is ineligible.
Regarding Claim 13:
Claim 13 which incorporates the rejection of Claim 10, recites a further abstract idea package, in response to the optimal error point being accepted, the optimal error point into the composite artificial intelligence model which is a mental process as it can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element implement the composite artificial intelligence model into a production environment which amounts to mere instructions to apply the abstract idea MPEP 2106.05(f). The claim is ineligible.
Regarding Claim 14:
Claim 14 incorporates the rejection of Claim 10. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element retrain, in response to the optimal error point being outside of the acceptance criteria, the composite artificial intelligence model which is generally linked to the abstract idea MPEP 2106.05(h). The claim is ineligible.
Regarding Claim 15:
Claim 15 which incorporates the rejection of Claim 14 recites a further abstract idea reconfiguring one or more hyperparameters associated with one or more component artificial intelligence models which is a mental process as it can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element receiving additional training data associated with the one or more component artificial intelligence models which is an extra solution activity of mere data gathering MPEP 2106.05(g) and further is a well understood routine and conventional activity MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)). The claim is ineligible.
Regarding Claim 16:
Claim 16 which incorporates the rejection of Claim 15 recites further abstract ideas comparing the historical data with the optimal error point, determining, in response to comparing the historical data and the optimal error point, one or more unoptimized component artificial intelligence models, and reconfiguring the one or more hyperparameters associated with the one or more unoptimized component artificial intelligence models which are mental processes as they can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element receiving historical data associated with the one or more component artificial intelligence models which is an extra solution activity of mere data gathering MPEP 2106.05(g) and further is a well understood routine and conventional activity MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765). The claim is ineligible.
Regarding Claim 17:
Claim 17 which incorporates the rejection of Claim 10, recites a further abstract idea the acceptance criteria is updated to an updated acceptance criteria in response to… which is a mental process as it can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element …receiving additional training data associated with the one or more component artificial intelligence models which is an extra solution activity of mere data gathering MPEP 2106.05(g) and further is a well understood routine and conventional activity MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765). The claim is ineligible. The claim is ineligible.
Regarding Claim 18:
Claim 18 which incorporates the rejection of Claim 10 recites further abstract ideas determining an amount of resources required to implement the composite artificial intelligence model and conserving one or more resources associated with the composite artificial intelligence model which are mental processes as they can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. The claim is ineligible.
Regarding Claim 19:
Step 1:
The claim recites a method which is one of the four statutory categories of patentable subject matter.
Step 2A prong 1:
The claim recites an abstract idea aggregating the one or more component level optimal error points which amounts to a mental process as it can be performed in a human mind.
The claim recites an additional abstract idea determining, in response to the aggregated one or more component level optimal error points, an optimal error point, wherein the optimal error point is associated with the composite artificial intelligence model which amounts to a mental process as it can be performed in a human mind.
The claim recites an additional abstract idea comparing the optimal error point with the acceptance criteria which amounts to a mental process as it can be performed in a human mind.
The claim recites an additional abstract idea accepting, in response to the optimal error point meeting the acceptance criteria, the optimal error point which amounts to a mental process as it can be performed in a human mind.
Step 2A prong 2:
The additional element of receiving, from a composite artificial intelligence model, wherein the composite artificial intelligence model comprises one or more component artificial intelligence models, one or more component level optimal error points, and wherein the one or more component level optimal error points are associated with the one or more component artificial intelligence models does not integrate the abstract idea into practical application because receiving data is considered an insignificant extra solution activity of “mere data gathering” MPEP 2106.05(g).
The additional element of receiving an acceptance criteria does not integrate the abstract idea into practical application because receiving data is considered an insignificant extra solution activity of “mere data gathering” MPEP 2106.05(g).
Step 2B:
The additional element of receiving, from a composite artificial intelligence model, wherein the composite artificial intelligence model comprises one or more component artificial intelligence models, one or more component level optimal error points, and wherein the one or more component level optimal error points are associated with the one or more component artificial intelligence models does not amount to significantly more because the additional element is an insignificant extra solution activity and further is a well understood routine and conventional activity. See MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
The additional element of receiving an acceptance criteria does not amount to significantly more because the additional element is an insignificant extra solution activity and further is a well understood routine and conventional activity. See MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)).
Therefore, the claim is ineligible.
Regarding Claim 20:
Claim 20 which incorporates the rejection of Claim 19, recites further abstract ideas creating a decision spectrum, define the acceptance criteria associated with the decision spectrum, determining, in response to the one or more received component metrics, a component score, and comparing the component score with the acceptance criteria; and determine whether the component score meets the acceptance criteria which are mental processes that can be performed in a human mind. The claim does not recite any additional elements that integrate the abstract idea into practical application or amount to significantly more. Specifically, the claim recites an additional element receiving one or more component metrics, wherein one or more component metrics are associated with the one or more component artificial intelligence models which is an extra solution activity of mere data gathering MPEP 2106.05(g) and further is a well understood routine and conventional activity MPEP 2106.05(d)(II)(i), (buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network)). The claim is ineligible.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Khapali et al. (US Patent Application Publication No US 20210034960 A1), hereinafter “Khapali”.
Regarding Claim 1, Khapali teaches:
A system for determining cumulative optimal global minima error for a system using composite artificial intelligence modeling (cumulative optimal global minima error is feedback data for optimal models which is determined after cycling through training and retraining until deployment see Figure 2, paragraph 6, “generate one or more feedback data sets for each model in the set of trained models… rank each model in the set of trained models based on the generated feedback data sets… initiate a retraining or deployment of one or more ranked models… automatically deploy one or more ranked models to one or more deployment environments based… on the ranking of the one or more trained models… responsive to not exceeding one or more adjusted thresholds, retrain each model in the set of trained models based… on the ranking of each trained model and corresponding hyperparameter sets”), the system comprising:
a processing device (paragraph 52, “a processor”);
a non-transitory storage device containing instructions when executed by the processing device, causes the processing device to perform the steps of (paragraph 52, “The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention”):
receive, from a composite artificial intelligence model, wherein the composite artificial intelligence model comprises one or more component artificial intelligence models, one or more component level optimal error points, and wherein the one or more component level optimal error points are associated with the one or more component artificial intelligence models (composite artificial intelligence model is set of trained models, Abstract, “generate one or more feedback data sets for each model in the set of trained models”, component level optimal error point is individual error related model data before aggregation into feedback data set, paragraph 33, “runs one or more evaluation, validation, and testing methods on the trained models 152A through 152Z… generate feedback data sets, feedback data, and feedback statistics including, but not limited to, predictive accuracy… error rates… precision, overfitting considerations, model fitness, and related system statistics”);
aggregate the one or more component level optimal error points (errors are aggregated when feedback data set is created for set of models, Abstract, “The one or more computer processors generate one or more feedback data sets for each model in the set of trained models”);
determine, in response to the aggregated one or more component level optimal error points, an optimal error point, wherein the optimal error point is associated with the composite artificial intelligence model (optimal error point is associated with top ranked model, paragraph 36, “CLP 150 utilizes one or more statistics or values contained within self-learning table 126 to arrange, rank, and/or order models”);
receive an acceptance criteria(paragraph 37, “CLP 150 utilizes a predetermined rank or score threshold, based on user, application, or model preferences, eliminating any models having a composite score or rank less than or below said threshold”);
compare the optimal error point with the acceptance criteria; and accept, in response to the optimal error point meeting the acceptance criteria, the optimal error point (model as characterized by its optimal error point is accepted, paragraph 41, “If the model feedback exceeds a model threshold (“Yes” branch, decision block 214), then CLP 150 selects and deploys the model”).
Regarding Claim 2, Khapali teaches the system of Claim 1. Khapali further teaches:
wherein comparing the optimal error point with the acceptance criteria comprises:
create a decision spectrum (ranking is decision spectrum, paragraph 37, “CLP 150 utilizes one or more statistics or values contained within self-learning table 126 to arrange, rank, and/or order models”);
define the acceptance criteria associated with the decision spectrum(paragraph 37, “CLP 150 utilizes a predetermined rank or score threshold, based on user, application, or model preferences, eliminating any models having a composite score or rank less than or below said threshold”);
receive one or more component metrics, wherein the one or more component metrics are associated with the one or more component artificial intelligence models (paragraph 21, “Self-learning table 126 contains data used by cognitive learning program 150, such as feedback data including, but not limited to… predictive accuracy… error rates… precision, overfitting considerations, model fitness, and related environment/system/server statistics”);
determine, in response to the one or more received component metrics, a component score (component score is rank, paragraph 36, “CLP 150 utilizes one or more statistics or values contained within self-learning table 126 to arrange, rank, and/or order models”);
compare the component score with the acceptance criteria; and determine whether the component score meets the acceptance criteria (rank is compared to rank threshold, paragraph 41, “If the model feedback exceeds a model threshold (“Yes” branch, decision block 214), then CLP 150 selects and deploys the model”).
Regarding Claim 3, Khapali teaches the system of Claim 1. Khapali further teaches:
wherein the acceptance criteria comprises a predetermined acceptance score associated with the composite artificial intelligence model (paragraph 37, “CLP 150 utilizes a predetermined rank or score threshold, based on user, application, or model preferences, eliminating any models having a composite score or rank less than or below said threshold”).
Regarding Claim 4, Khapali teaches the system of Claim 1. Khapali further teaches:
wherein executing the instructions further causes the processing device to:
package, in response to the optimal error point being accepted, the optimal error point into the composite artificial intelligence model (If above threshold optimal error point is part of final version of model before deployment, paragraph 41, “If the model feedback exceeds a model threshold (“Yes” branch, decision block 214), then CLP 150 selects and deploys the model”); and
implement the composite artificial intelligence model into a production environment (paragraph 24, CLP 150 stores and deploys the trained models to a plurality of environments including production).
Regarding Claim 5, Khapali teaches the system of Claim 1. Khapali further teaches:
wherein executing the instructions further causes the processing device to retrain, in response to the optimal error point being outside of the acceptance criteria, the composite artificial intelligence model (paragraph 24, “CLP 150 monitors a plurality of features contained within the feedback sets, in addition to feedback data delta. Responsive to a exceeding a feedback data threshold, CLP 150 adjusts the hyperparameters and training sets and retrains the previously trained models”).
Regarding Claim 6, Khapali teaches the system of Claim 5. Khapali further teaches:
wherein retraining the composite artificial intelligence model comprises:
reconfiguring one or more hyperparameters associated with the one or more component artificial intelligence models (paragraph 24, “CLP 150 adjusts the hyperparameters and training sets and retrains the previously trained models”); and
receiving additional training data associated with the one or more component artificial intelligence models (paragraph 30, “CLP 150 generates hyperparameter sets and associated training sets based on historical models and associated feedback data”).
Regarding Claim 7, Khapali teaches the system of Claim 6. Khapali further teaches:
wherein reconfiguring the one or more hyperparameters comprises:
receiving historical data associated with the one or more component artificial intelligence models (paragraph 24, “CLP 150 generates a plurality of feedback data sets corresponding to the plurality of trained models and stores said feedback data sets into self-learning table 126”, Fig 2 step 208);
comparing the historical data with the optimal error point (paragraph 38, “CLP 150 may utilize a plurality of models to dynamically adjust one or more thresholds that determine when to trigger or initiate CLP 150 to evaluate, train/retrain”, Fig 2 Step 212);
determining, in response to comparing the historical data and the optimal error point, one or more unoptimized component artificial intelligence models; and reconfiguring the one or more hyperparameters associated with the one or more unoptimized component artificial intelligence models (paragraph 40, “if the model feedback does not exceed a threshold (“No” branch, decision block 214), the CLP 150 returns to adjusting hyperparameters and training data (step 202)”, Fig 2 step 214 NO leading to step 202).
Regarding Claim 8, Khapali teaches the system of Claim 1. Khapali further teaches:
wherein the acceptance criteria is updated to an updated acceptance criteria in response to receiving additional training data associated with the one or more component artificial intelligence models (updated acceptance criteria is when threshold is dynamically adjusted in Figure 2 repeating steps 202-212, paragraph 24, “CLP 150 dynamically adjusts one or more thresholds associated with one or more model feedback statistics. CLP 150 monitors a plurality of features contained within the feedback sets, in addition to feedback data delta. Responsive to a exceeding a feedback data threshold, CLP 150 adjusts the hyperparameters and training sets and retrains the previously trained models. CLP 150 generates additional feedback data on the retrained models”).
Regarding Claim 9, Khapali teaches the system of Claim 1. Khapali further teaches:
wherein determining cumulative optimal global minima error for a system using composite artificial intelligence modeling further comprises:
determining an amount of resources required to implement the composite artificial intelligence model (Abstract, “retraining or deployment of one or more ranked models, based, at least in part, on one or more production environment requirements”); and
conserving one or more resources associated with the composite artificial intelligence model (paragraph 12, “system efficiency is improved by deploying a plurality of models to a plurality of environments based on model, environment, system, server requirements and statistics”).
Regarding Claim 10, Khapali teaches:
A computer program product for determining cumulative optimal global minima error for a system using composite artificial intelligence modeling (cumulative optimal global minima error is feedback data for optimal models which is determined after cycling through training and retraining until deployment see Figure 2, paragraph 6, “generate one or more feedback data sets for each model in the set of trained models… rank each model in the set of trained models based on the generated feedback data sets… initiate a retraining or deployment of one or more ranked models… automatically deploy one or more ranked models to one or more deployment environments based… on the ranking of the one or more trained models… responsive to not exceeding one or more adjusted thresholds, retrain each model in the set of trained models based… on the ranking of each trained model and corresponding hyperparameter sets”), the computer program product comprising a non-transitory computer-readable medium comprising code(paragraph 52, “The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention”) causing an apparatus to:
receive, from a composite artificial intelligence model, wherein the composite artificial intelligence model comprises one or more component artificial intelligence models, one or more component level optimal error points, and wherein the one or more component level optimal error points are associated with the one or more component artificial intelligence models (composite artificial intelligence model is set of trained models, Abstract, “generate one or more feedback data sets for each model in the set of trained models”, component level optimal error point is individual error related model data before aggregation into feedback data set, paragraph 33, “runs one or more evaluation, validation, and testing methods on the trained models 152A through 152Z… generate feedback data sets, feedback data, and feedback statistics including, but not limited to, predictive accuracy… error rates… precision, overfitting considerations, model fitness, and related system statistics”);
aggregate the one or more component level optimal error points (errors are aggregated when feedback data set is created for set of models, Abstract, “The one or more computer processors generate one or more feedback data sets for each model in the set of trained models”);
determine, in response to the aggregated one or more component level optimal error points, an optimal error point, wherein the optimal error point is associated with the composite artificial intelligence model (optimal error point is associated with top ranked model, paragraph 36, “CLP 150 utilizes one or more statistics or values contained within self-learning table 126 to arrange, rank, and/or order models”);
receive an acceptance criteria(paragraph 37, “CLP 150 utilizes a predetermined rank or score threshold, based on user, application, or model preferences, eliminating any models having a composite score or rank less than or below said threshold”);
compare the optimal error point with the acceptance criteria; and accept, in response to the optimal error point meeting the acceptance criteria, the optimal error point (model as characterized by its optimal error point is accepted, paragraph 41, “If the model feedback exceeds a model threshold (“Yes” branch, decision block 214), then CLP 150 selects and deploys the model”).
Regarding Claim 11, the rejection of 10 is incorporated and further, the claim is rejected for the
same reasons as set forth in Claim 2.
Regarding Claim 12, the rejection of 10 is incorporated and further, the claim is rejected for the
same reasons as set forth in Claim 3.
Regarding Claim 13, the rejection of 10 is incorporated and further, the claim is rejected for the
same reasons as set forth in Claim 4.
Regarding Claim 14, the rejection of 10 is incorporated and further, the claim is rejected for the
same reasons as set forth in Claim 5.
Regarding Claim 15, the rejection of 14 is incorporated and further, the claim is rejected for the
same reasons as set forth in Claim 6.
Regarding Claim 16, the rejection of 15 is incorporated and further, the claim is rejected for the
same reasons as set forth in Claim 7.
Regarding Claim 17, the rejection of 10 is incorporated and further, the claim is rejected for the
same reasons as set forth in Claim 8.
Regarding Claim 18, the rejection of 10 is incorporated and further, the claim is rejected for the
same reasons as set forth in Claim 9.
Regarding Claim 19, Khapali teaches:
A method for determining cumulative optimal global minima error for a system using composite artificial intelligence modeling (cumulative optimal global minima error is feedback data for optimal models which is determined after cycling through training and retraining until deployment see Figure 2, paragraph 6, “generate one or more feedback data sets for each model in the set of trained models… rank each model in the set of trained models based on the generated feedback data sets… initiate a retraining or deployment of one or more ranked models… automatically deploy one or more ranked models to one or more deployment environments based… on the ranking of the one or more trained models… responsive to not exceeding one or more adjusted thresholds, retrain each model in the set of trained models based… on the ranking of each trained model and corresponding hyperparameter sets”), the method comprising:
receiving, from a composite artificial intelligence model, wherein the composite artificial intelligence model comprises one or more component artificial intelligence models, one or more component level optimal error points, and wherein the one or more component level optimal error points are associated with the one or more component artificial intelligence models (composite artificial intelligence model is set of trained models, Abstract, “generate one or more feedback data sets for each model in the set of trained models”, component level optimal error point is individual error related model data before aggregation into feedback data set, paragraph 33, “runs one or more evaluation, validation, and testing methods on the trained models 152A through 152Z… generate feedback data sets, feedback data, and feedback statistics including, but not limited to, predictive accuracy… error rates… precision, overfitting considerations, model fitness, and related system statistics”);
aggregating the one or more component level optimal error points (errors are aggregated when feedback data set is created for set of models, Abstract, “The one or more computer processors generate one or more feedback data sets for each model in the set of trained models”);
determining, in response to the aggregated one or more component level optimal error points, an optimal error point, wherein the optimal error point is associated with the composite artificial intelligence model (optimal error point is associated with top ranked model, paragraph 36, “CLP 150 utilizes one or more statistics or values contained within self-learning table 126 to arrange, rank, and/or order models”);
receiving an acceptance criteria(paragraph 37, “CLP 150 utilizes a predetermined rank or score threshold, based on user, application, or model preferences, eliminating any models having a composite score or rank less than or below said threshold”);
comparing the optimal error point with the acceptance criteria; and accept, in response to the optimal error point meeting the acceptance criteria, the optimal error point (model as characterized by its optimal error point is accepted, paragraph 41, “If the model feedback exceeds a model threshold (“Yes” branch, decision block 214), then CLP 150 selects and deploys the model”).
Regarding Claim 20, the rejection of 19 is incorporated and further, the claim is rejected for the
same reasons as set forth in Claim 2.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JESSE CHEN COULSON whose telephone number is (571)272-4716. The examiner can normally be reached Monday-Friday 8:30-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JESSE C COULSON/
Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122