DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on XXXXXXXXXXXXXX has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims X are canceled.
Claims X are new.
Claims 1-20 are pending and have been examined.
This action is in reply to the papers filed on 03/11/2026 (effective filing date 04/21/2023).
Information Disclosure Statement
No Information Disclosure Statement has been filed.
Amendment
The present Office Action is based upon the original patent application filed on 04/21/2023 as modified by the amendment filed on 03/11/2026.
The information disclosure statement(s) submitted: xxxxxxxx, has/have been considered by the Examiner and made of record in the application file.
Reasons For Allowance
Prior-Art Rejection withdrawn
Claims 1-20 are allowable over the prior-art, however, these claims remain rejected under 35 USC §101 subject matter eligibility.
The closest prior art (See PTO-892, Notice of References Cited) does not teach the claimed: 1. (Currently amended) A method of managing artificial intelligence (Al) model rollouts, comprising: training, by a cloud environment, a second Al model, wherein the second Al model outputs a same Al feature as a first Al model in the cloud environment; specifying, by a model registry service in the cloud environment, a window of time in which the second Al model is to be rolled out to a plurality of tenant applications; generating, by a prediction service in the cloud environment, a first model value for the Al feature with the first Al model in the cloud environment; generating, by the prediction service in the cloud environment, a second model value for the Al feature with the second Al model in the cloud environment; transforming, by an Al feature combiner of the prediction service in the cloud environment, the first model value and the second model value into an output value for the Al feature for each of the plurality of tenant applications based on a timestamp associated with each of the plurality of tenant applications, wherein transformation of the first model value and the second model value into the output value for the Al feature for a respective tenant application of the plurality of tenant applications includes determining a first weight of the first model value and a second weight of the second model value based on comparison of a respective timestamp associated with the respective tenant application to the window of time in which the second AI model is to be rolled out to the plurality of tenant applications; and displaying, by the cloud environment, the output value for the AI feature to each of the plurality of tenant applications based on a timestamp associated with the tenant application, wherein the output value is one of an output of the first AI model, an output value of the second AI model, or a combined output value of the first AI model and the second AI model.
The closest prior-art (Odibat et al. 2024/0273396, Miller et al. 2023/0037733, Andersh et al. 2003/0100972, Zhang et al. 2019/0243753, Ma et al. 2015/0120244, Ruiz et al. 2024/0047052, XU et al. 2021/0201128, Narita 2022/0215297, NarasimhaMurthy et al. 2024/0371141, Woodford et al. 2019/0260794, Sawaf et al. 2022/0318887, Ma et al. 2019/0347113, Chen et al. 2019/0370603, Latapie et al. 2020/0364466, Alon et al. 2022/0156524) teach the features as disclosed in Non-final Rejection (12/11/2025), however, these cited references do not teach and the prior-art does not teach at least the following combination of features and/or elements:
training, by a cloud environment, a second Al model, wherein the second Al model outputs a same Al feature as a first Al model in the cloud environment; specifying, by a model registry service in the cloud environment, a window of time in which the second Al model is to be rolled out to a plurality of tenant applications; generating, by a prediction service in the cloud environment, a first model value for the Al feature with the first Al model in the cloud environment; generating, by the prediction service in the cloud environment, a second model value for the Al feature with the second Al model in the cloud environment; transforming, by an Al feature combiner of the prediction service in the cloud environment, the first model value and the second model value into an output value for the Al feature for each of the plurality of tenant applications based on a timestamp associated with each of the plurality of tenant applications, wherein transformation of the first model value and the second model value into the output value for the Al feature for a respective tenant application of the plurality of tenant applications includes determining a first weight of the first model value and a second weight of the second model value based on comparison of a respective timestamp associated with the respective tenant application to the window of time in which the second AI model is to be rolled out to the plurality of tenant applications; and displaying, by the cloud environment, the output value for the AI feature to each of the plurality of tenant applications based on a timestamp associated with the tenant application, wherein the output value is one of an output of the first AI model, an output value of the second AI model, or a combined output value of the first AI model and the second AI model.
Claim Rejections - 35 USC §101 - Withdrawn
Per Applicant’s amendments and arguments and considering new guidance in the MPEP, the rejections are withdrawn. Specifically, in Applicant’s Remarks (dated 03/14/2017, pgs. 8-11), Applicant traverses the 35 USC §101 rejections arguing that the amended claims recite new limitations that are not abstract, amount to significantly more, are directed to a practical application, etc… For example, Applicant argues….
In support of their arguments, Applicant cites to the following recent Fed. Cir. court cases (i.e., Alice Corp. v. CLS Bank Int’l, SRI Int’l, Inc. v. Cisco Systems, Inc., Ultramercial, Inc. v. Hulu, LLC, Berkheimer, Core Wireless, McRO, Enfish, Bascom, DDR, etc…).
Claim Rejections - 35 USC § 101
35 U.S.C. § 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101 as being directed to non-statutory subject matter because the claimed invention is directed to an abstract idea without significantly more. These claims recite a method, system, and computer readable medium for efficient AI feature lifecycle management through AI model updates.
Claim 1 recites [a] method of managing artificial intelligence (Al) model rollouts, comprising: training, by a cloud environment, a second Al model, wherein the second Al model outputs a same Al feature as a first Al model in the cloud environment; specifying, by a model registry service in the cloud environment, a window of time in which the second Al model is to be rolled out to a plurality of tenant applications; generating, by a prediction service in the cloud environment, a first model value for the Al feature with the first Al model in the cloud environment; generating, by the prediction service in the cloud environment, a second model value for the Al feature with the second Al model in the cloud environment; transforming, by an Al feature combiner of the prediction service in the cloud environment, the first model value and the second model value into an output value for the Al feature for each of the plurality of tenant applications based on a timestamp associated with each of the plurality of tenant applications, wherein transformation of the first model value and the second model value into the output value for the Al feature for a respective tenant application of the plurality of tenant applications includes determining a first weight of the first model value and a second weight of the second model value based on comparison of a respective timestamp associated with the respective tenant application to the window of time in which the second AI model is to be rolled out to the plurality of tenant applications; and displaying, by the cloud environment, the output value for the AI feature to each of the plurality of tenant applications based on a timestamp associated with the tenant application, wherein the output value is one of an output of the first AI model, an output value of the second AI model, or a combined output value of the first AI model and the second AI model.
The claims are being rejected according to the 2019 Revised Patent Subject Matter Eligibility Guidance (Federal Register, Vol. 84, No. 5, p. 50-57 (Jan. 7, 2019)).
Step 1: Does the Claim Fall within a Statutory Category?
Yes. Claims 1-10 recite a method and, therefore, are directed to the statutory class of a process. Claims 11-19 recite a system/apparatus and, therefore, are directed to the statutory class of machine. Claim 20 recites a non-transitory computer readable medium/computer product and, therefore, are directed to the statutory class of a manufacture.
Step 2A, Prong One: Is a Judicial Exception Recited?
Yes. The following tables identify the specific limitations that recite an abstract idea. The column that identifies the additional elements will be relevant to the analysis in step 2A, prong two, and step 2B.
Claim 1: Identification of Abstract Idea and Additional Elements, using Broadest Reasonable Interpretation
Claim Limitation
Abstract Idea
Additional Element
1. A method of managing artificial intelligence (AI) model rollouts, comprising:
No additional elements are positively claimed.
training, by a cloud environment, a second AI model, wherein the second AI model outputs a same AI feature as a first AI model in the cloud environment;
This limitation includes the step(s) of: training, by a cloud environment, a second AI model, wherein the second AI model outputs a same AI feature as a first AI model in the cloud environment.
No additional elements are positively claimed.
This limitation is directed to processing and/or communicating known information (e.g., training a model) to facilitate efficient AI feature lifecycle management through AI model updates which may be categorized as any of the following:
mathematical concept (mathematical relationships, mathematical formulas or equations, mathematical calculations)
“training … a second AI model … wherein the second AI model outputs a same AI feature as a first AI model…” (model training is mathematical computation, see PEG Abstract Idea Grouping 1).
and/or
mental process – concepts performed in the human mind (including an observation, evaluation, judgment, opinion)
and/or
certain method of organizing human activity –
fundamental economic principles or practices (including hedging, insurance, mitigating risk), and/or
commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations).
No additional elements are positively claimed.
“a cloud environment” and “AI model” are interpreted as purely software.
specifying, by a model registry service in the cloud environment, a window of time in which the second Al model is to be rolled out to a plurality of tenant applications;
This limitation includes the step(s) of: specifying, by a model registry service in the cloud environment, a window of time in which the second Al model is to be rolled out to a plurality of tenant applications.
No additional elements are positively claimed.
This limitation is directed to processing and/or communicating known information to facilitate efficient AI feature lifecycle management through AI model updates which may be categorized as any of the following:
mathematical concept (mathematical relationships, mathematical formulas or equations, mathematical calculations)
and/or
mental process – concepts performed in the human mind (including an observation, evaluation, judgment, opinion)
and/or
certain method of organizing human activity –
fundamental economic principles or practices (including hedging, insurance, mitigating risk), and/or
commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations).
“specifying … a window of time in which the second AI model is to be rolled out to a plurality of tenant applications;” (rollout scheduling across tenants is a managerial practice).
No additional elements are positively claimed.
“a model registry service in the cloud environment” and “AI model” are interpreted as purely software.
generating, by a prediction service in the cloud environment, a first model value for the Al feature with the first Al model in the cloud environment;
This limitation includes the step(s) of: generating, by a prediction service in the cloud environment, a first model value for the Al feature with the first Al model in the cloud environment.
No additional elements are positively claimed.
This limitation is directed to processing and/or communicating known information to facilitate efficient AI feature lifecycle management through AI model updates which may be categorized as any of the following:
mathematical concept (mathematical relationships, mathematical formulas or equations, mathematical calculations)
and/or
mental process – concepts performed in the human mind (including an observation, evaluation, judgment, opinion)
and/or
certain method of organizing human activity –
fundamental economic principles or practices (including hedging, insurance, mitigating risk), and/or
commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations).
No additional elements are positively claimed.
“a prediction service in the cloud environment” and “AI model” are interpreted as purely software.
generating, by the prediction service in the cloud environment, a second model value for the Al feature with the second Al model in the cloud environment;
This limitation includes the step(s) of: generating, by the prediction service in the cloud environment, a second model value for the Al feature with the second Al model in the cloud environment.
No additional elements are positively claimed.
This limitation is directed to processing and/or communicating known information to facilitate efficient AI feature lifecycle management through AI model updates which may be categorized as any of the following:
mathematical concept (mathematical relationships, mathematical formulas or equations, mathematical calculations)
and/or
mental process – concepts performed in the human mind (including an observation, evaluation, judgment, opinion)
and/or
certain method of organizing human activity –
fundamental economic principles or practices (including hedging, insurance, mitigating risk), and/or
commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations).
No additional elements are positively claimed.
“the prediction service in the cloud environment” and “AI model” are interpreted as purely software.
transforming, by an Al feature combiner of the prediction service in the cloud environment, the first model value and the second model value into an output value for the Al feature for each of the plurality of tenant applications based on a timestamp associated with each of the plurality of tenant applications, wherein transformation of the first model value and the second model value into the output value for the Al feature for a respective tenant application of the plurality of tenant applications includes determining a first weight of the first model value and a second weight of the second model value based on comparison of a respective timestamp associated with the respective tenant application to the window of time in which the second AI model is to be rolled out to the plurality of tenant applications; and
This limitation includes the step(s) of: transforming, by an Al feature combiner of the prediction service in the cloud environment, the first model value and the second model value into an output value for the Al feature for each of the plurality of tenant applications based on a timestamp associated with each of the plurality of tenant applications, wherein transformation of the first model value and the second model value into the output value for the Al feature for a respective tenant application of the plurality of tenant applications includes determining a first weight of the first model value and a second weight of the second model value based on comparison of a respective timestamp associated with the respective tenant application to the window of time in which the second AI model is to be rolled out to the plurality of tenant applications.
No additional elements are positively claimed.
This limitation is directed to processing and/or communicating known information (e.g., displaying information) to facilitate efficient AI feature lifecycle management through AI model updates which may be categorized as any of the following:
mathematical concept (mathematical relationships, mathematical formulas or equations, mathematical calculations)
and/or
mental process – concepts performed in the human mind (including an observation, evaluation, judgment, opinion)
and/or
certain method of organizing human activity –
fundamental economic principles or practices (including hedging, insurance, mitigating risk), and/or
commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations).
No additional elements are positively claimed.
“an Al feature combiner of the prediction service in the cloud environment” and “AI model” are interpreted as purely software.
displaying, by the cloud environment, the output value for the AI feature to each of the plurality of tenant applications based on a timestamp associated with the tenant application, wherein the output value is one of an output of the first AI model, an output value of the second AI model, or a combined output value of the first AI model and the second AI model.
This limitation includes the step(s) of: displaying, by the cloud environment, the output value for the AI feature to each of the plurality of tenant applications based on a timestamp associated with the tenant application, wherein the output value is one of an output of the first AI model, an output value of the second AI model, or a combined output value of the first AI model and the second AI model.
No additional elements are positively claimed.
This limitation is directed to processing and/or communicating known information (e.g., displaying information) to facilitate efficient AI feature lifecycle management through AI model updates which may be categorized as any of the following:
mathematical concept (mathematical relationships, mathematical formulas or equations, mathematical calculations)
“displaying … an output value … wherein the output value is one of an output … of the first AI model, an output value of the second AI model, or a combined output value of the first AI model and the second AI model.” The “combined output value” reflects mathematical combination/ensembling. See SAP Am., Inc. v. InvestPic, LLC, 898 F.3d 1161 (Fed. Cir. 2018); In re Board of Trustees of Leland Stanford Junior Univ., 991 F.3d 1245 (Fed. Cir. 2021).
and/or
mental process – concepts performed in the human mind (including an observation, evaluation, judgment, opinion)
“displaying … an output value … based on a timestamp … wherein the output value is one of [first model], [second model], or [combined].” Selecting among outputs based on time is, at a conceptual level, a conditional evaluation/decision.
See PEG Abstract Idea Grouping 3; Elec. Power Grp., LLC v. Alstom S.A., 830 F.3d 1350 (Fed. Cir. 2016).
and/or
certain method of organizing human activity –
fundamental economic principles or practices (including hedging, insurance, mitigating risk), and/or
commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations).
“displaying … an output value … to each of the plurality of tenant applications based on a timestamp associated with the tenant application…” (coordinating phased rollout/feature gating across applications is a business/operational practice). See PEG Abstract Idea Grouping 2; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359 (Fed. Cir. 2015); Intellectual Ventures I LLC v. Capital One Bank, 792 F.3d 1363 (Fed. Cir. 2015).
No additional elements are positively claimed.
“the cloud environment” and “AI model” are interpreted as purely software.
As shown above, under Step 2A, Prong One, the claims recite a judicial exception (an abstract idea). The claims are directed to the abstract idea of efficient AI feature lifecycle management through AI model updates, which, pursuant to MPEP 2106.04, is aptly categorized as a mathematical concept, mental process and/or a method of organizing human activity. Therefore, under Step 2A, Prong One, the claims recite a judicial exception.
The method claims do NOT recite any additional elements. Consequently, at least the method claims must be construed as abstract and capable of being performed mentally and/or manually with just pen and paper. The Office encourages Applicant to positively claim the structural features necessary to perform each individual method step and feature.
Next, the claims recite additional functional elements that are associated with the judicial exception, including: a processor and memory (system and CRM claims) for storing instructions. Examiner understands these limitations to be insignificant extrasolution activity. (See Accenture, 728 F.3d 1336, 108 U.S.P.Q.2d 1173 (Fed. Cir. 2013), citing Cf. Diamond v. Diehr, 450 U.S. 175, 191-192 (1981) ("[I]nsignificant post-solution activity will not transform an unpatentable principle in to a patentable process.”).
The aforementioned claims also recite additional technical elements including: a processor and memory to execute the system and, a non-transitory computer-readable medium for storing executable instructions. These limitations are recited at a high level of generality and appear to be nothing more than generic computer components. Claims that amount to nothing more than an instruction to apply the abstract idea using a generic computer do not render an abstract idea eligible. Alice Corp., 134 S. Ct. at 2358, 110 USPQ2d at 1983. See also 134 S. Ct. at 2389, 110 USPQ2d at 1984.
Step 2A, Prong Two: Is the Abstract Idea Integrated into a Practical Application?
No. The judicial exception is not integrated into a practical application. The additional elements listed above that relate to computing components are recited at a high level of generality (i.e., as generic components performing generic computer functions such as communicating, receiving, processing, analyzing, and outputting/displaying data) such that they amount to no more than mere instructions to apply the exception using generic computing components. Simply implementing the abstract idea on a generic computer is not a practical application of the abstract idea. Additionally, the claims do not purport to improve the functioning of the computer itself. There is no technological problem that the claimed invention solves. Rather, the computer system is invoked merely as a tool. Accordingly, the additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, these claims are directed to an abstract idea.
Furthermore, looking at the elements individually and in combination, under Step 2A, Prong Two, the claims as a whole do not integrate the judicial exception into a practical application because they fail to: improve the functioning of a computer or a technical field, apply the judicial exception in the treatment or prophylaxis of a disease, apply the judicial exception with a particular machine, effect a transformation or reduction of a particular article to a different state or thing, or apply the judicial exception beyond generally linking the use of the judicial exception to a particular technological environment. Rather, the claims merely use a computer as a tool to perform the abstract idea(s), and/or add insignificant extra-solution activity to the judicial exception, and/or generally link the use of the judicial exception to a particular technological environment.
Step 2B: Does the Claim Provide an Inventive Concept?
Next, under Step 2B, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, when considered both individually and as an ordered combination, do not amount to significantly more than the abstract idea. Furthermore, looking at the limitations as an ordered combination adds nothing that is not already present when looking at the elements taken individually. Simply put, as noted above, there is no indication that the combination of elements improves the functioning of a computer (or any other technology), and their collective functions merely provide conventional computer implementation. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements relating to computing components amount to no more than applying the exception using a generic computing components. Mere instructions to apply an exception using a generic computing component cannot provide an inventive concept. Furthermore, the broadest reasonable interpretation of the claimed computer components (i.e., additional elements) includes any generic computing components that are capable of being programmed to communicate, receive, send, process, analyze, output, or display data. Furthermore, Applicant’s Specification (PGPub. 2024/0354600 [0098 - one or more general-purpose processors such as a microprocessor, a central processing unit (CPU)]) refers to a general computer system, but they do not include any technically-specific computer algorithm or code.
Additionally, pursuant to the requirement under Berkheimer, the following citations are provided to demonstrate that the additional elements, identified as extra-solution activity, amount to activities that are well-understood, routine, and conventional. See MPEP 2106.05(d).
Capturing an image (code) with an RFID reader. Ritter, US Patent No. 7734507 (Col. 3, Lines 56-67); “RFID: Riding on the Chip” by Pat Russo. Frozen Food Age. New York: Dec. 2003, vol. 52, Issue 5; page S22.
Receiving or transmitting data over a network. Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362; OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014).
Storing and retrieving information in memory. Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93.
Outputting/Presenting data to a user. Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015); MPEP 2106.05(g)(3).
Using a machine learning model to determine user segment characteristics for an ad campaign. https://whites.agency/blog/how-to-use-machine-learning-for-customer-segmentation/.
Thus, taken alone and in combination, the additional elements do not amount to significantly more than the above-identified judicial exception (the abstract idea), and are ineligible under 35 USC 101.
Independent system claim 11 and CRM claim 20 also contains the identified abstract ideas, with the additional elements of a processor and storage medium, which are a generic computer components, and thus not significantly more for the same reasons and rationale above.
Dependent claims 2-10 and 12-19 further describe the abstract idea. The additional elements of the dependent claims fail to integrate the abstract idea into a practical application and do not amount to significantly more than the abstract idea. Thus, as the dependent claims remain directed to a judicial exception, and as the additional elements of the claims do not amount to significantly more, the dependent claims are not patent eligible.
As such, the claims are not patent eligible.
Invention Could be Performed Manually
It is conceivable that the invention could be performed manually without the aid of machine and/or computer. For example, Applicant claims training a model, specifying a window of time, displaying a value. Each of these features could be performed manually and/or with the aid of a simple generic computer to facilitate the transmission of data.
See also Leapfrog Enterprises, Inc. v. Fisher-Price, Inc., and In re Venner, which stand for the concept that automating manual activity and/or applying modern electronics to older mechanical devices to accomplish the same result is not sufficient to distinguish over the prior art. Here, applicant is merely claiming computers to facilitate and/or automate functions which used to be commonly performed by a human.
Leapfrog Enterprises, Inc. v. Fisher-Price, Inc., 485 F.3d 1157, 82 USPQ2d 1687 (Fed. Cir. 2007) "[a]pplying modern electronics to older mechanical devices has been commonplace in recent years…"). The combination is thus the adaptation of an old idea or invention using newer technology that is commonly available and understood in the art.
In In re Venner, 262 F.2d 91, 95, 120 USPQ 193, 194 (CCPA 1958), the court held that broadly providing an automatic or mechanical means to replace manual activity which accomplished the same result is not sufficient to distinguish over the prior art. MPEP 2144.04, III Automating a Manual Activity.
MPEP 2144.04 III - Automating a Manual Activity and In re Venner, 262 F.2d 91, 95, 120 USPQ 193, 194 (CCPA 1958) further stand for and provide motivation for using technology, hardware, computer, or server to automate a manual activity.
Therefore, the Office finds no improvements to another technology or field, no improvements to the function of the computer itself, and no meaningful limitations beyond generally linking the use of an abstract idea to a particular technological environment. Therefore, based on the two-part Alice Corp. analysis, there are no limitations in any of the claims that transform the exception (i.e., the abstract idea) into a patent eligible application.
Claim Rejections - Not an Ordered Combination
None of the limitations, considered as an ordered combination provide eligibility, because taken as a whole, the claims simply instruct the practitioner to implement the abstract idea with routine, conventional activity.
Claim Rejections - Preemption
Allowing the claims, as presently claimed, would preempt others from implementing efficient AI feature lifecycle management through AI model updates. Furthermore, the claim language only recites the abstract idea of performing this method, there are no concrete steps articulating a particular way in which this idea is being implemented or describing how it is being performed.
Examiner’s Response to Arguments
Per Applicants’ amendments/arguments, the rejections are withdrawn.
Applicant's arguments have been considered but are moot in view of the new ground(s) of rejection.
Applicants’ amendments have necessitated the new grounds of rejection noted above.
Examiner’s Response: Claim Rejections – 35 USC § 103
Per Applicants’ amendments/arguments, the rejections are withdrawn. See notes above for additional reasoning and rationale for dropping prior-art rejection including Applicant’s amendments and arguments and unique combination of features and elements not taught by the prior-art without hindsight reasoning.
Applicant's arguments have been considered but are moot in view of the new ground(s) of rejection.
Applicants’ amendments have necessitated the new grounds of rejection noted above.
Regarding Claim X, on page(s) 8-9 of Applicant’s Remarks / After Final Amendments (dated 07/15/2011), Applicant(s) argues that the cited reference(s) (Ellis and Vandermolen) fails to teach, describe, or suggest the amended features. Specifically, Applicant(s) argues that cited reference(s) do not teach, describe, or suggest the following: . With respect, Applicant’s arguments are deemed unpersuasive and the amended feature(s) remain rejected as follows.
With respect, Applicant’s arguments are deemed unpersuasive and the amended feature(s) remain rejected as follows.
Examiner’s Response: Claim Rejections – 35 USC §112
Per Applicants’ amendments/arguments, the rejections are withdrawn.
Applicant's arguments have been considered but are moot in view of the new ground(s) of rejection.
Applicants’ amendments have necessitated the new grounds of rejection noted above.
Examiner’s Response: Claim Rejections – 35 USC §101
Per Applicants’ amendments/arguments, the rejections are withdrawn. See notes above for additional reasoning and rationale for dropping 35 USC 101 rejection including Applicant’s amendments, arguments, lack of abstract idea, and practical integration.
Applicant's arguments have been considered but are moot in view of the new ground(s) of rejection.
Applicants’ amendments have necessitated the new grounds of rejection noted above.
Regarding Claims 1-20, on page(s) 9-11 of Applicant’s Remarks (dated 03/11/2026), Applicants traverse the 35 USC §101 rejections arguing the following: the claims are not abstract and integrated into a practical application and the invention constitutes significantly more than any alleged abstract idea. Applicant argues that the claims do not constitute mental processes as the steps cannot be practically performed in the human mind. Further, Applicant argues that the claims do not recite certain methods of organizing human activity. Applicant argues that the claims integrate any abstract idea into a practical application that improves the functioning of a computer by reducing a gap between output values of different version of AI models during a rollout and enabling user level customized rollout of new models.
With respect, the Office maintains the 35 USC §101 rejection as noted in the rejection above and further detailed herein.
It is clear from the Specification and the claim language that claim 1 focuses on an abstract idea, and not on an improvement to technology and/or a technical field.
The Specification is titled “Method for Efficient AI Feature Lifecycle Management Through AI Model Updates” and observes, in the Background section that “there is a need to smooth the disruptive user experiences caused by AI model rollouts” (Spec. [0002]). Applicant argues that that the claims do not constitute mental processes as the steps cannot be practically performed in the human mind. The claims indicate training a model by a cloud environment, however, there is no indication that a human could not also train a model by running multiple scenarios. Training a model is not inherently something that can only be accomplished by a computer. Furthermore, the other claimed features could also be performed by a human. For example, specifying a window of time in which an AI model is to be rolled out is something a human could reasonable be capable of and is not inherent to a computer. Likewise, generating a second model value is also capable of being performed by a human. Similar arguments may be made for transforming a first and second model value into an output value and displaying the output value. The rejection above, additionally lays out rationale for mathematical concepts and organizing human activities.
Step 2A, Prong 2: Analyze integration into a practical application; discuss any claimed technological improvement; address whether extra-solution activity or field-of-use limitations are present.
The claims do not integrate the abstract ideas into a practical application (MPEP 2106.04(d)(2); 2106.05(a)–(h)).
No improvement to computer functionality or any other technology is recited. The “cloud environment,” “data processing system,” “processor,” and “memory” are generic computing components performing their ordinary functions. The operations are framed at a results-oriented level: train a model; specify a time window; display an output selected from first/second/combined based on a timestamp. There is no specific computer architecture, model architecture, training algorithm, rollout mechanism (e.g., non-blocking swap, consistent hashing, transaction protocols), data structures, or protocols claimed that improve system performance or computer functioning (contrast Enfish, LLC v. Microsoft Corp., 822 F.3d 1327 (Fed. Cir. 2016); McRO, Inc. v. Bandai Namco, 837 F.3d 1299 (Fed. Cir. 2016); DDR Holdings, LLC v. Hotels.com, 773 F.3d 1245 (Fed. Cir. 2014)).
No particular machine is meaningfully tied to the exception beyond generic hardware (MPEP 2106.05(b)), and no transformation of an article to a different state or thing is present (MPEP 2106.05(c)).
The “cloud environment” and “tenant applications” are field-of-use limitations that merely situate the abstract rollout-management idea in a cloud/multi-tenant context without imposing meaningful technological restrictions (MPEP 2106.05(h); In re TLI Commc’ns LLC, 823 F.3d 607 (Fed. Cir. 2016)).
Steps like “displaying … an output value … to each … tenant application” constitute post-solution presentation of information and are considered insignificant extra-solution activity (MPEP 2106.05(g); Elec. Power Group).
Accordingly, the claims do not integrate the recited judicial exceptions into a practical application.
Step 2B: Assess whether additional elements are significantly more; discuss WURC with evidentiary considerations.
The additional claim elements, individually and in ordered combination, do not amount to “significantly more” than the abstract ideas (no inventive concept) (MPEP 2106.05(d)).
Generic computing components (processor, memory, cloud environment) performing well-understood, routine, conventional functions (training models, scheduling/rollout window specification, conditional selection/display of outputs to applications) are WURC. See buySAFE, Inc. v. Google, Inc., 765 F.3d 1350 (Fed. Cir. 2014); Elec. Power Group; Customedia Techs., LLC v. Dish Network Corp., 951 F.3d 1359 (Fed. Cir. 2020).
Invoking model “training” and “combined output value” at a high level does not supply an inventive concept. Courts have held that applying sophisticated mathematics/modeling to data to generate results is abstract and, absent a specific non-conventional implementation, does not provide significantly more. See SAP v. InvestPic; Stanford.
The ordered combination—train a second model, define a rollout window, and display outputs selected by timestamp (first vs second vs combined)—is a conventional A/B testing or phased rollout/feature-gating workflow in multi-tenant systems. Absent a particular non-conventional architectural arrangement (e.g., BASCOM’s distributed filtering placement), such coordination is insufficient. See BSG Tech LLC v. BuySeasons, Inc., 899 F.3d 1281 (Fed. Cir. 2018); Two-Way Media Ltd. v. Comcast Cable Commc’ns, LLC, 874 F.3d 1329 (Fed. Cir. 2017) (result-oriented network operations not enough).
Examiner evidentiary note (Berkheimer v. HP Inc., 881 F.3d 1360 (Fed. Cir. 2018)): If asserting WURC, support with citation to authoritative sources (e.g., textbooks/industry documentation on conventional multi-tenant rollout/feature-flagging, standard ML training pipelines, timestamp-based gating) or take official notice of the conventional nature of generic processors/memory/cloud execution and conditional display logic, subject to applicant’s rebuttal.
Conclusion: Eligible/ineligible under § 101.
Claims 1, 11, and 20 are ineligible under § 101. Each recites abstract ideas (mathematical concepts and organizing human activity with mental-process aspects), fails to integrate those ideas into a practical application, and lacks additional elements that amount to “significantly more.”
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
PERTINENT PRIOR ART – Patent Literature
The prior-art made of record and considered pertinent to applicant's disclosure.
Narita 2022/0215297 [0011 - the process includes specifying first distribution of a feature of data calculated by a second machine learning model…]
NarasimhaMurthy et al. 2024/0371141 [0065 - second AI model 130B is trained using at least some of the data used to train the first AI model 130A so the second AI model 130B is capable of characterizing images on which the first AI model 130A is trained. … stored in the memory 124B, the cloud]
Woodford et al. 2019/0260794 [0017 - one or more machine learning models such as a first Artificial Intelligence model trained on cloud events, administration, and related data, a second Artificial Intelligence model trained on SaaS events for a specific SaaS application, a third Artificial Intelligence model trained on potential cyber threats]
Sawaf et al. 2022/0318887 [0023 - an efficient way to train AI models in a hybrid cloud environment by providing a high-level API for training and automated tuning of AI models by efficiently running training jobs in the cloud]
Ma et al. 2019/0347113 [0037 - applications launched within the time-window with the preset length from the time point at which the sample usage information is collected as the start time point are monitored, and the preset number of applications launched first in the applications launched within the time-window are set as the sample labels for the sample usage information. Then the predetermined machine learning model is trained based on the sample usage information and the sample labels for the sample usage information, to obtain the application prediction model]
Chen et al. 2019/0370603 [0117 - determines a first application running in foreground at a sampling time in a preset sampling period, determines whether the first application is newly installed within a time window having a preset time length and ending with the sampling time, and uses the result of the determining as identity information of the first application and then trains a preset machine learning model based on sample data corresponding to the first application]
Latapie et al. 2020/0364466 [0046 - machine learning systems discussed below can be used to determine which timescales have commonalities and allow clustering and grouping timestamps in timescales which display periodicity]
Alon et al. 2022/0156524 [0004 - combining the first model output and the second model output to generate a combined output]
PERTINENT PRIOR ART – Non-Patent Literature (NPL)
The NPL prior-art made of record and considered pertinent to applicant's disclosure.
Domenico Cotroneo, Luigi De Simone, Pietro Liguori, Roberto Natella, Run-time failure detection via non-intrusive event analysis in a large-scale cloud computing platform, Journal of Systems and Software, Volume 198, 2023.
Ning, Zhen-Hu, Shen, Chang-Xiang, Zhao, Yong, Liang, Peng, Trusted Measurement Model Based on Multitenant Behaviors, The Scientific World Journal, 2014, 384967, 12 pages, 2014.
Conclusion
THIS ACTION IS MADE FINAL
Applicant’s amendment necessitated new grounds of rejection and FINAL Rejection.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW T. SITTNER whose telephone number is (571) 270-7137 and email: matthew.sittner@uspto.gov. The examiner can normally be reached on Monday-Friday, 8:00am - 5:00pm (Mountain Time Zone). Please schedule interview requests via email: matthew.sittner@uspto.gov
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sarah M. Monfeldt can be reached on (571) 270-1833.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW T SITTNER/
Primary Examiner, Art Unit 3629b
Claims 1, 11, 20 are rejected under 35 U.S.C. 103 as being unpatentable over: Odibat et al. 2024/0273396; in view of Miller et al. 2023/0037733; in further view of Andersh et al. 2003/0100972.
18/304,710 – Claim 1. Odibat et al. 2024/0273396 A method of managing artificial intelligence (AI) model rollouts (Odibat et al. 2024/0273396 [0001 - management of machine learning (ML) models in a multi-tenant cloud computing environment … managing core capabilities of continuous integration and continuous deployment (CI/CD) in artificial intelligence (AI) model development environments] Aspects of the present invention relate generally to lifecycle management of machine learning (ML) models in a multi-tenant cloud computing environment, and, more particularly, to a system and method for managing core capabilities of continuous integration and continuous deployment (CI/CD) in artificial intelligence (AI) model development environments, champion-challenger testing, model versioning, and storage/rollback of AI/ML models.), comprising: training, by a cloud environment (Odibat et al. 2024/0273396 [0001 - multi-tenant cloud computing environment] Aspects of the present invention relate generally to lifecycle management of machine learning (ML) models in a multi-tenant cloud computing environment, and, more particularly, to a system and method for managing core capabilities of continuous integration and continuous deployment (CI/CD) in artificial intelligence (AI) model development environments, champion-challenger testing, model versioning, and storage/rollback of AI/ML models.), a second AI model, wherein the second AI model outputs a same AI feature as a first AI model in the cloud environment (Odibat et al. 2024/0273396 [0025 - multi-tenancy cloud environments] Implementations of various embodiments may provide an improvement in the technical field of multi-tenancy cloud environments. In particular, the cloud environment includes operational ML models that may be used for multiple tenants in the course of their business practice to forecast cloud usage. Reusing an operational ML model and/or generating multiple versions of the same base model improves efficiency of the cloud environment in terms of storage and speed. An ML model, also referred to herein as “model Ops,” or “MLOps,” may be used to monitor and refresh the operational models that a tenant uses to forecast cloud usage in the conduct of their business. As elements of the tenant's business evolve over time, operational ML models may become inaccurate and/or obsolete. The MLOps framework, as described herein, enables the cloud environments and cloud servers to act more efficiently for multiple tenants. Accordingly, the MLOps provides an improvement to cloud server processor throughput/load, storage requirements for each model, etc. Further, the MLOps may keep the tenants' models up-to-date and accurate for business purposes. [0027 – training and using a machine learning model… using the trained model to generate an output in real time] Implementations of the present invention are necessarily rooted in computer technology. For example, the operations of the claims that recite use of the ML models for DMS and DCMM are computer-based and cannot be performed in the human mind. Training and using a machine learning model are, by definition, performed by a computer and cannot practically be performed as a mental process (or with pen and paper) due to the complexity and massive amounts of calculations involved. For example, an artificial neural network (ANN) may have millions or even billions of weights that represent connections among nodes in one or more layers of the model. Values of these weights are adjusted, e.g., via backpropagation and stochastic gradient descent, when training the model and are utilized in calculations when using the trained model to generate an output in real time (or near real time). Given this scale and complexity, it is simply not possible for the human mind, or for a person using pen and paper, to perform the number of calculations involved in training and/or using a machine learning model. For purpose of brevity, the term “neural network” may be used herein as an equivalent to the more accurate term of “artificial neural network” to indicate a computer-based artificial intelligence methodology. [0063 - communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; one or more devices that enable a user to interact with computer system/server 12; and/or any devices (e.g., network card, modem, etc.) that enable computer system/server 12 to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces] Computer system/server 12 may also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; one or more devices that enable a user to interact with computer system/server 12; and/or any devices (e.g., network card, modem, etc.) that enable computer system/server 12 to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces 22. Still yet, computer system/server 12 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via network adapter 20. As depicted, network adapter 20 communicates with the other components of computer system/server 12 via bus 18. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with computer system/server 12. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.[0097 - Outputs of the deep learning neural network 1021 include a model performance threshold … ] In an embodiment, service level agreement (SLA) 1011 may include time constraints in units of seconds. Resource constraints may include data for scalability, regulatory, maintainability, serviceability, security, and cost. Cost data may contain AssetId, asset type, unit cost, total cost, region, data center, period, and description. Outputs of the deep learning neural network 1021 include a model performance threshold 1030 and model refresh frequency 1040. Two different loss functions may be used to learn the outputs. As discussed above, softmax is used to determine the model refresh frequency 1040, and linear regression is used to identify the model performance threshold 1030. In particular, softmax is a generalization of the logistic function to multiple dimensions, and used in multinomial logistic regression, and may be used to normalize output of the neural network 1021. The deep learning neural network uses a Leaky Rectified Linear Unit (Leaky ReLU) function across the hidden layers. Leaky ReLU is a type of activation function based on a ReLU, but it has a small slope for negative values instead of a flat slope. The slope coefficient may be determined before training, i.e., the slope coefficient may not be learned during training. Input is passed to deep hidden layers where weights have been shared among each other for both outputs. After backpropagation and forward propagation models are used to reduce the loss and return the model refresh frequency 1040 and model performance threshold 1030 for the tenant.); specifying, by the cloud environment, a window of time in which the second AI model is to be rolled out to a plurality of tenant applications (Odibat et al. 2024/0273396 [0026 - cloud environment ML model variants, and deployment of the ML model variants to tenants] Implementations of various embodiments may provide an improvement in the technical field of identifying and refreshing ML models used by customers of a cloud environment. In particular, implementations use machine learning to train an AI-based dynamic model selector (DMS) to automatically identify the best model for each tenant at runtime, with either minimal or no human intervention. Implementations may transform an article to a different state or thing, for instance, modifying/updating tenant profiles and cloud environment ML model variants, and deployment of the ML model variants to tenants. In particular, implementations may cause a secondary device to perform a requested action and update stored context and state information related to the machine learning functions. Historical information may be dynamically updated along with configuration profiles for the tenants. In the present disclosure, by automating the machine learning cost forecasting models for cloud usage, time and cost savings may be provided to tenants. In known systems, a data scientist was needed to update the tenants' forecasting models. In contrast, in embodiments, by automating this step using machine learning, the tenant may perform cost optimization, determine whether to switch cloud service providers, or switch a usage plan with the same provider. For example, a tenant may spend $1 million per month on their cloud services. A quick cost forecast that identifies a near future increase to $2 million per month may allow the tenant to avoid future increases earlier. Accordingly, the tenant may save money.); and displaying, by the cloud environment, an output value for the AI feature to each of the plurality of tenant applications based on a timestamp associated with the tenant application (Odibat et al. 2024/0273396 [0063] Computer system/server 12 may also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, etc.; one or more devices that enable a user to interact with computer system/server 12; and/or any devices (e.g., network card, modem, etc.) that enable computer system/server 12 to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces 22. Still yet, computer system/server 12 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and/or a public network (e.g., the Internet) via network adapter 20. As depicted, network adapter 20 communicates with the other components of computer system/server 12 via bus 18. It should be understood that although not shown, other hardware and/or software components could be used in conjunction with computer system/server 12. Examples, include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc. [0097] In an embodiment, service level agreement (SLA) 1011 may include time constraints in units of seconds. Resource constraints may include data for scalability, regulatory, maintainability, serviceability, security, and cost. Cost data may contain AssetId, asset type, unit cost, total cost, region, data center, period, and description. Outputs of the deep learning neural network 1021 include a model performance threshold 1030 and model refresh frequency 1040. Two different loss functions may be used to learn the outputs. As discussed above, softmax is used to determine the model refresh frequency 1040, and linear regression is used to identify the model performance threshold 1030. In particular, softmax is a generalization of the logistic function to multiple dimensions, and used in multinomial logistic regression, and may be used to normalize output of the neural network 1021. The deep learning neural network uses a Leaky Rectified Linear Unit (Leaky ReLU) function across the hidden layers. Leaky ReLU is a type of activation function based on a ReLU, but it has a small slope for negative values instead of a flat slope. The slope coefficient may be determined before training, i.e., the slope coefficient may not be learned during training. Input is passed to deep hidden layers where weights have been shared among each other for both outputs. After backpropagation and forward propagation models are used to reduce the loss and return the model refresh frequency 1040 and model performance threshold 1030 for the tenant.), wherein the output value is one of an output of the first AI model, an output value of the second AI model, or a combined output value of the first AI model and the second AI model (Odibat et al. 2024/0273396 [0020] In embodiments, an operation or MLOps framework is established that can store models and model versions with the following details: target, prediction, accuracy, tenant/overall champions, categoric features list, numeric features list, etc. The tenant metadata, e.g., profiles and usage history, may be stored. The MLOps framework orchestrates the deployment of models using an AI Model dynamic model selector (DMS) to pick the best base model.).
Odibat et al. 2024/0273396 may not expressly disclose the “window of time” features, however, Miller et al. 2023/0037733 teaches a deployment period for deploying an algorithm which is interpreted as the claimed “window of time” (Miller et al. 2023/0037733 [claim 1] 1. A system associated with a live environment executing a current algorithm, comprising: at least one memory; instructions; and processor circuitry to execute the instructions to: manage execution of the current algorithm in the live environment, the live environment associated with a deployment platform; and manage, based on receipt of at least one potential replacement algorithm, a shadow environment to cause execution of the at least one potential replacement algorithm in the shadow environment, the shadow environment instantiated separately from the live environment, the shadow environment to utilize data from the live environment during execution of the at least one potential replacement algorithm in the shadow environment, wherein the deployment platform is to form a hybridized algorithm, the hybridized algorithm including the current algorithm and the at least one potential replacement algorithm, the at least one potential replacement algorithm deployed during a first deployment period and the current algorithm deployed during a second deployment period, the first deployment period different from the second deployment period.).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Odibat et al. 2024/0273396 to include the features as taught by Miller et al. 2023/0037733. One of ordinary skill in the art would have been motivated to do so to implement well known tools and features useful in AI feature lifecycle management through AI model updates which should prove to improve user experience, maximize profits, and optimize revenue.
Odibat et al. 2024/0273396 may not expressly disclose the “displaying an output value” features, however, Andersh et al. 2003/0100972 teaches (Andersh et al. 2003/0100972 [0006 - receive the output values from the model aggregator and to display the output values] In another embodiment, the invention is directed to a computer-readable medium containing instructions. The instructions cause a processor to instantiate a set of objects that encapsulate computational models and that include generic interfaces for invoking the computational models. The instructions further cause the processor to instantiate a model aggregator to distribute input values to the objects and to receive predicted output values from the objects. The instructions cause the processor to instantiate a control module to receive the output values from the model aggregator and to display the output values.).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Odibat et al. 2024/0273396 to include the features as taught by Andersh et al. 2003/0100972. One of ordinary skill in the art would have been motivated to do so to implement well known tools and features useful in AI feature lifecycle management through AI model updates which should prove to improve user experience, maximize profits, and optimize revenue.
18/304,710 – Claim 11. A data processing system, comprising: a processor; and a memory coupled to the processor to store instructions therein for managing artificial intelligence (AI) model rollouts, wherein the instructions, when executed by the processor, cause the processor to perform operations, the operations comprising: training a second AI model in a cloud environment, wherein the second AI model outputs a same AI feature as a first AI model in the cloud environment; specifying a window of time in which the second AI model is to be rolled out to a plurality of tenant applications; and displaying an output value for the AI feature to each of the plurality of tenant applications based on a timestamp associated with the tenant application, wherein the output value is one of an output value of the first AI model, an output value of the second AI model, or a combined output value of the first AI model and the second AI model.
Claim 11, has similar limitations as of Claim(s) 1, therefore it is REJECTED under the same rationale as Claim(s) 1.
18/304,710 – Claim 20. A non-transitory computer-readable medium that stores instructions for managing artificial intelligence (AI) model rollouts, which instructions, when executed by a data processing system comprising at least one hardware processor, cause the data processing system to perform operation comprising: training a second AI model in a cloud environment, wherein the second AI model outputs a same AI feature as a first AI model in the cloud environment; specifying a window of time in which a second AI model is to be rolled out to a plurality of tenant applications; and displaying an output value for the AI feature to each of the plurality of tenant applications based on a timestamp associated with the tenant application, wherein the output value is one of an output value of the first AI model, an output value of the second AI model, or a combined output value of the first AI model and the second AI model.
Claim 20, has similar limitations as of Claim(s) 1, therefore it is REJECTED under the same rationale as Claim(s) 1.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over: Odibat et al. 2024/0273396; in view of Miller et al. 2023/0037733; in further view of Andersh et al. 2003/0100972; in view of Zhang et al. 2019/0243753.
18/304,710 – Claim 2. Odibat et al. 2024/0273396 further teaches The method of claim 1, wherein the timestamp represents a time when the tenant application is signed up for receiving the output value from the cloud environment (Odibat et al. 2024/0273396 [0088] Referring again to FIG. 8, the flow of operations 1-10 are indicated by a circle with the number 1-10. Operations 1-10 are also described in detail below in conjunction with FIG. 11, and referred to as operation (1), operation (2), . . . , and operation (10), for simplicity. In this example, a model repository includes a model registry 801 for base models model 1 (803A), model 2 (803B), and model 3 (803C). Initially, a model is deployed at model deployment 820 for tenant A (841A) and tenant B (841B). A dynamic model selector 822, as discussed in more detail in conjunction with FIG. 9, may be used to select the appropriate model for deployment. As illustrated, model 1 (803A) is initially deployed to both tenant A (841A) and tenant B (841B). Model 1 (803A) may be considered a default “champion” model that has gone thru champion/challenger testing, and is the default for tenants A (841A) and B (841B). Model 2 (803B), model 3 (803C), and model 1.2 (803D) may be champion models for at least one specific tenant. In this illustration the models used by tenant A (841A) and tenant B (841B) are monitored by tenant model monitoring 830. The tenant model monitoring receives a configuration profile from dynamic configuration 832. Tenant model monitoring 830 may be triggered based on time, tenant profiles, data/concept drift and tenant usage, as indicated in the tenant configuration.) .
Odibat et al. 2024/0273396 may not expressly disclose the “timestamp” features, however, Zhang et al. 2019/0243753 teaches (Zhang et al. 2019/0243753 [0309 - a first timestamp registering a receipt of the availability information at the cloud services 1552 or at the sub-service of the cloud application 1554 and receiving a second-time stamp registering the cloud services 1552 or the sub-service of the cloud application 1554 sending the availability information back to the cloud check control engine] The system 1550 may further be utilized for testing the subject system 1550 in a latency testing process utilizing one or more digital computers, according to an embodiment. The cloud check control engine may further measure the amount of elapsed time (e.g., latency) by receiving a first timestamp registering a receipt of the availability information at the cloud services 1552 or at the sub-service of the cloud application 1554 and receiving a second-time stamp registering the cloud services 1552 or the sub-service of the cloud application 1554 sending the availability information back to the cloud check control engine. The cloud check control engine may generate a measure of latency by subtracting first stamp from the second-time stamp. The intermittency metric 1141 and the severity metric 1170 may be applied as meta-metrics for the cloud service 1552 or the sub-services of the cloud application 1554 (e.g. latency). For example, if the industry average latency is 1 millisecond for the cloud services 1552 and the measurement unit is hourly, service latency >1 millisecond may be defined as an intermittent failure with value=0 (E.G., FAIL).).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Odibat et al. 2024/0273396 to include the features as taught by Zhang et al. 2019/0243753. One of ordinary skill in the art would have been motivated to do so to implement well known tools and features useful in AI feature lifecycle management through AI model updates which should prove to improve user experience, maximize profits, and optimize revenue.
18/304,710 – Claim 12. The data processing system of claim 11, wherein the timestamp represents a time when the tenant application is signed up for receiving the output value from the cloud environment.
Claim 12, has similar limitations as of Claim(s) 2, therefore it is REJECTED under the same rationale as Claim(s) 2.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over: Odibat et al. 2024/0273396; in view of Miller et al. 2023/0037733; in further view of Andersh et al. 2003/0100972; in view of Ma et al. 2015/0120244.
18/304,710 – Claim 3. Odibat et al. 2024/0273396 further teaches The method of claim 1, wherein the combined output value is generated using a smoothing algorithm running in the cloud environment (Odibat et al. 2024/0273396 [0046-0047 – cloud environment][0085 - algorithms] In known systems, there may be 100 different algorithms that may be used for a tenant for their cloud usage forecasting. A data scientist may need to make educated guesses to determine which subset of those 100 algorithms to test for use with the tenant before selecting one to deploy. For instance, the data scientist may guess that linear regression or logistical regression many be viable candidates, but the testing and validation of these algorithms is time intensive. Further, as the tenant's data changes, the whole process may need to be manually repeated. In the examples discussed herein, the term Model 1 may refer to a particular machine learning (ML) algorithm that may be used or customized for the tenant. The term Model 1.2 may refer to the same algorithm that is trained with different data, or another variant of Model 1. Model 2 may refer to a different algorithm than Model 1 or Model 1.2. In known systems, manually selecting the best model for a tenant is time-consuming, expensive and may be error prone, requiring empirical trial and error tests. Further, refreshing the models when the tenant's data or business changes and degrades the accuracy of the initial model selected may require a full manual refresh. In contrast, embodiments as described herein provide an automated system for selecting, deploying, and refreshing machine learning models for multiple tenants. [0102 – multi-tenant cloud environment running various algorithmic models] FIG. 11 is a flow diagram depicting an exemplary method 1100 for managing model operations (MLOps), in accordance with aspects of the invention. In a multi-tenant cloud environment, multiple tenants will subscribe to machine learning (ML) models from one or more providers to make forecasts, predictions, or to help manage their business' cloud and resource usage. A tenant corresponds to a configuration file, also referred to as a configuration profile, and will be identified as a specific category based on their business type, size, location, cost, and resource constraints, etc. A cloud model provider may develop their own models using in-house data scientists 1101 and/or use third party models developed by outside data scientists or organizations 1103. Each model may correspond to a different ML algorithm such as random forest, decision trees, gradient boosting trees, support vector machine (SVM), Naïve Bayes, KNN, linear regression, logical regression, deep learning, and other algorithms now known or to be developed in the future. It will be understood that different algorithms work better for different data sets (both size and type), and resource constraints. Therefore, some models will be better suited for a tenant based on its configuration profile. The cloud model provider receives the models from the data scientists and/or third party in block 1110. The cloud model provider stores the various models in a model registry in block 1120. The model registry may be a local storage media or be accessed across a network remotely. The model registry may be distributed between or among more than one physical storage media, such as the storage system 34 of FIG. 1. The ML models are stored with a corresponding identifier for model version, and criteria for use as a default model based on a tenant's configuration profile. Some models may be designated as champion models for one or more tenant category.).
Odibat et al. 2024/0273396 may not expressly disclose the “smoothing algorithm” features, however, Ma et al. 2015/0120244 teaches (Ma et al. 2015/0120244 [0055 - smoothing algorithm…] Finally, in operation 208, apparatus 100 may optionally include means, such as processor 102 or the like, for applying a smoothing algorithm to adjust the road width of at least one section. In some embodiments, the smoothing algorithm is based on an arbitrary number N of neighbors, in which case the road width is adjusted to be either the mean value or median value of road widths of the N sections of the point cloud centered on the section being evaluated. In some embodiments, the smoothing algorithm may be applied only in situations where there are occlusions in the section of the point cloud being evaluated. Additionally or alternatively, the smoothing algorithm may be applied if the calculated road width of the section being evaluated differs from its neighbors by more than a predetermined amount or percentage. Accordingly, by preventing sudden changes of road width between sections, the smoothing algorithm is likely to reduce the impact of "noise" in the point cloud data, and concomitantly to increase the accuracy of the overall road width estimation.).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Odibat et al. 2024/0273396 to include the features as taught by Ma et al. 2015/0120244. One of ordinary skill in the art would have been motivated to do so to implement well known tools and features useful in AI feature lifecycle management through AI model updates which should prove to improve user experience, maximize profits, and optimize revenue.
18/304,710 – Claim 13. The data processing system of claim 11, wherein the combined output value is generated using a smoothing algorithm running in the cloud environment.
Claim 13, has similar limitations as of Claim(s) 3, therefore it is REJECTED under the same rationale as Claim(s) 3.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over: Odibat et al. 2024/0273396; in view of Miller et al. 2023/0037733; in further view of Andersh et al. 2003/0100972; in view of Ruiz et al. 2024/0047052.
18/304,710 – Claim 4. Odibat et al. 2024/0273396 further teaches The method of claim 1, wherein the smoothing algorithm takes an weighted average of the output value from the first AI model and the output value from the second AI model to generate the combined output value (Odibat et al. 2024/0273396 [0087 – AI models] In embodiments, a framework for multi-tenant MLOps, where each tenant is provided their own customized version of the operational ML model is discussed herein. A model version for a specific project is dynamically selected using an artificial intelligence-based (AI-based) machine learning model operations (MLOps). The MLOps AI-based model is dynamically tuned over time based on service level agreements (SLA), resource constraints, and cost using a new AI model. The model refresh for tenants is triggered based on tenant usage, cost thresholds, and tenant profile along with other approaches, such as data and target drifts. The number of operational ML models needed may depend on similarity across tenants rather than number of tenants. For example, N similar tenants may utilize one operational ML model. [0097] In an embodiment, service level agreement (SLA) 1011 may include time constraints in units of seconds. Resource constraints may include data for scalability, regulatory, maintainability, serviceability, security, and cost. Cost data may contain AssetId, asset type, unit cost, total cost, region, data center, period, and description. Outputs of the deep learning neural network 1021 include a model performance threshold 1030 and model refresh frequency 1040. Two different loss functions may be used to learn the outputs. As discussed above, softmax is used to determine the model refresh frequency 1040, and linear regression is used to identify the model performance threshold 1030. In particular, softmax is a generalization of the logistic function to multiple dimensions, and used in multinomial logistic regression, and may be used to normalize output of the neural network 1021. The deep learning neural network uses a Leaky Rectified Linear Unit (Leaky ReLU) function across the hidden layers. Leaky ReLU is a type of activation function based on a ReLU, but it has a small slope for negative values instead of a flat slope. The slope coefficient may be determined before training, i.e., the slope coefficient may not be learned during training. Input is passed to deep hidden layers where weights have been shared among each other for both outputs. After backpropagation and forward propagation models are used to reduce the loss and return the model refresh frequency 1040 and model performance threshold 1030 for the tenant.).
Odibat et al. 2024/0273396 may not expressly disclose the “weighted average” features, however, Ruiz et al. 2024/0047052 teaches (Ruiz et al. 2024/0047052 [0075 - forecasting machine learning models include autoregressive models that use past observations of a target variable as an input to predict future values (e.g., ARIMA (AutoRegressive Integrated Moving Average)); seasonal autoregressive integrated moving-average (SARIMA) models extend ARIMA by considering seasonal patterns in addition to the regular autoregressive, integrated, and moving average components; exponential smoothing models that use weighted averages…] The present invention can employ one or more machine learning models, such as a forecasting model. The forecasting models can be employed to address or handle different types of forecasting tasks. Some of the main types of forecasting machine learning models include autoregressive models that use past observations of a target variable as an input to predict future values (e.g., ARIMA (AutoRegressive Integrated Moving Average)); seasonal autoregressive integrated moving-average (SARIMA) models extend ARIMA by considering seasonal patterns in addition to the regular autoregressive, integrated, and moving average components; exponential smoothing models that use weighted averages of past observations to predict future values and are useful for time series data with no clear trend or seasonality (e.g., Simple Exponential Smoothing, Double Exponential Smoothing (Holt's method), and Triple Exponential Smoothing (Holt-Winters' method)); long short-term memory (LSTM) networks which is a type of recurrent neural network (RNN) capable of learning long-term dependencies in time series data and are particularly effective for processing sequential data with complex temporal patterns; gradient boosting machines (GBM) which is an ensemble learning technique that builds multiple decision trees to make predictions (e.g., XGBoost and LightGBM for time series forecasting); prophet or Facebook Prophet incorporates time series components such as seasonality and holidays and can effectively handle missing data and outliers; neural prophet is an extension of the Facebook Prophet model that utilizes neural networks to capture complex temporal patterns and relationships in the data; DeepAR which is a probabilistic forecasting model based on recurrent neural networks that can handle uncertainty in time series data; VAR (Vector Autoregression) models which can be used for multivariate time series forecasting, where multiple related variables are forecasted simultaneously; and CNN (Convolutional Neural Networks) for Time Series models which can be adapted for time series forecasting, especially when dealing with 1D sequences. The choice or selection of the proper forecasting model depends on the specific characteristics of the time series data, the presence of trends and seasonality, the amount of data available, and the desired level of interpretability and complexity.).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Odibat et al. 2024/0273396 to include the features as taught by Ruiz et al. 2024/0047052. One of ordinary skill in the art would have been motivated to do so to implement well known tools and features useful in AI feature lifecycle management through AI model updates which should prove to improve user experience, maximize profits, and optimize revenue.
18/304,710 – Claim 14. The data processing system of claim 11, wherein the smoothing algorithm takes an weighted average of the output value from the first AI model and the output value from the second AI model to generate the combined output value.
Claim 14, has similar limitations as of Claim(s) 4, therefore it is REJECTED under the same rationale as Claim(s) 4.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over: Odibat et al. 2024/0273396; in view of Miller et al. 2023/0037733; in further view of Andersh et al. 2003/0100972; in view of XU et al. 2021/0201128.
18/304,710 – Claim 10. Odibat et al. 2024/0273396 further teaches The method of claim 1, wherein the output value displayed by the cloud environment to each of the plurality of tenant applications is a prediction score representing a probability that a task is to be successfully closed (Odibat et al. 2024/0273396 [0020 - prediction] In embodiments, an operation or MLOps framework is established that can store models and model versions with the following details: target, prediction, accuracy, tenant/overall champions, categoric features list, numeric features list, etc. The tenant metadata, e.g., profiles and usage history, may be stored. The MLOps framework orchestrates the deployment of models using an AI Model dynamic model selector (DMS) to pick the best base model. [0084 - predictive model] FIG. 8 illustrates a hybrid flow diagram and block diagram 800 depicting a method for managing Model Ops (MLOps), in accordance with aspects of the invention. Typically, a model deployed for client use has a base version. In embodiments, to automatically trigger model refresh, the operational ML models in use by tenants are monitored. In model monitoring, the predictive model performance needs to be regularly monitored to invoke a new iteration in the ML process. Data drift and concept drift also need to be monitored. An ML model may have dozens or hundreds of features or variables in the model. For example, a supervised model may have 70-80 or more variables. Data drift is the change in distribution for each feature over time. Concept drift is change in model performance due to changes in the underlying concept of the business. For example, a mortgage loan business may expand their region and change their approval algorithms. These changes might increase their use of the cloud, for instance for customer service or application interfaces. Thus, if the tenant adds a new region to their business, the model concept may drift and require a refresh. Once a drift is detected, a model refresh may be automatically triggered. By automatically triggering a model refresh, unnecessary delay may be avoided. For example, unnecessary delay in known systems may include waiting for the data scientists to become involved in the monitor/refresh cycle. A model may be refreshed based on certain metrics and pre-defined guidelines in order to keep it relevant and accurate. Providing accurate and relevant models may involve model validation. Refreshed models may need to be verified that they are fit for deployment. In order to verify refreshed models, the model's predictive performance is tested against a specific baseline model. This process for model verification may be more challenging in a multi-tenant environment where each tenant has unique business models, requirements, and desired outcomes from the models. [0099 - probabilities] The model management 1020 considers the tenant type and tenant resources, e.g., bronze, sliver, gold, or entry, standard, enterprise, etc., so that the trained deep learning model 1021 is able to execute on the data 1010 of the tenant successfully within the resource constraints 1015, cost 1013 and performance 1011 constraints for the tenant. In an example, the deep learning 1021 is a neural network that may use a softmax function 1022 to assign decimal probabilities to each class in a multi-class problem. Softmax function 1022 may be used to determine a best model refresh frequency. Linear regression 1024 may be used to identify a best model performance threshold. Linear regression may be a supervised learning technique that involves learning the relationship between the features and the target. Alternatively, other algorithms and techniques may be used with the neural network 1021 to help determine model refresh frequency and model performance or accuracy thresholds.).
Odibat et al. 2024/0273396 may not expressly disclose the “prediction scoring” features, however, XU et al. 2021/0201128 teaches (XU et al. 2021/0201128 [0022 - a scoring system can predict the probability that a task can be closed or not based on historical data and the current status of the task] According to another embodiment, a scoring system can predict the probability that a task can be closed or not based on historical data and the current status of the task. The scoring system is highly configurable and can incorporate data from a variety of data sources, and can easily be extended to other machine learning tasks by changing a number of modules/configuration in the scoring system. The scoring system can use real-time data from a task database system, and therefore can provide accurate, timely prediction results to users. An explanation for each prediction score can be generated to help users identify reasons for high/low scores, so that the users can take immediate actions accordingly. Feature importance information for all tasks can be collected and provided to users to help them understand fields that are deterministic. [0066 - prediction scores] In one embodiment, the ML pipeline can be used to generate a number of types of prediction scores, including time-agnostic scores, scores by end of time period (ETP), and scores by a close date. A time-agnostic score, which is not tied to any specific time, measures the probability that a task can be closed as a won deal without considering the close date. A score by ETP measures the probability that a task can be closed by the end of a particular period (e.g., a quarter or a year). A score by a close date measures the probability that a task can be close by a specified close date. Depending on the type of score that the ML pipeline is configured to generate, different set of features can be used to train an ML model.).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Odibat et al. 2024/0273396 to include the features as taught by XU et al. 2021/0201128. One of ordinary skill in the art would have been motivated to do so to implement well known tools and features useful in AI feature lifecycle management through AI model updates which should prove to improve user experience, maximize profits, and optimize revenue.
No Prior-art Rejection / Potentially Allowable
Claims 5-9 and 15-19 cannot be rejected with prior-art. Individual claimed features are taught in the prior-art, however, the unique combination of features and elements are not taught by the prior-art without hindsight reasoning. These claims are further rejected to as being dependent upon a rejected base claim but might possibly be allowable if rewritten in independent form including all the limitations of the base claim and any intervening claims.
18/304,710 – Claim 5. The method of claim 4, wherein an weight of the output value of the first AI model gradually decreases from a start of the rollout window to and an end of the rollout window, and an weight of the output value of the second AI model proportionally decreases ().
18/304,710 – Claim 6. The method of claim 5, wherein the weight of the output value of the first AI model is 100% and the weight of the output value of the second AI model is 0 at the start of the rollout window.
18/304,710 – Claim 7. The method of claim 5, wherein the weight of the output value of the first AI model is 0 and the weight of the output value of the second AI model is 100% at the end of the rollout window.
18/304,710 – Claim 8. The method of claim 4, wherein the output value displayed to each of the plurality of tenant applications is accompanied by a plurality of explanatory factors.
18/304,710 – Claim 9. The method of claim 8, wherein when the output value is the combined output value of the first AI model and the second AI model, the plurality of explanatory factors include more explanatory factors from one of the first AI model or the second AI model with a greater weight given in generating the combined output value.
18/304,710 – Claim 15. The data processing system of claim 14, wherein an weight of the output value of the first AI model gradually decreases from a start of the rollout window to and an end of the rollout window, and an weight of the output value of the second AI model proportionally decreases.
18/304,710 – Claim 16. The data processing system of claim 15, wherein the weight of the output value of the first AI model is 100% and the weight of the output value of the second AI model is 0 at the start of the rollout window.
18/304,710 – Claim 17. The data processing system of claim 15, wherein the weight of the output value of the first AI model is 0 and the weight of the output value of the second AI model is 100% at the end of the rollout window.
18/304,710 – Claim 18. The data processing system of claim 14, wherein the output value displayed to each of the plurality of tenant applications is accompanied by a plurality of explanatory factors.
18/304,710 – Claim 19. The data processing system of claim 18, wherein when the output value is the combined output value of the first AI model and the second AI model, the plurality of explanatory factors include more explanatory factors from one of the first AI model or the second AI model with a greater weight given in generating the combined output value.
Prosecution Insights