Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 15 recites:
generating a collaborative assignment recommendation from a first candidate triage group generative AI model and at least a second candidate triage group generative AI model by processing the incident request with the first candidate triage group generative AI model and the at least a second candidate triage group using training data associated with each respective candidate triage group; and
In order to maintain consistency within the terms, examiner suggest to amend the claim to read:
generating a collaborative assignment recommendation from a first candidate triage group generative AI model and at least a second candidate triage group generative AI model by processing the incident request with the first candidate triage group generative AI model and the at least a second candidate triage group generative AI model using training data associated with each respective candidate triage group; and
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Step 1 analysis for all claims:
In the instant case, claims 1-7 are directed to a process, claims 8-14 are directed to a system and claims 15-20 are directed to a manufacture. Thus, each of the claims falls within one of the four statutory categories (i.e., process, machine, manufacture, or composition of matter).
Claim 1:
Step 2A, Prong 1 analysis:
The claim recites in part:
processing an incident request… associated with a cloud computing system. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses receiving a text-based description of the cloud system issue and figuring out what part of the system is wrong.
identifying a candidate triage group generative artificial intelligence (AI) model by processing the incident request. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses choosing a machine learning model that can choose a group to give the incident request to in order to solve the incident request using the text-based description.
generating an assignment recommendation from the candidate triage group generative AI model. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses suggesting a group to give the incident request to in order to solve the incident request.
selecting a target triage group for triaging the incident request by processing the assignment recommendation from the candidate triage group generative AI model. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses choosing a group using the suggestion to give the incident request to.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
using a triage engine; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
by processing the incident request with the candidate triage group generative AI model using training data associated with the respective candidate triage group; and; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
using a triage engine;
The computer elements are recited at a high-level of generality such and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
by processing the incident request with the candidate triage group generative AI model using training data associated with the respective candidate triage group; and;
The Computer elements are recited at a high-level of generality such and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 2:
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
providing the incident request to the target triage group; and; which amounts to extra-solution activity of transmitting data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
automatically triaging the incident request using the target triage group.; which amounts to extra-solution activity of transmitting data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
providing the incident request to the target triage group; and;
This limitation is directed to receiving input at an interface on a computing device, wherein the input comprises a dataset, an analysis for the dataset, and an output medium which amounts to extra-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
automatically triaging the incident request using the target triage group.
This limitation is directed to receiving input at an interface on a computing device, wherein the input comprises a dataset, an analysis for the dataset, and an output medium which amounts to extra-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 3:
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
identifying a candidate historical incident from an incident database; which amounts to extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
identifying a candidate historical incident from an incident database; As discussed above, the additional element of finding a specific event from a collection of previous events amounts to extra-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 4:
Step 2A, Prong 1 analysis:
The claim recites in part:
wherein identifying the candidate triage group generative AI model includes processing the candidate historical incident to identify a candidate triage group associated with triaging the candidate historical incident. As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses using a previous incident event that happened along with the associated group that mitigated it to decide on what model to use to triage that current incident.
Step 2A, Prong 2 analysis:
There are no additional elements that individually or in combination integrate the judicial exception into a practical application.
Step 2B analysis:
There are no additional elements individually or in combination that amount to significantly more than the judicial exception.
Claim 5:
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
training the candidate triage group generative AI model; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
using a troubleshooting guide associated with the candidate triage group, a plurality of historical incidents, and a plurality of candidate triage group-specific documents; The limitation of specifics of the training data amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (See MPEP 2106.05(h)).
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
training the candidate triage group generative AI model; Training is recited at a high-level of generality with no detail of the training process such and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
using a troubleshooting guide associated with the candidate triage group, a plurality of historical incidents, and a plurality of candidate triage group-specific documents. As explained by the Supreme Court, a claim directed to a judicial exception cannot be made eligible "simply by having the applicant acquiesce to limiting the reach of the patent for the formula to a particular technological use." Diamond v. Diehr, 450 U.S. 175, 192 n.14, 209 USPQ 1, 10 n. 14 (1981). Thus, limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 6:
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein the candidate triage group generative AI model is a large language model (LLM).; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
wherein the candidate triage group generative AI model is a large language model (LLM).; The machine learning model is recited at a high level of generality and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. The training of the machine learning model is recited at a high-level of generality with no detail of the training process such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f)). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 7:
Step 2A, Prong 1 analysis:
The claim recites in part:
wherein generating the assignment recommendation includes generating a collaborative assignment recommendation from a first candidate triage group generative AI model and at least a second candidate triage group generative AI model; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses more details of suggesting a group to give the incident request to in order to solve the incident request. This involves using the recommendations from at least 2 models in order to come up with one recommendation for the group to mitigate the incident.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
by processing the incident request using the first candidate triage group generative AI model and the at least a second candidate triage group generative AI model; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
wherein each of the first candidate triage group generative AI model and the at least a second candidate triage group generative AI model are trained using data associated with each respective candidate triage group; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
by processing the incident request using the first candidate triage group generative AI model and the at least a second candidate triage group generative AI model;
The Computer element “candidate triage group generative AI model” are recited at a high-level of generality such and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
wherein each of the first candidate triage group generative AI model and the at least a second candidate triage group generative AI model are trained using data associated with each respective candidate triage group;
Training is recited at a high-level of generality with no detail of the training process such and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 8:
Step 2A, Prong 1 analysis:
The claim recites in part:
to process an incident request … associated with a cloud computing system; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses receiving a text-based description of the cloud system issue and figuring out what part of the system is wrong.
to identify a candidate historical incident from an incident database, to identify a plurality of candidate triage group generative artificial intelligence (AI) models by processing the incident request and the candidate historical incident; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses choosing previous event from a database and then a machine learning model using that previous event and the current incident in order choose a group to give the incident request to in order to solve the incident request.
to generate a first assignment recommendation from a first candidate triage group generative AI model by processing the incident request with the first candidate triage group generative AI model using training data associated with the first candidate triage group; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses suggesting a group to give the incident request to in order to solve the incident request using previous incident data.
to generate at least a second assignment recommendation from at least a second candidate triage group generative AI model by processing the incident request with the at least a second candidate triage group generative AI model using training data associated with the second candidate triage group, and; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses suggesting at least one other group to give the incident request to in order to solve the incident request using previous incident data.
to select a target triage group for triaging the incident request by processing the first assignment recommendation and the at least a second assignment recommendation.; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses choosing a group to give the request to using the suggestions of groups given.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
a memory; and a processor configured; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
using a triage engine; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
a memory; and a processor configured;
Computer elements are recited at a high-level of generality such and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
using a triage engine;
The Computer element is recited at a high-level of generality such and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 9:
Claim 9 recites substantially similar limitations for claim 2 and is therefore rejected on the same basis.
Claim 10:
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
wherein identifying the candidate historical incident includes identifying a most similar incident from the incident database.; which amounts to extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
wherein identifying the candidate historical incident includes identifying a most similar incident from the incident database.;
As discussed above, the additional element of finding a specific event from a collection of previous events amounts to extra-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 11:
Claim 11 recites substantially similar limitations for claim 4 and is therefore rejected on the same basis.
Claim 12:
Claim 12 recites substantially similar limitations for claim 5 and is therefore rejected on the same basis.
Claim 13:
Claim 13 recites substantially similar limitations for claim 6 and is therefore rejected on the same basis.
Claim 14:
Claim 14 recites substantially similar limitations for claim 7 and is therefore rejected on the same basis.
Claim 15:
Step 2A, Prong 1 analysis:
The claim recites in part:
processing an incident request … associated with a cloud computing system; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses receiving a text-based description of the cloud system issue and figuring out what part of the system is wrong.
identifying a plurality of candidate triage group generative artificial intelligence (AI) models by processing the incident request; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses choosing a machine learning model using the current incident in order choose a group to give the incident request to in order to solve the incident request.
generating a collaborative assignment recommendation from a first candidate triage group generative AI model and at least a second candidate triage group generative AI model by processing the incident request with the first candidate triage group generative AI model and the at least a second candidate triage group using training data associated with each respective candidate triage group; and; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses using suggestions to finalize one suggestion for a group to give the incident request to in order to solve the incident request using previous incident data.
selecting a target triage group for triaging the incident request by processing the collaborative assignment recommendation; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses choosing a group to give the request to using the final suggestion of a group.
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
using a triage engine; which are recited at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computer component (See MPEP 2106.05(f))
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising;
Computer elements are recited at a high-level of generality such and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
using a triage engine;
The Computer element is recited at a high-level of generality such and amounts to no more than adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea (See MPEP 2106.05(f))
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 16:
Claim 16 recites substantially similar limitations for claim 2 and is therefore rejected on the same basis.
Claim 17:
Step 2A, Prong 2 analysis:
The judicial exception is not integrated into a practical application. In particular, the claim recites the additional elements of:
identifying a plurality of most similar candidate historical incidents from an incident database.; which amounts to extra-solution activity of gathering data for use in the claimed process. As described in MPEP 2106.05(g), limitations that amount to merely adding insignificant extra-solution activity to a judicial exception do not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application.
Accordingly, at Step 2A, prong two, the additional elements individually or in combination do not integrate the judicial exception into a practical application
Step 2B analysis:
In accordance with Step 2B, the claim does not include additional elements that are sufficient to amount to significantly more that the judicial exception. As discussed above, the additional elements of:
identifying a plurality of most similar candidate historical incidents from an incident database.;
As discussed above, the additional element of finding a specific event from a collection of previous events amounts to extra-solution activity of gathering data for use in the claimed process. The courts have found limitations directed to obtaining information electronically, recited at a high level of generality, to be well-understood, routine, and conventional (see MPEP 2106.05(d)(II), “receiving or transmitting data over a network”, "electronic record keeping," and "storing and retrieving information in memory").
Accordingly, at Step 2B the additional elements individually or in combination do not amount to significantly more than the judicial exception.
Claim 18:
Step 2A, Prong 1 analysis:
The claim recites in part:
wherein identifying the plurality of candidate triage group generative AI models includes processing the plurality of most similar candidate historical incidents to identify a plurality of candidate triage groups associated with triaging the plurality of most similar candidate historical incidents.; As drafted and under its broadest reasonable interpretation, this limitation covers performance of the limitation in the mind (including an observation, evaluation, judgment, opinion) or with the aid of pencil and paper but for the recitation of generic computer components. For example, this limitation encompasses using a previous incident events that happened along with the associated groups that mitigated them to decide on what models to use to triage that current incident.
Step 2A, Prong 2 analysis:
There are no additional elements that individually or in combination integrate the judicial exception into a practical application.
Step 2B analysis:
There are no additional elements individually or in combination that amount to significantly more than the judicial exception.
Claim 19:
Claim 19 recites substantially similar limitations for claim 5 and is therefore rejected on the same basis.
Claim 20:
Claim 20 recites substantially similar limitations for claim 6 and is therefore rejected on the same basis.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-20 are rejected under 35 U.S.C 103 as being unpatentable over Schreiber et al. (US 11595243 B1, hereinafter Schreiber) in view of Ahmed et al. (Recommending Root-Cause and Mitigation Steps for Cloud Incidents using Large Language Models, hereinafter Ahmed).
Regarding Claim 1
Schreiber teaches:
A computer-implemented method, executed on a computing device, comprising:
(Schreiber [col 10 lines 59-67, col 11 line 1] “FIG. 6 is a flow diagram illustrating operations 600 of a method for automated incident triage and diagnosis according to some embodiments. Some or all of the operations 600 (or other processes described herein, or variations, and/or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions, and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executing collectively on one or more processors.”; Examiner’s note (EN): this paragraph denotes a computer-implemented method for automated incident triage)
processing an incident request using a triage engine associated with a cloud computing system;
(Schreiber [col 12 lines 6-9] “In some embodiments, the operations include receiving a request from a monitoring service of a provider network to identify a cause of an incident affecting performance of one or more services or components,”
[col 11 lines 9-10] The operations 600 include, at block 602, receiving incident data associated with an incident.
[col 11 lines 16-19] In some embodiments, the incident data is received from a monitoring service of the provider network which requests an incident management service to analyze the incident.;
(EN): receiving and analyzing the incident reads on “processing”)
(Schreiber [col 1 lines 60-66] “The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for automated incident triage and diagnosis. According to some embodiments, an incident management service is responsible for automatically identifying possible causes of an incident, and the teams most likely able to assist in mitigating the incident, when an incident is detected.”;
(EN): “incident management service” reads on “triage engine” which denotes a system that analyzes incident data to provide an assignment (Ex. to a team) in order mitigate the incident more efficiently)
(Schreiber [col 11 lines 9-19] “Alternatively, in some embodiments, the incident is associated with a service or component of a customer application running in a provider network. … In some embodiments, the incident data is received from a monitoring service of the provider network which requests an incident management service to analyze the incident.”;
[col 2 lines 45 - 48] “A provider network 100 (or, “cloud” provider network) provides users with the ability to use one or more of a variety of types of computing-related resources such as compute resources”;
(EN): this paragraph denotes the association of the cloud computing system with the triage engine)
identifying a candidate triage group … model by processing the incident request;
(Schreiber [col 11 lines 20-23] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206.”;
[col 8 lines 4-7] “In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208.”;
EN: candidate triage group model reads as model that analyzes incident details and generates candidate teams that could mitigate it; this paragraph denotes the identification of “one or more additional models 202-206” which reads on “candidate triage group model”)
generating an assignment recommendation from the candidate triage group … model by processing the incident request with the candidate triage group … model using training data associated with the respective candidate triage group; and
(Schreiber [col 11 lines 23 - 28] “In some embodiments, the at least one machine learning model comprises an ensemble of machine learning models including a … machine learning model to identify a team associated with the incident...”;
(EN): this paragraph denotes the team assignment recommendation; identify a team associated with the incident” reads on “assignment recommendation”)
(Schreiber [col 12 lines 6-36] “In some embodiments, the operations include receiving a request from a monitoring service of a provider network to identify a cause of an incident affecting performance of one or more services or components, the request including performance metric data associated with the provider network, obtaining state data from an incident data repository, the state data indicating changes to the provider network over a previous time period prior to the incident, providing the performance metric data and the state data to an incident machine learning model, the incident machine learning model trained to predict a cause of incidents affecting the provider network and to predict a team to resolve the incidents affecting the provider network, receiving inference results from the incident machine learning model indicating at least one cause of the incident and at least one team to resolve the incident, and automatically executing one or more mitigation actions to mitigate the incident based at least on the inference results.
In some embodiments, automatically executing one or more mitigation actions to mitigate the incident based at least on the inference results, further includes based on a confidence value associated with the at least one team to resolve the incident, automatically engaging the at least one team to mitigate the incident. In some embodiments, automatically executing one or more mitigation actions to mitigate the incident based at least on the inference results, further includes based on a confidence value associated with the at least one team to resolve the incident, automatically sending a request to a service update manager to roll back the state change.”
(EN): this paragraph denotes processing of an incident using candidate triage group model)
(Schreiber [col 11 lines 43 - 51] “In some embodiments, the operations further include providing the past incident data to a model training system of a machine learning service of the provider network, wherein the model training system is configured to train the at least one machine learning model using the past incident data. In some embodiments, at least a portion of the past incident data is auto-labeled to indicate a cause of a corresponding past incident and details of a resolution of the corresponding past incident.”;
(EN): this paragraph denotes the training process which uses training data associated with the resolution (candidate triage group); “details of a resolution of the corresponding past incident” reads on
“training data associated with the respective candidate triage group”)
selecting a target triage group for triaging the incident request by processing the assignment recommendation from the candidate triage group … model using the triage engine.
(Schreiber [col 11 lines 29-36] “The operations 600 further include, at block 606, automatically executing the one or more mitigation actions to mitigate the incident. In some embodiments, the one or more mitigation actions include at least one of engagement of a team indicated by the at least one machine learning model to be associated with a cause of the incident or roll back of a state change indicated by the at least one machine learning model to be associated with the cause of the incident.”;
(EN): engaging a team that was predicted reads on “selecting a target triage group … by processing assignment recommendation”; automatically engaging the team that can resolve the incident correctly makes mitigation more efficient reads on triaging an incident; [col 2 lines 12-21] “incident management service helps focus and automate part of the diagnosis process, which reduces the amount of time required to identify relevant mitigation strategies and return the service(s) to operation sooner. Additionally, embodiments allow incidents to be analyzed using machine learning while the incidents are occurring. As such, rather than being applied retrospectively after incidents have already occurred and been mitigated, embodiments improve the mitigation response itself to an ongoing incident.”)
(Schreiber [col 1 lines 60-66] “The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for automated incident triage and diagnosis. According to some embodiments, an incident management service is responsible for automatically identifying possible causes of an incident, and the teams most likely able to assist in mitigating the incident, when an incident is detected.”;
[col 11 lines 4-8] “In some embodiments, one or more (or all) of the operations 600 are performed by incident management service 102, model training system 120, etc. of the other figures.”;
(EN): “incident management service” reads on “triage engine” as denoted earlier; the second passage describes all the processes being executed on incident management service 102)
Schreiber teaches a candidate triage group model but does not explicitly teach:
candidate triage group generative AI model.
However, Ahmed further teaches:
candidate triage group generative AI model
(Ahmed [Abstract] “Incident management for cloud services is a complex process involving several steps and has a huge impact on both service health and developer productivity. On-call engineers require significant amount of domain knowledge and manual effort for root causing and mitigation of production incidents. Recent advances in artificial intelligence has resulted in state-of the-art large language models like GPT-3.x (both GPT-3.0 and GPT-3.5), which have been used to solve a variety of problems ranging from question answering to text summarization. In this work, we do the first large-scale study to evaluate the effectiveness of these models for helping engineers root cause and mitigate production incidents. We do a rigorous study at Microsoft, on more than 40,000 incidents and compare several large language models ... Lastly, our human evaluation with actual incident owners show the efficacy and future potential of using artificial intelligence for resolving cloud incidents.”;
(EN): this passage denotes using GPT-3.x , which reads on “generative artificial (AI) model”, to mitigate production incidents in order to help engineers which reads on “triage” )
Before the effective date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the method of triaging an incident with the generative artificial intelligence (AI) model of Ahmed in order to generate more complex solutions for mitigation of incidents.
(Ahmed [Section III. Methodology, C. Model configuration, 2nd paragraph, page 5] “For natural language translation learning the mapping between the words from different natural languages is essential to generate good quality translation. Code summarization is slightly different from these two, where the input is much longer than the output. However, Ahmed and Devanbu found that all the necessary information for code summarization is extracted from the identifiers, and obfuscating the identifiers hurts the models [33]. Generating root causes and mitigation plans is much more complex than these problems, where the input may not contain handy information. The models need to be able to generate more diverse and creative solutions to answer the question. Our problem is more aligned with code generation problems where the input does not carry most information. For these types of problems, it is found that instead of using the encoder-decoder model, decoder-only models (e.g., GPT-3.x) are more successful where we only focus on the following tokens considering the prior tokens generated by the models. It is well-established that encoder decoder models are not as successful as decoder-only models in code generation tasks.”;
(EN): “encoder decoder models” reads on machine learning models; “decoder-only model” reads on “generative artificial intelligence (AI) model”; this paragraph denotes the usage of generative AI models instead of machine learning models)
Regarding Claim 2
The combination of Schreiber and Ahmed teaches all of the limitations of claim 1 as cited above, and Schreiber further teaches:
providing the incident request to the target triage group;
(Schreiber [col 6 lines 19-26] “ In some embodiments, the incident model 116 outputs a probability of one or more services being associated with the root cause of the incident and a probability of a particular incident team as being relevant to resolving the incident. Optionally, depending on the probability, the incident orchestrator may automatically take one or more mitigation actions. For example, at A, the incident orchestrator may automatically engage a specific incident response team. This may include sending a request to incident team manager 118 which notifies members of the incident response team”;
(EN): this paragraph denotes providing the incident request to a predicted team that would mitigate it)
and automatically triaging the incident request using the target triage group.
(Schreiber [col 6 lines 19-37] “In some embodiments, the incident model 116 outputs a probability of one or more services being associated with the root cause of the incident and a probability of a particular incident team as being relevant to resolving the incident. Optionally, depending on the probability, the incident orchestrator may automatically take one or more mitigation actions. For example, at A, the incident orchestrator may automatically engage a specific incident response team. This may include sending a request to incident team manager 118 which notifies members of the incident response team. In some embodiments, additional investigation tasks may be assigned to the engaged experts. Additionally, or alternatively, the incident orchestrator 112 may automatically update, roll back, or otherwise change the deployment of a service identified as being likely to be a cause of the incident. For example, at B, the incident orchestrator 112 may send a request to a service update manager 122 to update or roll back a specific service 110 to resolve the incident.”;
(EN): this paragraph denotes the automatic triaging using the predicted triage group)
Regarding Claim 3
The combination of Schreiber and Ahmed teaches all of the limitations of claim 1 as cited above, and Schreiber further teaches:
identifying a candidate historical incident from an incident database.
(Schreiber [col 9 lines 30-38] “As shown in FIG. 3 , the historical incident data 300 can be provided to incident inference system 302. In some embodiments, incident inference system 302 may include a machine learning model trained, e.g., using a smaller, manually labeled training dataset. This machine learning model may be the same or similar to the machine learning models described above. In some embodiments, the incident inference system 302 can receive historical incident data 300 associated with one historical incident.”;
[col 4 lines 37-39] “The incident data repository 114 includes data related to past incidents.”
(EN): “historical incident data 300 can be provided to” denotes retrieving from an incident data repository as the incident data is stored in incident data repository; incident data repository reads on “incident database”)
Regarding Claim 4
The combination of Schreiber and Ahmed teaches all of the limitations of claim 1 as cited above, and Schreiber further teaches:
wherein identifying the candidate triage group … model includes processing the candidate historical incident to identify a candidate triage group associated with triaging the candidate historical incident.
(Schreiber [col 4 lines 37-52] The incident data repository 114 includes data related to past incidents. For example, a past incident's data may include: its start time and timeline of events; an impact description including impacted services, metrics and locations; a list of engaged groups and experts; state changes that occurred to the provider network 100 within 24 hours before the incident, including code changes and deployments, customer behavior changes, infrastructure issues, etc., with a timeline of each such event; post-incident analysis indicating the root cause of the incident and other relevant details. In some embodiments, the incident data repository is associated with a continuous data collection process that regularly (e.g., continuously or periodically) updates the incident data repository 114 with similar details for new incidents and ongoing events from various sources.
[col 5 lines 18-21] “…the incident orchestrator 112 may determine that an incident is occurring. At numeral 3, while the incident is occurring, the incident orchestrator 112 can retrieve past incident data from incident data repository 114”
[col 7 lines 56 - 67, col 8 lines 1-7] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206.”;
(EN): this paragraph denotes incident orchestrator that receives past incident data in relation to the current incident and chooses which models or model to execute)
Regarding Claim 5
The combination of Schreiber and Ahmed teaches all of the limitations of claim 1 as cited above, and Schreiber further teaches:
training the candidate triage group … model using a troubleshooting guide associated with the candidate triage group,
Schreiber [col 6 lines 62-67, col 7 lines 1-15] Once the incident has been resolved, the participants may provide additional feedback via interactive incident management tool 124. For example, the participants may identify the actual root cause that led to the incident. Additionally, or alternatively, the specific mitigation actions, the team responsible for identifying and implementing the actions, state changes that occurred and which state changes were relevant to the incident, etc. may also be provided. This serves as the ground truth incident label for this incident. This data is then added to incident data repository 114, at numeral 7. In some embodiments, when incident data repository 114 is updated with new incident information, a new training task is triggered which retrains incident model(s) 116. For example, a model training system of a machine learning service of the provider network may be invoked to start a new training task. The incident data maintained in incident data repository 114 is then provided to train a new incident model (or retrain the existing model). Once trained, the incident model is then deployed (e.g., via a model hosting service) where it can be used for future incident diagnosis.”;
(EN): feedback about “specific mitigation actions, the team responsible for identifying and implementing the actions, state changes that occurred and which state changes were relevant to the incident” reads on “troubleshooting guide”; this paragraph denotes the training of triage model using a guide)
a plurality of historical incidents,
Schreiber [col 7 lines 37-44] “In some embodiments, separate incident data and/or state data repositories may be maintained for each customer. The past incident data, current incident data, and state change data is then provided to incident models 116. In particular, incident models 116 may include a customer-specific model which has been trained on past … incident data to determine the likely causes of a … incident.”;
(EN): this paragraph denotes the training of candidate triage model using past incident data; “past incident data” reads on “historical incidents”)
and a plurality of candidate triage group-specific documents.
Schreiber teaches the training of a training the candidate triage group model but does not explicitly teach:
candidate triage group generative AI model.
However, Ahmed further teaches:
candidate triage group generative AI model
as cited above in claim 1.
Regarding Claim 6
The combination of Schreiber and Ahmed teaches all of the limitations of claim 1 as cited above, and Schreiber does not explicitly teach:
wherein the candidate triage group generative AI model is a large language model (LLM).
However, Ahmed teaches:
wherein the candidate triage group generative AI model is a large language model (LLM).
(Ahmed [Section VII. Conclusion] “With this work, we show that state-of-the-art large language
models such as GPT-3 and GPT-3.5 are effective to help with incident management, specifically, to identify root causes and mitigation steps.”;
(EN): “GPT-3 and GPT-3.5” reads on “LLM”)
Regarding Claim 7
The combination of Schreiber and Ahmed teaches all of the limitations of claim 1 as cited above, and Schreiber further teaches:
wherein generating the assignment recommendation includes
(Schreiber [col 11 lines 23 - 28] “In some embodiments, the at least one machine learning model comprises an ensemble of machine learning models including a … machine learning model to identify a team associated with the incident...”;
(EN): this paragraph denotes the team assignment recommendation; identify a team associated with the incident” reads on “assignment recommendation”)
generating a collaborative assignment recommendation from a first candidate triage group … model and at least a second candidate triage group … model by
(Schreiber [col 7 lines 56 - 67, col 8 lines 1-7] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206. By using an ensemble of models, a more accurate result may be obtained than any one model on its own. For example, the results of one model may be provided to another model (along with some or all of the input data provided to the first model) to obtain an inference result. In some embodiments, the results of multiple models may be combined by metamodel 200 to produce the predicted results 208. In some embodiments, metamodel 200 may determine an order in which models 202-206 are to process the input data, together with results of the previously executed models.”;
(EN): inferred result from multiple models reads on “collaborative assignment recommendation”)
processing the incident request using the first candidate triage group … model and the at least a second candidate triage group …model,
Schreiber [] “As shown in FIG. 2 , the models may include a service model 202 and a team model 204 and may optionally include additional models, depending on implementation needs. In some embodiments, the service model 202 can be invoked to identify the products, services, or systems of the provider network or of the customer's infrastructure (e.g., applications, instances, etc.) most likely to be responsible for the incident. The service model can receive the metrics and state data obtained by the incident orchestrator 112, as described above, and can output values for various products, services, or systems which indicate how likely that particular product, service, or system is responsible for the current incident. The service results can be used as input to team model 204 or otherwise combined with the results of team model 204. In some embodiments, metamodel 200 may also include state change model 205. State change model 205 can receive the state data obtained from incident orchestrator 112 and output values for various state changes that have occurred within the most recent window of time since the incident was detected. The values output by state change model 205 may indicate a likelihood that a given state change is associated with (e.g., a likely cause of) the current incident. In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208. As discussed, the predicted teams/services 208 may include a plurality of teams and a plurality of services, each with a corresponding score indicating the likelihood that that team or service is relevant to the current incident.”;
(EN): this paragraph denotes the processing of the incident by a first model and then at least another one; “service model” reads on first model and “team model” reads on second model;
wherein each of the first candidate triage group … model and the at least a second candidate triage group … model are trained using data associated with each respective candidate triage group.
(Schreiber [col 9 lines 51-66] “A model training system 120 may then be used to train the incident model(s) 116. 120 A machine learning model, generally, may be thought of as one or more equations that are “trained” using a set of data. In some embodiments, the model training system 120 provides ML functionalities as a web service, as discussed further below. As the model is trained, the performance of the model can be checked using verification data 312. In some embodiments, verification data 312 may include a subset of the manually labeled training data 314 that was not used for training the model. When the model performs up to a pre-determined standard on the verification data, the model is considered trained and is output by the model training system 120. In some embodiments, the resulting trained incident model(s) 116 are then provided to a model hosting system where it can receive inference requests from the incident orchestrator 112.”;
(EN): this paragraph denotes the training of each of the models)
Schreiber teaches a candidate triage group model but does not explicitly teach:
candidate triage group generative AI model.
However, Ahmed further teaches:
candidate triage group generative AI model
as cited above in claim 1.
Regarding Claim 8
Schreiber teaches:
A computing system comprising: a memory; and a processor configured to
(Schreiber [col 10 lines 59-67, col 11 line 1] “FIG. 6 is a flow diagram illustrating operations 600 of a method for automated incident triage and diagnosis according to some embodiments. Some or all of the operations 600 (or other processes described herein, or variations, and/or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions, and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executing collectively on one or more processors.”;
(EN): this paragraph computer system for triage along with a processor)
[col 4 lines 61-67; col 5 lines 1-10] “At numeral 1, incident orchestrator 112 can receive data from one or more monitoring services 108 which includes various performance metrics which characterize the performance of one or more services 110 of provider network 100. For example, the performance metrics (also referred to as performance data) may include CPU, memory usage, latency, response time, etc. and may also include availability, health, or other service quality indicators or aggregations of any or all of these metrics. In some embodiments, the monitoring service provides data which allow a user to monitor their applications, respond to system-wide performance changes, optimize resource utilization, and get a unified view of operational health. The data may include monitoring and operational data in the form of logs, metrics, and events, providing performance data for computing resources, applications, and services that run on provider network-managed resources.”;
(EN): this passage denotes metrics that are tracked including memory usage which reads on “memory” as a part of the computer system)
process an incident request using a triage engine associated with a cloud computing system,
(Schreiber [col 11 lines 9-19] “Alternatively, in some embodiments, the incident is associated with a service or component of a customer application running in a provider network. … In some embodiments, the incident data is received from a monitoring service of the provider network which requests an incident management service to analyze the incident.”;
[col 2 lines 45 - 48] “A provider network 100 (or, “cloud” provider network) provides users with the ability to use one or more of a variety of types of computing-related resources such as compute resources”;
(EN): this paragraph denotes the association of the cloud computing system with the triage engine)
to identify a candidate historical incident from an incident database,
(Schreiber [col 9 lines 30-38] “As shown in FIG. 3 , the historical incident data 300 can be provided to incident inference system 302. In some embodiments, incident inference system 302 may include a machine learning model trained, e.g., using a smaller, manually labeled training dataset. This machine learning model may be the same or similar to the machine learning models described above. In some embodiments, the incident inference system 302 can receive historical incident data 300 associated with one historical incident.”;
[col 4 lines 37-39] “The incident data repository 114 includes data related to past incidents.” (EN): “historical incident data 300 can be provided to” denotes retrieving from an incident data repository as the incident data is stored in incident data repository; incident data repository reads on “incident database”)
to identify a plurality of candidate triage group … models by processing the incident request and the candidate historical incident,
(Schreiber [col 4 lines 37-52] The incident data repository 114 includes data related to past incidents. For example, a past incident's data may include: its start time and timeline of events; an impact description including impacted services, metrics and locations; a list of engaged groups and experts; state changes that occurred to the provider network 100 within 24 hours before the incident, including code changes and deployments, customer behavior changes, infrastructure issues, etc., with a timeline of each such event; post-incident analysis indicating the root cause of the incident and other relevant details. In some embodiments, the incident data repository is associated with a continuous data collection process that regularly (e.g., continuously or periodically) updates the incident data repository 114 with similar details for new incidents and ongoing events from various sources.
[col 5 lines 18-21] “…the incident orchestrator 112 may determine that an incident is occurring. At numeral 3, while the incident is occurring, the incident orchestrator 112 can retrieve past incident data from incident data repository 114”
[col 7 lines 56 - 67, col 8 lines 1-7] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206.”;
[col 8 lines 4-7] “In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208.”;
(EN): candidate triage group model reads as model that analyzes incident details and generates candidate teams that could mitigate it; these paragraphs denote an incident orchestrator that receives past incident data in relation to the current incident and chooses which candidate triage group model (models 202-206) to execute; incident models 116 includes model coordinator and additional models 202-206; the incident model receives the historical data and then “a model coordinator 201 which coordinates the execution of one or more additional models 202-206.”; [col 5 lines 65-56, col 6 lines 1-8] “When data associated with a current incident is received, the incident model 116 can then infer the likely root cause of the incident and its most relevant details given the incidents data known at the start of the incidents and within its first minutes (e.g., impact description and metrics, and the list of events in the recent 24 hours). In some embodiments, the incident model 116 can use both Natural Language Processing (NLP) and tabular data in the given inputs to learn and infer which of the recent events and which of the potentially relevant resources and areas are most relevant to the current incident.”; this paragraph shows that the incident model is able to use the historical data and coordinates which additional model to execute which are the candidate triage group model)
to generate a first assignment recommendation from a first candidate triage group … model
(Schreiber [col 11 lines 23 - 28] “In some embodiments, the at least one machine learning model comprises an ensemble of machine learning models including a … machine learning model to identify a team associated with the incident...”; (EN): this paragraph denotes the team assignment recommendation; identify a team associated with the incident” reads on “assignment recommendation”)
by processing the incident request with the first candidate triage group … model
(Schreiber [col 12 lines 6-36] “In some embodiments, the operations include receiving a request from a monitoring service of a provider network to identify a cause of an incident affecting performance of one or more services or components, the request including performance metric data associated with the provider network, obtaining state data from an incident data repository, the state data indicating changes to the provider network over a previous time period prior to the incident, providing the performance metric data and the state data to an incident machine learning model, the incident machine learning model trained to predict a cause of incidents affecting the provider network and to predict a team to resolve the incidents affecting the provider network, receiving inference results from the incident machine learning model indicating at least one cause of the incident and at least one team to resolve the incident, and automatically executing one or more mitigation actions to mitigate the incident based at least on the inference results.
In some embodiments, automatically executing one or more mitigation actions to mitigate the incident based at least on the inference results, further includes based on a confidence value associated with the at least one team to resolve the incident, automatically engaging the at least one team to mitigate the incident. In some embodiments, automatically executing one or more mitigation actions to mitigate the incident based at least on the inference results, further includes based on a confidence value associated with the at least one team to resolve the incident, automatically sending a request to a service update manager to roll back the state change.”
(EN): this paragraph denotes processing of an incident using candidate triage group model)
using training data associated with the first candidate triage group,
(Schreiber [col 11 lines 43 - 51] “In some embodiments, the operations further include providing the past incident data to a model training system of a machine learning service of the provider network, wherein the model training system is configured to train the at least one machine learning model using the past incident data. In some embodiments, at least a portion of the past incident data is auto-labeled to indicate a cause of a corresponding past incident and details of a resolution of the corresponding past incident.”; (EN): this paragraph denotes the training process which uses training data associated with the resolution (team candidate triage group); “details of a resolution of the corresponding past incident” reads on
“training data associated with the first candidate triage group”)
to generate at least a second assignment recommendation from at least a second candidate triage group … model
(Schreiber [col 7 lines 56 - 67, col 8 lines 1-7] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206. By using an ensemble of models, a more accurate result may be obtained than any one model on its own. For example, the results of one model may be provided to another model (along with some or all of the input data provided to the first model) to obtain an inference result. In some embodiments, the results of multiple models may be combined by metamodel 200 to produce the predicted results 208. In some embodiments, metamodel 200 may determine an order in which models 202-206 are to process the input data, together with results of the previously executed models.”;
[col 8 lines 4-7] “In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208.”;
(EN): this paragraph discloses that multiple models produce triage results which reads on “at least a second assignment recommendation”)
by processing the incident request with the at least a second candidate triage group … model using training data associated with the second candidate triage group, and
Schreiber [col 8 lines 8-36] “As shown in FIG. 2 , the models may include a service model 202 and a team model 204 and may optionally include additional models, depending on implementation needs. In some embodiments, the service model 202 can be invoked to identify the products, services, or systems of the provider network or of the customer's infrastructure (e.g., applications, instances, etc.) most likely to be responsible for the incident. The service model can receive the metrics and state data obtained by the incident orchestrator 112, as described above, and can output values for various products, services, or systems which indicate how likely that particular product, service, or system is responsible for the current incident. The service results can be used as input to team model 204 or otherwise combined with the results of team model 204. In some embodiments, metamodel 200 may also include state change model 205. State change model 205 can receive the state data obtained from incident orchestrator 112 and output values for various state changes that have occurred within the most recent window of time since the incident was detected. The values output by state change model 205 may indicate a likelihood that a given state change is associated with (e.g., a likely cause of) the current incident. In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208. As discussed, the predicted teams/services 208 may include a plurality of teams and a plurality of services, each with a corresponding score indicating the likelihood that that team or service is relevant to the current incident.”; (EN): this paragraph denotes the processing of the incident by at least a second model; “service model” reads on one model and “team model” reads on another model;
to select a target triage group for triaging the incident request by processing the first assignment recommendation and the at least a second assignment recommendation using the triage engine.
Schreiber [col 8 lines 8-36] “As shown in FIG. 2 , the models may include a service model 202 and a team model 204 and may optionally include additional models, depending on implementation needs. In some embodiments, the service model 202 can be invoked to identify the products, services, or systems of the provider network or of the customer's infrastructure (e.g., applications, instances, etc.) most likely to be responsible for the incident. The service model can receive the metrics and state data obtained by the incident orchestrator 112, as described above, and can output values for various products, services, or systems which indicate how likely that particular product, service, or system is responsible for the current incident. The service results can be used as input to team model 204 or otherwise combined with the results of team model 204. In some embodiments, metamodel 200 may also include state change model 205. State change model 205 can receive the state data obtained from incident orchestrator 112 and output values for various state changes that have occurred within the most recent window of time since the incident was detected. The values output by state change model 205 may indicate a likelihood that a given state change is associated with (e.g., a likely cause of) the current incident. In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208. As discussed, the predicted teams/services 208 may include a plurality of teams and a plurality of services, each with a corresponding score indicating the likelihood that that team or service is relevant to the current incident.”;
(EN): this paragraph denotes using the results of different models (including a first and second) to output “a plurality of teams and a plurality of services”
(Schreiber [col 11 lines 29-36] “The operations 600 further include, at block 606, automatically executing the one or more mitigation actions to mitigate the incident. In some embodiments, the one or more mitigation actions include at least one of engagement of a team indicated by the at least one machine learning model to be associated with a cause of the incident or roll back of a state change indicated by the at least one machine learning model to be associated with the cause of the incident.”;
(EN): engaging a team that was predicted reads on “selecting a target triage group … by processing assignment recommendation”; this paragraph denotes the choose of a team which reads on “select[ing] a target triage group for triaging”); automatically engaging the team that can resolve the incident correctly makes mitigation more efficient reads on triaging an incident; [col 2 lines 12-21] “…incident management service helps focus and automate part of the diagnosis process, which reduces the amount of time required to identify relevant mitigation strategies and return the service(s) to operation sooner. Additionally, embodiments allow incidents to be analyzed using machine learning while the incidents are occurring. As such, rather than being applied retrospectively after incidents have already occurred and been mitigated, embodiments improve the mitigation response itself to an ongoing incident.”)
(Schreiber [col 1 lines 60-66] “The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for automated incident triage and diagnosis. According to some embodiments, an incident management service is responsible for automatically identifying possible causes of an incident, and the teams most likely able to assist in mitigating the incident, when an incident is detected.”;
[col 11 lines 4-8] “In some embodiments, one or more (or all) of the operations 600 are performed by incident management service 102, model training system 120, etc. of the other figures.”;
(EN): “incident management service” reads on “triage engine” as denoted earlier; the second passage describes all the processes being executed on incident management service 102)
Schreiber teaches a candidate triage group model but does not explicitly teach:
candidate triage group generative AI model.
However, Ahmed further teaches:
candidate triage group generative AI model
(Ahmed [Abstract] “Incident management for cloud services is a complex process involving several steps and has a huge impact on both service health and developer productivity. On-call engineers require significant amount of domain knowledge and manual effort for root causing and mitigation of production incidents. Recent advances in artificial intelligence has resulted in state-of the-art large language models like GPT-3.x (both GPT-3.0 and GPT-3.5), which have been used to solve a variety of problems ranging from question answering to text summarization. In this work, we do the first large-scale study to evaluate the effectiveness of these models for helping engineers root cause and mitigate production incidents. We do a rigorous study at Microsoft, on more than 40,000 incidents and compare several large language models ... Lastly, our human evaluation with actual incident owners show the efficacy and future potential of using artificial intelligence for resolving cloud incidents.”;
(EN): this passage denotes using GPT-3.x , which reads on “generative artificial (AI) model”, to mitigate production incidents in order to help engineers which reads on “triage” )
Before the effective date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the method of triaging an incident with the generative artificial intelligence (AI) model of Ahmed in order to generate more complex solutions for mitigation of incidents.
(Ahmed [Section III. Methodology, C. Model configuration, 2nd paragraph, page 5] “For natural language translation learning the mapping between the words from different natural languages is essential to generate good quality translation. Code summarization is slightly different from these two, where the input is much longer than the output. However, Ahmed and Devanbu found that all the necessary information for code summarization is extracted from the identifiers, and obfuscating the identifiers hurts the models [33]. Generating root causes and mitigation plans is much more complex than these problems, where the input may not contain handy information. The models need to be able to generate more diverse and creative solutions to answer the question. Our problem is more aligned with code generation problems where the input does not carry most information. For these types of problems, it is found that instead of using the encoder-decoder model, decoder-only models (e.g., GPT-3.x) are more successful where we only focus on the following tokens considering the prior tokens generated by the models. It is well-established that encoder decoder models are not as successful as decoder-only models in code generation tasks.”;
(EN): “encoder decoder models” reads on machine learning models; “decoder-only model” reads on “generative artificial intelligence (AI) model”; this paragraph denotes the usage of generative AI models instead of machine learning models)
Regarding Claim 9
Claim 9 recites substantially similar limitations for claim 2 and is therefore rejected on the same basis.
Regarding Claim 10
The combination of Schreiber and Ahmed teaches all of the limitations of claim 8 as cited above, and Schreiber further teaches:
wherein identifying the candidate historical incident includes identifying a most similar incident from the incident database.
(Schreiber [col 5 lines 59-67, col 6 lines 1 -8] “The incident model 116 may be trained on the prior incident data that has been labeled with its resolution information. This enables the incident model 116 to learn to associate specific inputs (e.g., a description of the incident, metric data associated with affected services, etc.) with the mitigation actions and/or teams that were deployed to resolve the past incidents. When data associated with a current incident is received, the incident model 116 can then infer the likely root cause of the incident and its most relevant details given the incidents data known at the start of the incidents and within its first minutes (e.g., impact description and metrics, and the list of events in the recent 24 hours). In some embodiments, the incident model 116 can use both Natural Language Processing (NLP) and tabular data in the given inputs to learn and infer which of the recent events and which of the potentially relevant resources and areas are most relevant to the current incident.”
(EN): this paragraph denotes that the incident model is able to find similar incidents and is able to find “the potentially relevant resources and areas are most relevant to the current incident” reads on “identifying a most similar incident from the incident database”)
Regarding Claim 11
The combination of Schreiber and Ahmed teaches all of the limitations of claim 10 as cited above, and Schreiber further teaches:
wherein identifying the candidate triage group … model includes processing the candidate historical incident to identify a candidate triage group associated with triaging the candidate historical incident.
(Schreiber [col 5 lines 59-67, col 6 lines 1 -8] “The incident model 116 may be trained on the prior incident data that has been labeled with its resolution information. This enables the incident model 116 to learn to associate specific inputs (e.g., a description of the incident, metric data associated with affected services, etc.) with the mitigation actions and/or teams that were deployed to resolve the past incidents. When data associated with a current incident is received, the incident model 116 can then infer the likely root cause of the incident and its most relevant details given the incidents data known at the start of the incidents and within its first minutes (e.g., impact description and metrics, and the list of events in the recent 24 hours). In some embodiments, the incident model 116 can use both Natural Language Processing (NLP) and tabular data in the given inputs to learn and infer which of the recent events and which of the potentially relevant resources and areas are most relevant to the current incident.”
(EN): this denotes the finding of a “similar” historical incident
[col 11 lines 20-23] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206.”;
[col 8 lines 4-7] “In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208.”;
(EN): this paragraph denotes the incident models 116 using the candidate historical incident data by a model coordinator 201 to execute “additional models 202-206” which are candidate triage group models since they “output the predicted teams/services”
Regarding Claim 12
Claim 12 recites substantially similar limitations for claim 5 and is therefore rejected on the same basis.
Regarding Claim 13
Claim 13 recites substantially similar limitations for claim 6 and is therefore rejected on the same basis.
Regarding Claim 14
The combination of Schreiber and Ahmed teaches all of the limitations of claim 8 as cited above, and Schreiber further teaches:
generate a collaborative assignment recommendation using the first candidate triage group generative AI model and the at least a second candidate triage group … model
(Schreiber [col 7 lines 56 - 67, col 8 lines 1-7] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206. By using an ensemble of models, a more accurate result may be obtained than any one model on its own. For example, the results of one model may be provided to another model (along with some or all of the input data provided to the first model) to obtain an inference result. In some embodiments, the results of multiple models may be combined by metamodel 200 to produce the predicted results 208. In some embodiments, metamodel 200 may determine an order in which models 202-206 are to process the input data, together with results of the previously executed models.”;
(EN): inferred result from multiple models reads on “collaborative assignment recommendation”)
by processing the incident request using each of the first candidate triage group generative AI model and the at least a second candidate triage group … model until the first candidate triage group generative AI model and the at least a second candidate triage group … model recommend the same candidate triage group for triaging the incident request.
(Schreiber [col 7 lines 56 - 67, col 8 lines 1-7] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206. By using an ensemble of models, a more accurate result may be obtained than any one model on its own. For example, the results of one model may be provided to another model (along with some or all of the input data provided to the first model) to obtain an inference result. In some embodiments, the results of multiple models may be combined by metamodel 200 to produce the predicted results 208. In some embodiments, metamodel 200 may determine an order in which models 202-206 are to process the input data, together with results of the previously executed models.”;
(EN): combined result of the automated triage method reads on recommending the same triage group”)
Schreiber teaches a candidate triage group model but does not explicitly teach:
the candidate triage group generative AI model.
However, Ahmed further teaches:
the candidate triage group generative AI model
as cited above in claim 8.
Regarding Claim 15
A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored thereon which, when executed by a processor, cause the processor to perform operations comprising:
(Schreiber [col 10 lines 59-67, col 11 line 1] “FIG. 6 is a flow diagram illustrating operations 600 of a method for automated incident triage and diagnosis according to some embodiments. Some or all of the operations 600 (or other processes described herein, or variations, and/or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions, and are implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executing collectively on one or more processors.”;
[col 1 lines 60-66] “The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for automated incident triage and diagnosis. According to some embodiments, an incident management service is responsible for automatically identifying possible causes of an incident, and the teams most likely able to assist in mitigating the incident, when an incident is detected.”;
(EN): non transitory computer readable medium for triage)
processing an incident request using a triage engine associated with a cloud computing system;
(Schreiber [col 11 lines 9-19] “Alternatively, in some embodiments, the incident is associated with a service or component of a customer application running in a provider network. … In some embodiments, the incident data is received from a monitoring service of the provider network which requests an incident management service to analyze the incident.”;
[col 2 lines 45 - 48] “A provider network 100 (or, “cloud” provider network) provides users with the ability to use one or more of a variety of types of computing-related resources such as compute resources”;
(EN): this paragraph denotes the association of the cloud computing system with the triage engine)
identifying a plurality of candidate triage group … models by processing the incident request;
(Schreiber [col 5 lines 18-21] “…the incident orchestrator 112 may determine that an incident is occurring. At numeral 3, while the incident is occurring, the incident orchestrator 112 can retrieve past incident data from incident data repository 114”
[col 7 lines 56 - 67, col 8 lines 1-7] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206.”;
[col 8 lines 4-7] “In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208.”;
(EN): candidate triage group model reads as model that analyzes incident details and generates candidate teams that could mitigate it; these paragraphs denote an incident orchestrator that receives data in relation to the current incident and chooses which candidate triage group models (models 202-206) to execute)
generating a collaborative assignment recommendation from a first candidate triage group … model and at least a second candidate triage group … model by processing the incident request with the first candidate triage group … model and the at least a second candidate triage group using training data associated with each respective candidate triage group;
(Schreiber [col 7 lines 56 - 67, col 8 lines 1-7] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206. By using an ensemble of models, a more accurate result may be obtained than any one model on its own. For example, the results of one model may be provided to another model (along with some or all of the input data provided to the first model) to obtain an inference result. In some embodiments, the results of multiple models may be combined by metamodel 200 to produce the predicted results 208. In some embodiments, metamodel 200 may determine an order in which models 202-206 are to process the input data, together with results of the previously executed models.”;
(EN): inferred result from multiple models reads on “collaborative assignment recommendation”)
Schreiber [col 8 lines 8 - 36] “As shown in FIG. 2 , the models may include a service model 202 and a team model 204 and may optionally include additional models, depending on implementation needs. In some embodiments, the service model 202 can be invoked to identify the products, services, or systems of the provider network or of the customer's infrastructure (e.g., applications, instances, etc.) most likely to be responsible for the incident. The service model can receive the metrics and state data obtained by the incident orchestrator 112, as described above, and can output values for various products, services, or systems which indicate how likely that particular product, service, or system is responsible for the current incident. The service results can be used as input to team model 204 or otherwise combined with the results of team model 204. In some embodiments, metamodel 200 may also include state change model 205. State change model 205 can receive the state data obtained from incident orchestrator 112 and output values for various state changes that have occurred within the most recent window of time since the incident was detected. The values output by state change model 205 may indicate a likelihood that a given state change is associated with (e.g., a likely cause of) the current incident. In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208. As discussed, the predicted teams/services 208 may include a plurality of teams and a plurality of services, each with a corresponding score indicating the likelihood that that team or service is relevant to the current incident.”;
(EN): this paragraph denotes the processing of the incident by a first model and then at least another one; “service model” reads on first model and “team model” reads on second model;
(Schreiber [col 9 lines 51-66] “A model training system 120 may then be used to train the incident model(s) 116. 120 A machine learning model, generally, may be thought of as one or more equations that are “trained” using a set of data. In some embodiments, the model training system 120 provides ML functionalities as a web service, as discussed further below. As the model is trained, the performance of the model can be checked using verification data 312. In some embodiments, verification data 312 may include a subset of the manually labeled training data 314 that was not used for training the model. When the model performs up to a pre-determined standard on the verification data, the model is considered trained and is output by the model training system 120. In some embodiments, the resulting trained incident model(s) 116 are then provided to a model hosting system where it can receive inference requests from the incident orchestrator 112.”;
(EN): this paragraph denotes the training of each of the models)
and selecting a target triage group for triaging the incident request by processing the collaborative assignment recommendation using the triage engine.
Schreiber [col 8 lines 8-36] “As shown in FIG. 2 , the models may include a service model 202 and a team model 204 and may optionally include additional models, depending on implementation needs. In some embodiments, the service model 202 can be invoked to identify the products, services, or systems of the provider network or of the customer's infrastructure (e.g., applications, instances, etc.) most likely to be responsible for the incident. The service model can receive the metrics and state data obtained by the incident orchestrator 112, as described above, and can output values for various products, services, or systems which indicate how likely that particular product, service, or system is responsible for the current incident. The service results can be used as input to team model 204 or otherwise combined with the results of team model 204. In some embodiments, metamodel 200 may also include state change model 205. State change model 205 can receive the state data obtained from incident orchestrator 112 and output values for various state changes that have occurred within the most recent window of time since the incident was detected. The values output by state change model 205 may indicate a likelihood that a given state change is associated with (e.g., a likely cause of) the current incident. In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208. As discussed, the predicted teams/services 208 may include a plurality of teams and a plurality of services, each with a corresponding score indicating the likelihood that that team or service is relevant to the current incident.”;
(EN): this paragraph denotes using the results of different models (including a first and second) to output “a plurality of teams and a plurality of services”
(Schreiber [col 11 lines 29-36] “The operations 600 further include, at block 606, automatically executing the one or more mitigation actions to mitigate the incident. In some embodiments, the one or more mitigation actions include at least one of engagement of a team indicated by the at least one machine learning model to be associated with a cause of the incident or roll back of a state change indicated by the at least one machine learning model to be associated with the cause of the incident.”;
(EN): engaging a team that was predicted reads on “selecting a target triage group … by processing assignment recommendation”; this paragraph denotes the choose of a team which reads on “select[ing] a target triage group for triaging”)
Schreiber teaches a candidate triage group model but does not explicitly teach:
candidate triage group generative AI model.
However, Ahmed further teaches:
candidate triage group generative AI model
(Ahmed [Abstract] “Incident management for cloud services is a complex process involving several steps and has a huge impact on both service health and developer productivity. On-call engineers require significant amount of domain knowledge and manual effort for root causing and mitigation of production incidents. Recent advances in artificial intelligence has resulted in state-of the-art large language models like GPT-3.x (both GPT-3.0 and GPT-3.5), which have been used to solve a variety of problems ranging from question answering to text summarization. In this work, we do the first large-scale study to evaluate the effectiveness of these models for helping engineers root cause and mitigate production incidents. We do a rigorous study at Microsoft, on more than 40,000 incidents and compare several large language models ... Lastly, our human evaluation with actual incident owners show the efficacy and future potential of using artificial intelligence for resolving cloud incidents.”;
(EN): this passage denotes using GPT-3.x , which reads on “generative artificial (AI) model”, to mitigate production incidents in order to help engineers which reads on “triage” )
Before the effective date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine the method of triaging an incident with the generative artificial intelligence (AI) model of Ahmed in order to generate more complex solutions for mitigation of incidents.
(Ahmed [Section III. Methodology, C. Model configuration, 2nd paragraph, page 5] “For natural language translation learning the mapping between the words from different natural languages is essential to generate good quality translation. Code summarization is slightly different from these two, where the input is much longer than the output. However, Ahmed and Devanbu found that all the necessary information for code summarization is extracted from the identifiers, and obfuscating the identifiers hurts the models [33]. Generating root causes and mitigation plans is much more complex than these problems, where the input may not contain handy information. The models need to be able to generate more diverse and creative solutions to answer the question. Our problem is more aligned with code generation problems where the input does not carry most information. For these types of problems, it is found that instead of using the encoder-decoder model, decoder-only models (e.g., GPT-3.x) are more successful where we only focus on the following tokens considering the prior tokens generated by the models. It is well-established that encoder decoder models are not as successful as decoder-only models in code generation tasks.”;
(EN): “encoder decoder models” reads on machine learning models; “decoder-only model” reads on “generative artificial intelligence (AI) model”; this paragraph denotes the usage of generative AI models instead of machine learning models)
Regarding Claim 16
Claim 16 recites substantially similar limitations for claim 2 and is therefore rejected on the same basis.
Regarding Claim 17
The combination of Schreiber and Ahmed teaches all of the limitations of claim 8 as cited above, and Schreiber further teaches:
identifying a plurality of most similar candidate historical incidents from an incident database.
(Schreiber [col 5 lines 59-67, col 6 lines 1 -8] “The incident model 116 may be trained on the prior incident data that has been labeled with its resolution information. This enables the incident model 116 to learn to associate specific inputs (e.g., a description of the incident, metric data associated with affected services, etc.) with the mitigation actions and/or teams that were deployed to resolve the past incidents. When data associated with a current incident is received, the incident model 116 can then infer the likely root cause of the incident and its most relevant details given the incidents data known at the start of the incidents and within its first minutes (e.g., impact description and metrics, and the list of events in the recent 24 hours). In some embodiments, the incident model 116 can use both Natural Language Processing (NLP) and tabular data in the given inputs to learn and infer which of the recent events and which of the potentially relevant resources and areas are most relevant to the current incident.”
(EN): this paragraph denotes that the incident model is able to find “most similar incidents” as it is able to find “the potentially relevant resources and areas are most relevant to the current incident”)
Schreiber teaches candidate triage group models but does not explicitly teach:
candidate triage group generative AI models.
However, Ahmed further teaches:
candidate triage group generative AI models
as cited above in claim 15.
Regarding Claim 18
wherein identifying the plurality of candidate triage group … models includes processing the plurality of most similar candidate historical incidents to identify a plurality of candidate triage groups associated with triaging the plurality of most similar candidate historical incidents.
(Schreiber [col 5 lines 59-67, col 6 lines 1 -8] “The incident model 116 may be trained on the prior incident data that has been labeled with its resolution information. This enables the incident model 116 to learn to associate specific inputs (e.g., a description of the incident, metric data associated with affected services, etc.) with the mitigation actions and/or teams that were deployed to resolve the past incidents. When data associated with a current incident is received, the incident model 116 can then infer the likely root cause of the incident and its most relevant details given the incidents data known at the start of the incidents and within its first minutes (e.g., impact description and metrics, and the list of events in the recent 24 hours). In some embodiments, the incident model 116 can use both Natural Language Processing (NLP) and tabular data in the given inputs to learn and infer which of the recent events and which of the potentially relevant resources and areas are most relevant to the current incident.”
(EN): this denotes the finding of a “similar” historical incident
[col 11 lines 20-23] “FIG. 2 is a diagram illustrating an example meta model for automated incident triage and diagnosis according to some embodiments. As shown in FIG. 2 , in some embodiments, incident orchestrator 112 invokes incident model(s) 116. In some embodiments, incident models 116 includes an ensemble of models. For example, the incident models 116 may include a metamodel 200 which includes a model coordinator 201 which coordinates the execution of one or more additional models 202-206.”;
[col 8 lines 4-7] “In some embodiments, the results of each model are provided to the model coordinator 201 which uses the results of the models 202-206 to output the predicted teams/services 208.”;
(EN): this paragraph denotes the incident models 116 using the candidate historical incident data by a model coordinator 201 to execute “additional models 202-206” which are candidate triage group models since they “output the predicted teams/services”)
Schreiber teaches candidate triage group models but does not explicitly teach:
candidate triage group generative AI models.
However, Ahmed further teaches:
candidate triage group generative AI models
as cited above in claim 15.
Regarding Claim 19
Claim 19 recites substantially similar limitations for claim 5 and is therefore rejected on the same basis.
Regarding Claim 20
Claim 20 recites substantially similar limitations for claim 6 and is therefore rejected on the same basis.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JIAHE NIU whose telephone number is (571)270-0152. The examiner can normally be reached 8am-5pm.
Conclusion
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Omar Fernandez Rivas can be reached at (571) 272-2589. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JIAHE NIU/Examiner, Art Unit 2128
/OMAR F FERNANDEZ RIVAS/Supervisory Patent Examiner, Art Unit 2128
Prosecution Insights