DETAILED ACTION
1. This communication is in response to the amendments filed on March 27, 2026 for Application No. 18/141,232 in which Claims 1-20 are presented for examination.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
3. The amendments filed on March 27, 2026 have been considered. Claims 1, 7-8, and 14-15 have been amended. Thus, Claims 1-20 are pending and presented for examination.
4. Applicant’s amendments filed March 27, 2026 with respect to the 35 U.S.C. 112(b) rejection have been fully considered and are persuasive. Thus, the 35 U.S.C. 112(b) rejection has been withdrawn.
5. Applicant's arguments filed March 27, 2026 with respect to the 35 U.S.C. 101 rejection have been fully considered but they are not persuasive.
Applicant’s Arguments on Pgs. 7-9 of Arguments/Remarks state:
“Claim 1 recites "analyzing the support ticket using an artificial intelligence (AI) model to generate a set of predicted resolution statistics including predicted values for each of the one or more desired statistical parameters," "querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics," (emphasis added) and "aggregating the set of explanations for each of the plurality of support tickets to generate one or more insights regarding the one or more desired statistical parameters." Claim 1 focuses on a particular method to improve AI-based support ticket triage and resource allocation by using an XAI explanation aggregation model that aggregates explanations of a predictive AI models' predictions regarding support ticket resolution statistics to diagnose and improve the performance of a support stack. Specification, paragraphs [0012] - [0014]. The specification states that the claimed invention addresses technical problems with conventional AI-based support ticket triage and resource allocation methods, stating that "understanding why certain support tickets are highly complex or difficult to resolve can also be a useful diagnostic tool for the company's support stack, since fixing identified problems can save significant amounts of time and money." Id. at paragraph [0013]. Therefore, claim 1 recites features that improve the functioning of a computer, or effect an improvement to another technology or technical field.
In addition, courts have also found that applying or using the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception (as discussed in MPEP § 2106.05(e)) can also integrate the exception into a practical application. Here, claim 1 also recites the interaction between the AI model and the XAI algorithm, namely "querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics," as recited in claim 1. Indeed, claim 1 does not merely recite the XAI algorithm as performing its usual function in a support ticket triage context, as asserted by the Office action on page 4. Instead, claim 1 recites the XAI algorithm "querying...the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics." Therefore, claim 1 recites features that apply the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
Even assuming arguendo that claim 1 is directed to an abstract idea under the Step 2A analysis, claim 1 would still be patent eligible according to the Step 2B analysis. Under the Step 2B analysis, examiners should determine whether "the element (or combination of elements) amounts to significantly more than the exception itself." A claim including "additional subject matter that is unconventional" provides "an inventive concept" and amounts to "significantly more" than the exception itself. (See, 2019 Revised Patent Subject Matter Eligibility Guidance).
Here, claim 1 recites "analyzing the support ticket using an artificial intelligence (AI) model to generate a set of predicted resolution statistics including predicted values for each of the one or more desired statistical parameters" and "querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics" (emphasis added). Indeed, claim l's recitation of the XAI algorithm "querying...the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics" does not merely recite the XAI algorithm as performing its usual function in a support ticket triage context. Instead, this feature of claim 1 is subject matter that is unconventional and provides an inventive concept, thus amounting to significantly more than the exception itself.
Based on the foregoing, Applicant respectfully submits that independent claim 1 is directed to patent eligible subject matter. Independent claims 8 and 15 recite features similar to those recited in independent claim 1, and are thus patent eligible for similar reasons. Claims 2-7, 9-14 and 16-20 depend from independent claims 1, 8 and 15 respectively, and are directed to eligible subject matter for similar reasons, as well as for the additional eligible subject matter they recite.”
Examiner respectfully disagrees. At Step 2A Prong 1, the newly added limitation “querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics” may still be performed by mathematical process, as the “querying” limitation directly states that the querying is performed using an explainable artificial intelligence (XAI) algorithm – this clearly indicates the use of mathematical processes/calculations, including LIME and/or SHAP explainable algorithms. Furthermore, “querying” in the context of the claim limitation just indicates that synthetic support tickets are fed into the AI model, using the XAI algorithm to generate a set of explanations for the set of predicted resolution statistics – again, supporting the fact that the limitation recites mathematical processes without significantly more. Although Applicant states that the “querying” of the AI model is unconventional, this is not evident by the currently drafted claim language, in which the “querying” is analogous to merely inputting synthetic support tickets into the AI model without significantly more. It is not clear, nor defined by the claim, how “querying, using an explainable artificial intelligence algorithm, the AI model with a set of synthetic support tickets […]” provides an unconventional approach which would amount to significantly more than the exception itself, when this claim language merely indicates inputting data into a model and using mathematical processes to correspondingly generate a set of explanations. This does not provide an inventive concept.
Regarding Applicant’s arguments that the currently drafted claims provide a supposed technological improvement, Examiner disagrees for substantially the same reasons as stated above. The instant claims are still very generic/broad and such a technological improvement is not recognized, nor reflected by, the currently drafted claim language.
Thus, the 35 U.S.C. 101 rejection is maintained.
6. Applicant’s arguments filed March 27, 2026 with respect to the 35 U.S.C. 103 rejection have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Note: Examiner has introduced the Ribeiro reference (“Why Should I Trust You? Explaining the Predictions of Any Classifier”) to teach the newly added limitations of the Independent claims.
Claim Rejections - 35 USC § 101
7. 35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
8. Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Step 1: Claim 1 is a method type claim. Therefore, Claims 1-7 are directed to either a process, machine, manufacture, or composition of matter.
2A Prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation by mathematical calculation but for the recitation of generic computer components, then it falls within the “Mathematical Concepts” grouping of abstract ideas.
for each of a plurality of support tickets input to the support stack: analyzing the support ticket […] to generate a set of predicted resolution statistics including predicted values for each of the one or more desired statistical parameters (mental process – other than reciting “using an artificial intelligence (AI) model”, analyzing a plurality of support tickets to generate a set of predicted resolution statistics may be performed manually by a user observing/analyzing each support ticket of the plurality and accordingly using judgement/evaluation to generate a set of predicted resolution statistics (per Applicant’s specification Par. [0018], these resolution statistics may include a predicted amount of time required to resolve the ticket, number of reassignments required to resolve the ticket, relative difficult required to resolve the tickets, whether a service level agreement was breached, etc.) including predicted values for each of the one or more desired statistical parameters, based on said analysis of each support ticket)
querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics (mathematical process – querying the AI model with a set of synthetic support tickets using an explainable artificial intelligence (XAI) algorithm to generate a set of explanations for the set of predicted resolution statistics may be performed by mathematical process, utilizing an XAI algorithm such as Shapley additive explanations (SHAP) as supported by Applicant’s specification Par. [0030]. Furthermore, the querying of the AI model is directly performed by using an XAI algorithm (mathematical process) or alternatively, the “querying […] the AI model with a set of synthetic support tickets” can also simply amount to merely adding the words “apply it” to the judicial exception at Step 2A Prong 2 and Step 2B)
aggregating the set of explanations for each of the plurality of support tickets to generate one or more insights regarding the one or more desired statistical parameters (mental process – aggregating the set of explanations to generate one or more insights regarding the one or more desired statistical parameters may be performed manually by a user observing/analyzing the set of explanations for each of the plurality of support tickets and the desired statistical parameters and accordingly using judgement/evaluation to aggregate the explanations and generate one or more insights regarding the one or more desired statistical parameters, based on said analysis)
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
receiving an indication of one or more desired statistical parameters to be optimized, the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack (Adding insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g))
[…] using an artificial intelligence (Al) model […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of applying a black box AI model to generate predictions without significantly more)
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
receiving an indication of one or more desired statistical parameters to be optimized, the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
[…] using an artificial intelligence (Al) model […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of applying a black box AI model to generate predictions without significantly more. This cannot provide an inventive concept)
For the reasons above, Claim 1 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 2-7. The additional limitations of the dependent claims are addressed below.
Regarding Claim 2:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 2 depends on.
[…] predict values for each of the one or more desired statistical parameters based on data in a support ticket (mental process – other than reciting “AI model”, predicting values for each of the one or more desired statistical parameters may be performed manually by a user observing/analyzing the data in a support ticket and accordingly using judgement/evaluation to predict values for each of the one or more desired statistical parameters based on said analysis of the support ticket)
Step 2A Prong 2 & Step 2B:
training the AI model […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner's note: high level recitation of training a machine learning model without significantly more. This cannot provide an inventive concept)
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 3:
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 3 depends on.
Step 2A Prong 2 & Step 2B:
[…] wherein the Al model is trained using a database of previously resolved support tickets and corresponding resolution statistics (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner's note: high level recitation of training a machine learning model using previously determined data without significantly more. This cannot provide an inventive concept)
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 4:
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 4 depends on.
Step 2A Prong 2 & Step 2B:
[…] wherein each explanation in the set of explanations comprises: a plurality of different words that the support ticket is comprised of; and for each of the plurality of different words, an associated cost regarding a desired statistical parameter of the one or more desired statistical parameters (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying that the explanations comprise a plurality of different words and an associated cost regarding a desired statistical parameter does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 5:
Step 2A Prong 1: See the rejection of Claim 4 above, which Claim 5 depends on.
for each word among the set of explanations, averaging the associated cost regarding the desired statistical parameter across each explanation where the word occurs (mental process – averaging the associated cost regarding the desired statistical parameter across each explanation where the word occurs may be performed manually by a user observing/analyzing each word among the set of explanations and their associated costs and accordingly averaging the relevant costs)
Step 2A Prong 2 & Step 2B:
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 6:
Step 2A Prong 1:
See the rejection of Claim 1 above, which Claim 6 depends on.
Step 2A Prong 2 & Step 2B:
wherein the set of statistical parameters comprises: an amount of time required to resolve a support ticket, a number of reassignments required to resolve the support ticket, a personnel cost required to resolve the support ticket, and an indication of whether any terms of a service level agreement (SLA) were breached (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying the statistical parameters does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 7:
Step 2A Prong 1: See the rejection of Claim 1 above, which Claim 7 depends on.
generating the set of synthetic support tickets, wherein each of the set of synthetic support tickets comprises a permutation of the support ticket (mental process – generating a set of synthetic support tickets may be performed manually by a user observing/analyzing a plurality of support tickets and accordingly using judgement/evaluation to apply a permutation/variation to each support ticket in order to generate a set of synthetic support tickets)
Step 2A Prong 2 & Step 2B:
wherein the set of explanations for the set of predicted resolution statistics of the support ticket are generated based on predicted resolution statistics generated by the AI model for each of the set of synthetic support tickets and the set of predicted resolution statistics (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying how the set of explanations are generated does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 1.
Regarding Claim 8:
Step 1: Claim 8 is a system type claim. Therefore, Claims 8-14 are directed to either a process, machine, manufacture, or composition of matter.
2A Prong 1: If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation by mathematical calculation but for the recitation of generic computer components, then it falls within the “Mathematical Concepts” grouping of abstract ideas.
[…] to predict values for each of the one or more desired statistical parameters based on data in a support ticket (mental process – other than reciting “AI model”, predicting values for each of the one or more desired statistical parameters may be performed manually by a user observing/analyzing the support tickets and their corresponding data and accordingly using judgement/evaluation to predict values for each of the desired statistical parameters based on said analysis of the data in the support ticket)
for each of a plurality of support tickets input to the support stack: analyze the support ticket using the artificial intelligence (AI) model to generate a set of predicted resolution statistics including predicted values for each of the one or more desired statistical parameters (mental process – other than reciting “using an artificial intelligence (AI) model”, analyzing a plurality of support tickets to generate a set of predicted resolution statistics may be performed manually by a user observing/analyzing each support ticket of the plurality and accordingly using judgement/evaluation to generate a set of predicted resolution statistics (per Applicant’s specification Par. [0018], these resolution statistics may include a predicted amount of time required to resolve the ticket, number of reassignments required to resolve the ticket, relative difficult required to resolve the tickets, whether a service level agreement was breached, etc.) including predicted values for each of the one or more desired statistical parameters, based on said analysis of each support ticket)
query, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics (mathematical process – querying the AI model with a set of synthetic support tickets using an explainable artificial intelligence (XAI) algorithm to generate a set of explanations for the set of predicted resolution statistics may be performed by mathematical process, utilizing an XAI algorithm such as Shapley additive explanations (SHAP) as supported by Applicant’s specification Par. [0030]. Furthermore, the querying of the AI model is directly performed by using an XAI algorithm (mathematical process) or alternatively, the “querying […] the AI model with a set of synthetic support tickets” can also simply amount to merely adding the words “apply it” to the judicial exception at Step 2A Prong 2 and Step 2B)
aggregate the set of explanations for each of the plurality of support tickets to generate one or more insights regarding the one or more desired statistical parameters (mental process – aggregating the set of explanations to generate one or more insights regarding the one or more desired statistical parameters may be performed manually by a user observing/analyzing the set of explanations for each of the plurality of support tickets and the desired statistical parameters and accordingly using judgement/evaluation to aggregate the explanations and generate one or more insights regarding the one or more desired statistical parameters, based on said analysis)
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
a system comprising: a memory; and a processing device operatively coupled to the memory […] (recited at a high-level of generality (i.e., as a generic system comprising generic memory and a generic processing device) such that it amounts to no more than mere instructions to apply the exception using generic computer components)
receive an indication of one or more desired statistical parameters to be optimized, the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack (Adding insignificant extra-solution activity to the judicial exception – see MPEP 2106.05(g))
train an artificial intelligence (AI) model […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training an AI model using previously determined data without significantly more)
[…] using the artificial intelligence (AI) model […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of applying a black box AI model to generate predictions without significantly more)
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
a system comprising: a memory; and a processing device operatively coupled to the memory […] (mere instructions to apply the exception using generic computer components cannot provide an inventive concept)
receive an indication of one or more desired statistical parameters to be optimized, the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack (MPEP 2106.05(d)(II) indicates that merely “Receiving or transmitting data over a network” is a well-understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed limitation is well-understood, routine, conventional activity is supported under Berkheimer)
train an AI model […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of training an AI model using previously determined data without significantly more. This cannot provide an inventive concept)
[…] using the artificial intelligence (AI) model […] (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level recitation of applying a black box AI model to generate predictions without significantly more. This cannot provide an inventive concept)
For the reasons above, Claim 8 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 9-14. The additional limitations of the dependent claims are addressed below.
Regarding Claim 9:
Step 2A Prong 1:
See the rejection of Claim 8 above, which Claim 9 depends on.
Step 2A Prong 2 & Step 2B:
wherein the Al model comprises a neural network (Field of Use – limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception does not amount to significantly more than the exception itself, and cannot integrate a judicial exception into a practical application; in this case specifying that the AI model comprises a neural network does not integrate the exception into a practical application nor amount to significantly more – See MPEP 2106.05(h))
Accordingly, under Step 2A Prong 2 and Step 2B, these additional elements do not integrate the abstract idea into practical application because they do not impose any meaningful limits on practicing the abstract idea, as discussed above in the rejection of claim 8.
Claim 10 recites substantially the same limitations as Claim 3, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 11 recites substantially the same limitations as Claim 4, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 12 recites substantially the same limitations as Claim 5, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 13 recites substantially the same limitations as Claim 6, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 14 recites substantially the same limitations as Claim 7, in the form of a system, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Independent Claim 15 recites substantially the same limitations as Claim 1, in the form of a non-transitory computer readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
For the reasons above, Claim 15 is rejected as being directed to an abstract idea without significantly more. This rejection applies equally to dependent claims 16-20. The additional limitations of the dependent claims are addressed below.
Claim 16 recites substantially the same limitations as Claim 2, in the form of a non-transitory computer readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 17 recites substantially the same limitations as Claim 3, in the form of a non-transitory computer readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 18 recites substantially the same limitations as Claim 4, in the form of a non-transitory computer readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 19 recites substantially the same limitations as Claim 5, in the form of a non-transitory computer readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim 20 recites substantially the same limitations as Claim 6, in the form of a non-transitory computer readable medium, including generic computer components. The claim is also directed to performing mental processes/mathematical calculations without significantly more, therefore it is rejected under the same rationale.
Claim Rejections - 35 USC § 103
9. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
10. Claims 1-3, 6, 8-10, 13, 15-17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Diaz et al. (hereinafter Diaz) (“XAI for Churn Prediction in B2B Models: A Use Case in an Enterprise Software Company”), in view of Srinivasan et al. (hereinafter Srinivasan) (US PG-PUB 20220292386), further in view of Ribeiro et al. (hereinafter Ribeiro) (“Why Should I Trust You? Explaining the Predictions of Any Classifier”).
Regarding Claim 1, Diaz teaches a method (Diaz, Pg. 1, “The objective of this work consists of the design of a methodological process that contributes to analyzing the explainability of AI algorithm predictions, Explainable Artificial Intelligence (XAI), for which we analyze the binary target variable abandonment in a B2B environment, considering the relationships that the partner (customer) has with the Contact Center, and focusing on a business software distribution company.”, thus, a method is disclosed) comprising:
receiving an indication of one or more desired statistical parameters to be optimized (Diaz, Pg. 8, “The RFID model is based on the parameters of recency, frequency, importance, and duration of interactions between the customer and Contact Center during a defined period of time [3]. This model helps us to determine the value of the customer from the point of view of their interactions with the Contact Center, as well as providing us with a segmentation and a strategy of actions to be carried out for each group of customers.”, therefore, an indication of one or more desired statistical parameters to be optimized (recency, frequency, importance, and duration (RFID) of interactions) is received. These parameters may be optimized, by the RFID and AI models, in order to determine customer value and provide insights/a strategy of action for each group of customers), the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack (Diaz discloses the parameters comprising recency, frequency, importance, and duration of interactions between the customer and contact center – however, Srinivasan is introduced below to teach where the one or more desired statistical parameters are part of a set of statistical parameters relating to performance of a support stack);
for each of a plurality of support tickets input to the support stack (Diaz, Pg. 8, “From the ticket information stored in a conventional operational CRM, the model obtains two types of recommendations for customers based on the history of their interactions with the customer service: individualized and grouped. The model is parameterized with the information provided by customer service experts”, therefore, customer support ticket information is stored in a conventional operational customer relationship management (CRM) model and a plurality of processing steps comprising the proposed methodology (See Diaz Figures 4 & 5 on Pg. 8) are completed for each of a plurality of support tickets input into the CFM):
analyzing the support ticket using an artificial intelligence (Al) model to generate a set of predicted resolution statistics including predicted values for each of the one or more desired statistical parameters (Diaz, Pg. 13, “Once the above steps are completed, some of the pre-model, data visualization, and exploration techniques are applied to explore, interpret, and gain initial insight into the dataset and thus predict churn or non-churn. The application of these techniques will help to identify the key features of the model and, being model-independent, they are applicable to any dataset and prior to any initial selection of the chosen ML model […] Once the first approximation and evaluation of the dataset has been made, we can divide it into training and test, considering that the variable x corresponds to the RFID criteria (recency, frequency, importance, and duration) and y is the variable to be predicted, i.e., customer abandonment data (yes/no)”, thus, the tickets of the CRM (processed by the preceding RFID model as shown by Figures 4 & 5 on Diaz Pg. 8) may be analyzed using an AI model (See Table 7 on Pg. 12 for list of AI models which may be used), in order to generate a set of predicted resolution statistics (customer churn or no customer churn), including the desired statistical parameters (recency, frequency, importance, and duration)); and
querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics (Diaz discloses analyzing a set of predicted resolution statistics using an XAI algorithm to generate a set of explanations per Diaz Pg. 14, Diaz does not explicitly disclose querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics – See introduction of Ribeiro reference below for explicit disclosure of this limitation); and
aggregating the set of explanations for each of the plurality of support tickets to generate one or more insights regarding the one or more desired statistical parameters (Diaz, Pg. 14, “All these evaluations will give us a global vision of the selected model and will explain which characteristics are determinant in customer abandonment, and thus guide the necessary compensatory actions to mitigate it”, therefore, the set of explanations for each of the plurality of support tickets of the CRM may be aggregated, in order to generate one or more insights (global vision used to guide compensatory actions to mitigate customer churn/abandonment) regarding the one or more desired statistical parameters (RFID parameters)).
Diaz discloses the parameters comprising recency, frequency, importance, and duration of interactions between the customer and contact center – However, Diaz does not explicitly disclose the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack
However, Srinivasan teaches the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack (Srinivasan, Abstract, “The method may include obtaining a field data of an IT support service ticket and obtaining a multi-score prediction engine. The method may further include predicting metric scores of a plurality of IT support service metrics for the support service ticket based on the field data by executing the multi-score prediction engine.”, therefore, the one or more desired statistical parameters may be part of a set of statistical parameters relating to performance of a support stack (IT support service metrics which are based on obtained field data). It is further outlined by Srinivasan Claim 15 that the plurality of IT support service metrics may comprise first-time-fix indicator, service level agreement compliance value, turn-around time, ticket reopen count, or ticket reassignment count – hence, these parameters relate directly to performance of a support stack)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of claim 1, as disclosed by Diaz to include where the one or more desired statistical parameters are part of a set of statistical parameters relating to performance of a support stack as disclosed by Srinivasan. One of ordinary skill in the art would have been motivated to make this modification to enhance performance of a support stack, by optimizing desired statistical parameters that directly relate to performance of the support stack (Srinivasan, Par. [0002], “Information technology (IT) support services are moving away from traditional service level agreement (SLA) to experience level agreement (XLA). The SLA measures the process or completion of an objective, focusing on the output of the IT support services. The XLA measures the outcome and value, focusing on end-user experience and productivity. Traditionally, IT support services have been measuring themselves on technical metrics such as availability and performance of applications and underlying infrastructure. However, in spite of a satisfactory measurement result of these technical metrics, it does not necessarily warrant good end-user experience. In addition, a customer satisfaction survey sent to an end user immediately after a support service may not accurately and timely reflect the real experience of the end user.”)
While Diaz discloses analyzing the set of predicted resolution statistics using an explainable artificial intelligence algorithm (such as SHAP or LIME) to generate a set of explanations for the set of predicted resolution statistics (See Diaz Pg. 14), Diaz in view of Srinivasan does not explicitly disclose querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics
However, Ribeiro teaches querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics (Ribeiro, Pg. 3, “We sample instances around x by drawing nonzero elements of x uniformly at random (where the number of such draws is also uniformly sampled). Given a perturbed sample z ∈ {0,1}d (which contains a fraction of the nonzero elements of x), we recover the sample in the original representation z ∈ Rd and obtain f(z), which is used as a label for the explanation model. Given this dataset Z of perturbed samples with the associated labels, we optimize Eq. (1) to get an explanation ξ(x). The primary intuition behind LIME is presented in Figure 3, where we sample instances both in the vicinity of x (which have a high weight due to πx) and far away from x (low weight from πx). Even though the original model may be too complex to explain globally, LIME presents an explanation that is locally faithful (linear in this case), where the locality is captured by πx.”, therefore, the AI model (classifier) may be queried using an XAI algorithm (LIME) with a set of synthetic/perturbed (as supported by Applicant’s specification Par. [0023]) samples (Ribeiro generally discloses explaining any classifier or regressor seemingly regardless of data type/modality – See Diaz for teaching of the samples specifically comprising support tickets) to generate a set of explanations for the set of predicted statistics)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of claim 1, as disclosed by Diaz in view of Srinivasan to include querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics, as disclosed by Ribeiro. One of ordinary skill in the art would have been motivated to make this modification to enable a robust and interpretable model, which may detect complex interactions and accordingly improve model predictions by utilizing explainable artificial intelligence (Ribeiro, Pg. 3, “Even though the original model may be too complex to explain globally, LIME presents an explanation that is locally faithful (linear in this case), where the locality is captured by πx. It is worth noting that our method is fairly robust to sampling noise since the samples are weighted by πx in Eq. (1). We now present a concrete instance of this general framework.”).
Regarding Claim 2, Diaz in view of Srinivasan in view of Ribeiro teaches the method of claim 1, further comprising: training the Al model to predict values for each of the one or more desired statistical parameters based on data in a support ticket (Diaz, Pg. 18, “When we talk about training a model, we are talking about adjusting parameters β, for which we need to define the objective function that best fits the training data xi, and produce as a response the best fitted value to yi. A notable feature of the objective functions is that they consist of two parts, the training loss, and the regularization term”, thus, the selected AI model may be trained, in order to predict values of the one or more desired statistical parameters based on data in a support ticket (RFID parameters)).
Regarding Claim 3, Diaz in view of Srinivasan in view of Ribeiro teaches the method of claim 1, wherein the Al model is trained using a database of previously resolved support tickets and corresponding resolution statistics (Diaz, Pg. 8, “From the ticket information stored in a conventional operational CRM, the model obtains two types of recommendations for customers based on the history of their interactions with the customer service: individualized and grouped. The model is parameterized with the information provided by customer service experts”, thus, the model is trained on processed RFID data using a database (See CRM customer support database in Diaz Figure 5) of previously resolved/historical support tickets and their relevant resolution statistics/parameters).
Regarding Claim 6, Diaz in view of Srinivasan in view of Ribeiro teaches the method of claim 1, wherein the set of statistical parameters comprises:
an amount of time required to resolve a support ticket, a number of reassignments required to resolve the support ticket, a personnel cost required to resolve the support ticket, and an indication of whether any terms of a service level agreement (SLA) were breached (Srinivasan, Claim 15, “The method of claim 10, wherein the plurality of IT support service metrics comprise first-time-fix indicator, service level agreement compliance value, turn-around time, ticket reopen count, or ticket reassignment count.”, thus, the set of statistical parameters include an amount of time required to resolve a support ticket (turn-around time), a number of reassignments required to resolve the ticket (ticket reassignment count), a personnel cost required to resolve the ticket (first-time-fix indicator which corresponds to whether the issue is fixed when reported for the first time, see Srinivasan Par. [0024] & instant specification Par. [0018] which equates the personnel cost to a “relative difficulty” required to resolve the ticket), and an indication of whether any terms of an SLA were breached (service level agreement compliance value)).
The reasons of obviousness have been noted in the rejection of Claim 1 above and applicable herein.
Regarding Claim 8, Diaz teaches a system (Diaz, Pg. 2, “A crucial aspect of the customer-centric philosophy is to consider that communications between the company and customer are bidirectional, and that the customer wants to be served in an integral, consistent way and through any channel. The importance of technology combined with strategy is fundamental, and systems based on customer relationship management (CRM) allow multichannel integration and therefore provide a deeper knowledge of the customer for better customer management [12].”, thus, systems are disclosed) comprising:
a memory; and a processing device operatively coupled to the memory, the processing device to (Diaz teaches a methodology process and systems for implementing the methodology process, but does not explicitly disclose wherein the system comprises a memory and processing device – Srinivasan is introduced below for teaching a system comprising a memory and processing device):
receive an indication of one or more desired statistical parameters to be optimized (Diaz, Pg. 8, “The RFID model is based on the parameters of recency, frequency, importance, and duration of interactions between the customer and Contact Center during a defined period of time [3]. This model helps us to determine the value of the customer from the point of view of their interactions with the Contact Center, as well as providing us with a segmentation and a strategy of actions to be carried out for each group of customers.”, therefore, an indication of one or more desired statistical parameters to be optimized (recency, frequency, importance, and duration (RFID) of interactions) is received. These parameters may be optimized in order to determine customer value and provide insights/a strategy of action for each group of customers), the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack (Diaz discloses the parameters comprising recency, frequency, importance, and duration of interactions between the customer and contact center – however, Srinivasan is introduced below to teach where the one or more desired statistical parameters are part of a set of statistical parameters relating to performance of a support stack);
train an artificial intelligence (AI) model to predict values for each of the one or more desired statistical parameters based on data in a support ticket (Diaz, Pg. 18, “When we talk about training a model, we are talking about adjusting parameters β, for which we need to define the objective function that best fits the training data xi, and produce as a response the best fitted value to yi. A notable feature of the objective functions is that they consist of two parts, the training loss, and the regularization term”, thus, the selected AI model may be trained, in order to predict values of the one or more desired statistical parameters based on data in a support ticket (RFID parameters));
for each of a plurality of support tickets input to the support stack (Diaz, Pg. 8, “From the ticket information stored in a conventional operational CRM, the model obtains two types of recommendations for customers based on the history of their interactions with the customer service: individualized and grouped. The model is parameterized with the information provided by customer service experts”, therefore, customer support ticket information is stored in a conventional operational customer relationship management (CRM) model and a plurality of processing steps comprising the proposed methodology (See Diaz Figures 4 & 5 on Pg. 8) are completed for each of a plurality of support tickets input into the CFM):
analyze the support ticket using the artificial intelligence (AI) model to generate a set of predicted resolution statistics including predicted values for each of the one or more desired statistical parameters (Diaz, Pg. 13, “Once the above steps are completed, some of the pre-model, data visualization, and exploration techniques are applied to explore, interpret, and gain initial insight into the dataset and thus predict churn or non-churn. The application of these techniques will help to identify the key features of the model and, being model-independent, they are applicable to any dataset and prior to any initial selection of the chosen ML model […] Once the first approximation and evaluation of the dataset has been made, we can divide it into training and test, considering that the variable x corresponds to the RFID criteria (recency, frequency, importance, and duration) and y is the variable to be predicted, i.e., customer abandonment data (yes/no)”, thus, the tickets of the CRM (processed by the preceding RFID model as shown by Figures 4 & 5 on Diaz Pg. 8) may be analyzed using an AI model (See Table 7 on Pg. 12 for list of AI models which may be used), in order to generate a set of predicted resolution statistics (customer churn or no customer churn), including the desired statistical parameters (recency, frequency, importance, and duration));; and
query, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics (Diaz discloses analyzing a set of predicted resolution statistics using an XAI algorithm to generate a set of explanations per Diaz Pg. 14, Diaz does not explicitly disclose query, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics – See introduction of Ribeiro reference below for explicit disclosure of this limitation); and
aggregate the set of explanations for each of the plurality of support tickets to generate one or more insights regarding the one or more desired statistical parameters (Diaz, Pg. 14, “All these evaluations will give us a global vision of the selected model and will explain which characteristics are determinant in customer abandonment, and thus guide the necessary compensatory actions to mitigate it”, therefore, the set of explanations for each of the plurality of support tickets of the CRM may be aggregated, in order to generate one or more insights (global vision used to guide compensatory actions to mitigate customer churn/abandonment) regarding the one or more desired statistical parameters (RFID parameters)).
Diaz does not explicitly disclose wherein the system comprises a memory; and a processing device operatively coupled to the memory […]
However, Srinivasan teaches a memory; and a processing device operatively coupled to the memory (Srinivasan, Claim 16, “A system, comprising: a memory having stored thereon executable instructions; a processor in communication with the memory, the processor when executing the instructions configured to: […]”, thus, a system comprising a memory and a processing device operatively coupled to the memory is disclosed) […]
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system, as disclosed by Diaz to include a memory and a processing device operatively coupled to the memory, as disclosed by Srinivasan. One of ordinary skill in the art would have been motivated to make this modification to enable an efficient system, comprising a memory for storing instructions and a processor for executing instructions, for analyzing computer-based support tickets (Srinivasan, Par. [0006], “In another embodiment, a system for evaluating user experience on IT support services is disclosed. The system may include a memory having stored thereon executable instructions and a processor in communication with the memory. When executing the instructions, the processor may be configured to obtain a field data of an IT support service ticket.”).
Diaz discloses the parameters comprising recency, frequency, importance, and duration of interactions between the customer and contact center – However, Diaz does not explicitly disclose the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack
However, Srinivasan teaches the one or more desired statistical parameters being part of a set of statistical parameters relating to performance of a support stack (Srinivasan, Abstract, “The method may include obtaining a field data of an IT support service ticket and obtaining a multi-score prediction engine. The method may further include predicting metric scores of a plurality of IT support service metrics for the support service ticket based on the field data by executing the multi-score prediction engine.”, therefore, the one or more desired statistical parameters may be part of a set of statistical parameters relating to performance of a support stack (IT support service metrics which are based on obtained field data). It is further outlined by Srinivasan Claim 15 that the plurality of IT support service metrics may comprise first-time-fix indicator, service level agreement compliance value, turn-around time, ticket reopen count, or ticket reassignment count – hence, these parameters relate directly to performance of a support stack)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of claim 8, as disclosed by Diaz in view of Srinivasan to include where the one or more desired statistical parameters are part of a set of statistical parameters relating to performance of a support stack as disclosed by Srinivasan. One of ordinary skill in the art would have been motivated to make this modification to enhance performance of a support stack, by optimizing desired statistical parameters that directly relate to performance of the support stack (Diaz, Par. [0002], “Information technology (IT) support services are moving away from traditional service level agreement (SLA) to experience level agreement (XLA). The SLA measures the process or completion of an objective, focusing on the output of the IT support services. The XLA measures the outcome and value, focusing on end-user experience and productivity. Traditionally, IT support services have been measuring themselves on technical metrics such as availability and performance of applications and underlying infrastructure. However, in spite of a satisfactory measurement result of these technical metrics, it does not necessarily warrant good end-user experience. In addition, a customer satisfaction survey sent to an end user immediately after a support service may not accurately and timely reflect the real experience of the end user.”)
While Diaz discloses analyzing the set of predicted resolution statistics using an explainable artificial intelligence algorithm (such as SHAP or LIME) to generate a set of explanations for the set of predicted resolution statistics (See Diaz Pg. 14), Diaz in view of Srinivasan does not explicitly disclose query, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics
However, Ribeiro teaches query, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics (Ribeiro, Pg. 3, “We sample instances around x by drawing nonzero elements of x uniformly at random (where the number of such draws is also uniformly sampled). Given a perturbed sample z ∈ {0,1}d (which contains a fraction of the nonzero elements of x), we recover the sample in the original representation z ∈ Rd and obtain f(z), which is used as a label for the explanation model. Given this dataset Z of perturbed samples with the associated labels, we optimize Eq. (1) to get an explanation ξ(x). The primary intuition behind LIME is presented in Figure 3, where we sample instances both in the vicinity of x (which have a high weight due to πx) and far away from x (low weight from πx). Even though the original model may be too complex to explain globally, LIME presents an explanation that is locally faithful (linear in this case), where the locality is captured by πx.”, therefore, the AI model (classifier) may be queried using an XAI algorithm (LIME) with a set of synthetic/perturbed (as supported by Applicant’s specification Par. [0023]) samples (Ribeiro generally discloses explaining any classifier or regressor seemingly regardless of data type/modality – See Diaz for teaching of the samples specifically comprising support tickets) to generate a set of explanations for the set of predicted statistics)
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the system of claim 8, as disclosed by Diaz in view of Srinivasan to include querying, using an explainable artificial intelligence (XAI) algorithm, the AI model with a set of synthetic support tickets to generate a set of explanations for the set of predicted resolution statistics, as disclosed by Ribeiro. One of ordinary skill in the art would have been motivated to make this modification to enable a robust and interpretable model, which may detect complex interactions and accordingly improve model predictions by utilizing explainable artificial intelligence (Ribeiro, Pg. 3, “Even though the original model may be too complex to explain globally, LIME presents an explanation that is locally faithful (linear in this case), where the locality is captured by πx. It is worth noting that our method is fairly robust to sampling noise since the samples are weighted by πx in Eq. (1). We now present a concrete instance of this general framework.”).
Regarding Claim 9, Diaz in view of Srinivasan in view of Ribeiro teaches the system of claim 8, wherein the Al model comprises a neural network (Diaz, Pg. 13, “Other algorithms could also have been used in the predictive process, such as deep neural networks [76], CatBoost, or LightGBM [77,78].”, thus, the AI model used in the predictive process may comprise a neural network).
Claim 10 recites substantially the same limitations as Claim 3, in the form of a system, therefore it is rejected under the same rationale.
Claim 13 recites substantially the same limitations as Claim 6, in the form of a system, therefore it is rejected under the same rationale.
Regarding Claim 15, Diaz in view of Srinivasan in view of Ribeiro teaches a non-transitory computer-readable medium having instructions stored thereon which, when executed by a processing device (Srinivasan, Par. [0062], “The instructions may be stored in a tangible database service medium that is other than a transitory signal, such as a flash memory, a Random Access Memory (RAM), a Read Only Memory (ROM), an Erasable Programmable Read Only Memory (EPROM); or on a magnetic or optical disc, such as a Compact Disc Read Only Memory (CDROM), Hard Disk Drive (HDD), or other magnetic or optical disk; or in or on another machine-readable medium. A product, such as a computer program product, may include a database service medium and instructions stored in or on the medium, and the instructions when executed by the circuitry in a device may cause the device to implement any of the processing described above or illustrated in the drawings.”, therefore, a non-transitory computer readable medium having instructions to be executed by a processor/processing device is disclosed) cause the processing device to: […]
The rest of the claim language in Claim 15 recites substantially the same limitations as Claim 1, in the form of a non-transitory computer-readable medium, therefore it is rejected under the same rationale.
The reasons of obviousness have been noted in the rejections of Independent Claim 1 and Independent Claim 8 above and applicable herein.
Claim 16 recites substantially the same limitations as Claim 2, in the form of a non-transitory computer-readable medium, therefore it is rejected under the same rationale.
Claim 17 recites substantially the same limitations as Claim 3, in the form of a non-transitory computer-readable medium, therefore it is rejected under the same rationale.
Claim 20 recites substantially the same limitations as Claim 6, in the form of a non-transitory computer-readable medium, therefore it is rejected under the same rationale.
11. Claims 4-5, 7, 11-12, 14, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Diaz et al. (hereinafter Diaz) (“XAI for Churn Prediction in B2B Models: A Use Case in an Enterprise Software Company”), in view of Srinivasan et al. (hereinafter Srinivasan) (US PG-PUB 20220292386), in view of Ribeiro et al. (hereinafter Ribeiro) (“Why Should I Trust You? Explaining the Predictions of Any Classifier”), further in view of Zicari et al. (hereinafter Zicari) (“Combining deep ensemble learning and explanation for intelligent ticket management”).
Regarding Claim 4, Diaz in view of Srinivasan in view of Ribeiro teaches the method of claim 1.
Diaz in view of Srinivasan in view of Ribeiro does not explicitly disclose wherein each explanation in the set of explanations comprises:
a plurality of different words that the support ticket is comprised of; and
for each of the plurality of different words, an associated cost regarding a desired statistical parameter of the one or more desired statistical parameters.
However, Zicari teaches wherein each explanation in the set of explanations comprises:
a plurality of different words that the support ticket is comprised of; and for each of the plurality of different words, an associated cost regarding a desired statistical parameter of the one or more desired statistical parameters (Zicari, Pg. 2, “To help human operators understand and check the classification suggested for a ticket 𝑥 by a discovered deep ensemble 𝑀, the framework is devised to provide them with two kinds of artifacts: (1) LIMEbased local explanations capturing the importance of ticket words (viewed as binary data features) in the class predictions yielded by 𝑀 for 𝑥; (2) local word clouds summarizing the contents of 𝑘 example tickets looking most similar to 𝑥 in the latent space learnt by (the HLFE sub-net of) 𝑀.”, thus, each explanation, produced by the local interpretable model-agnostic explanations (LIME) model, comprises a plurality of words that the support ticket is composed of, as well as an associated cost/importance of those ticket words).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of claim 1, as disclosed by Diaz in view of Srinivasan in view of Ribeiro to include wherein each explanation in the set of explanations comprises: a plurality of different words that the support ticket is comprised of; and for each of the plurality of different words, an associated cost regarding a desired statistical parameter of the one or more desired statistical parameters, as disclosed by Zicari . One of ordinary skill in the art would have been motivated to make this modification to enable the extraction of local explanations from a black box model, which may highlight the words/features of the support ticket that most influenced the model prediction, hence improving model explainability (Zicari, Pg. 2, “This gap could be bridged by exploiting existing post-hoc explanation methods (like LIME and SHAP) (Guidotti et al., 2018) to extract local explanations from such a black box model, which highlights the features of a ticket that influenced the most on the prediction returned for it by the model.”).
Regarding Claim 5, Diaz in view of Srinivasan in view of Ribeiro in view of Zicari teaches the method of claim 4, wherein aggregating the set of explanations comprises:
for each word among the set of explanations, averaging the associated cost regarding the desired statistical parameter across each explanation where the word occurs (Zicari, Pg. 9, “To provide the user with a compact view of these neighborhoods of 𝑥, a word cloud is produced for each set 𝑆 chosen among 𝑘 𝑀(𝑥) and all class-wise subsets 𝑘 𝑀(𝑥)|𝑐; the word cloud depicts each of the 𝑞 most relevant terms in 𝑆 with a size that is proportional to the relevance of the term — in our framework prototype system, 𝑞 is set to 100 by default. The relevance of each term 𝑡 for a neighbor set 𝑆, is computed as a weighted average of the TF-IDF scores of 𝑡 over the neighbors in 𝑆, where each neighbor is assigned a weight that depends on how close it is to 𝑥 in the extended latent space (the lower the distance, the higher the weight).”, thus, a word cloud is produced, comprising each word among the set of explanations and a weighted average of the associated cost across each explanation where the word occurs. This is better depicted by Figures 15 and 16 on Zicari Pg. 15).
The reasons of obviousness have been noted in the rejection of Claim 4 above and applicable herein.
Regarding Claim 7, Diaz in view of Srinivasan in view of Ribeiro teaches the method of claim 1.
While Ribeiro first introduces the local interpretable model-agnostic explanations (LIME) algorithm and the generation of synthetic samples for querying any AI/classifier model (See Ribeiro Pg. 3 Section 3.3) to generate explanations and Diaz discloses the use of the LIME algorithm, Diaz in view of Srinivasan in view of Ribeiro do not explicitly disclose generating the set of synthetic support tickets, wherein each of the set of synthetic support tickets comprises a permutation of the support ticket, and wherein the set of explanations for the set of predicted resolution statistics of the support ticket are generated based on predicted resolution statistics generated by the AI model for each of the set of synthetic support tickets and the set of predicted resolution statistics.
However, Zicari teaches generating the set of synthetic support tickets, wherein each of the set of synthetic support tickets comprises a permutation of the support ticket (Zicari, Pg. 8, “Essentially, in order to approximate the behavior of 𝑀 around the instance 𝑥, for which we want an explanation of the classification yield by 𝑀, a (sparse) linear classification model is trained via Ridge regression on a number (namely, 5000) of artificial instances, computed by perturbing 𝑥; each of these instances is labeled with the class that 𝑀 predicts for it, and it is weighted according to its proximity to 𝑥 (measured on the basis of cosine similarity), in order to make the model pay more attention to the instances closer to 𝑥 (the closer the instance, the higher the misclassification error/loss). Specifically, for each ticket class, a one-vs-all ridge classifier is computed over the 6 features (i.e., binarized versions of the ticket terms) most correlated to the class.”, thus, the instance x (which may comprise the plurality of support tickets input into the machine learning model, See Figure 5 and Section 5 on Zicari Pg. 7) may be perturbed to produce artificial instances/synthetic support tickets comprising a permutation of the original support ticket), and wherein the set of explanations for the set of predicted resolution statistics of the support ticket are generated based on predicted resolution statistics generated by the AI model for each of the set of synthetic support tickets and the set of predicted resolution statistics (Zicari, Pg. 8, “The resulting set of linear classifiers allows the user to understand how the selected terms impacted on the decision that 𝑀 took for 𝑥. These classifiers are presented in the form of visual artifacts like the ones shown in Fig. 15, which reports the LIME explanation obtained for a real-life ticket message (shown in the same figure). Each vertical bar corresponds to one of the ticket classes and shows the coefficients of the top 6 terms considered for the class. For example, in the linear sub-model of class 2, term start is associated with a weight of 0.07 — i.e., if removing this term, the membership probability of class 2 would decrease by 0.07.”, thus, a set of explanations for the set of predicted resolution statistics of the support ticket and of the synthetic/artificial support ticket are generated).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method of claim 1, as disclosed by Diaz in view of Srinivasan in view of Ribeiro include generating the set of synthetic support tickets, wherein each of the set of synthetic support tickets comprises a permutation of the support ticket, and wherein the set of explanations for the set of predicted resolution statistics of the support ticket are generated based on predicted resolution statistics generated by the AI model for each of the set of synthetic support tickets and the set of predicted resolution statistics, as disclosed by Zicari. One of ordinary skill in the art would have been motivated to make this modification to minimize misclassification error/loss and improve model understandability by identifying which features most influence a model’s predictions (Zicari, Pg. 8, “each of these instances is labeled with the class that 𝑀 predicts for it, and it is weighted according to its proximity to 𝑥 (measured on the basis of cosine similarity), in order to make the model pay more attention to the instances closer to 𝑥 (the closer the instance, the higher the misclassification error/loss). Specifically, for each ticket class, a one-vs-all ridge classifier is computed over the 6 features (i.e., binarized versions of the ticket terms) most correlated to the class. The resulting set of linear classifiers allows the user to understand how the selected terms impacted on the decision that 𝑀 took for 𝑥. These classifiers are presented in the form of visual artifacts like the ones shown in Fig. 15, which reports the LIME explanation obtained for a real-life ticket message (shown in the same figure).”).
Claim 11 recites substantially the same limitations as Claim 4, in the form of a system, therefore it is rejected under the same rationale.
Claim 12 recites substantially the same limitations as Claim 5, in the form of a system, therefore it is rejected under the same rationale.
Claim 14 recites substantially the same limitations as Claim 7, in the form of a system, therefore it is rejected under the same rationale.
Claim 18 recites substantially the same limitations as Claim 4, in the form of a non-transitory computer-readable medium, therefore it is rejected under the same rationale.
Claim 19 recites substantially the same limitations as Claim 5, in the form of a non-transitory computer-readable medium, therefore it is rejected under the same rationale.
Conclusion
12. Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
13. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Devika S Maharaj whose telephone number is (571)272-0829. The examiner can normally be reached Monday - Thursday 8:30am - 5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571)270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEVIKA S MAHARAJ/Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123
Prosecution Insights