DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 03/29/2024, 02/24/2025, 03/12/2025, 07/08/2025, 07/14/2025, 10/07/2025, 12/22/2025, and 02/17/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1,
Claim 1 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 1 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“the question being usable to establish a level of confidence in the output”
“obtaining, using at least the question and contextual data, insights intended to provide a response to the question”
“making a first determination, based on a level of success of the insights, regarding whether the insights are acceptable”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass using the question to establish a level of confidence in the output (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can establish a level of confidence in the output using the question); using the question and contextual data to obtain insights intended to provide a response to the question (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can obtain insights intended to provide a response to the question using the question and contextual data); and making a first determination regarding whether the insights are acceptable based on a level of success of the insights (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, based on a level of success of the insights, make a first determination regarding whether the insights are acceptable).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitations:
“an inference model”
“a second large language model (LLM)”
“the insights being generated by a third LLM”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). The limitations:
“obtaining a question, … the question being generated by a second large language model (LLM)”
“in a first instance of the first determination in which the insights are acceptable: providing the insights to a downstream consumer for use in interpreting the output”
As drafted, are additional elements that correspond to insignificant extra-solution activity. In particular, the additional elements are merely directed towards mere data gathering. See MPEP 2106.05(g). Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 2,
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 2 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“in a second instance of the first determination in which the insights are not acceptable: obtaining updated insights”
“making a second determination regarding whether the updated insights are acceptable”
“in a first instance of the second determination in which the updated insights are not acceptable: continuing to iteratively modify the updated insights until the modified updated insights are acceptable”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass obtaining updated insights in a second instance of the first determination wherein the insights are not acceptable (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, when the insights are not acceptable, obtain updated insights): making a second determination regarding whether the updated insights are acceptable (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can make a second determination on if the updated insights are acceptable); and iteratively modifying the updated insights until the modified updated insights are acceptable in a first instance of the second determination wherein the updated insights are not acceptable (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, when the updated insights are not acceptable, iteratively modify the updated insights until the modified updated insights are acceptable).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The recitation of additional elements in claim 1 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 1 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 3,
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 3 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“modifying the question to obtain an updated question”
“using the updated question as input … to generate the updated insights”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass modifying the question to obtain and updated question (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can obtain an updated question by modifying the question); and using the updated question to generated updated insights (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can generate updated insights using the updated question).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitation:
“the third LLM”
As drafted, is an additional element that amounts to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). In addition, the recitation of additional elements in claim 2 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 2 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 4,
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 4 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: Please see the analysis of claim 2. The limitations of claim 4 are only additional elements to the abstract ideas of claim 2.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitation:
“providing instructions to the third LLM, the instructions indicating that a portion of the contextual data is not to be used to generate the updated insights”
As drafted, is an additional element that corresponds to insignificant extra-solution activity. In particular, the additional element is merely directed towards mere data gathering. See MPEP 2106.05(g). In addition, the recitation of additional elements in claim 2 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 2 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 5,
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 5 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: Please see the analysis of claim 1. The limitations of claim 5 are only additional elements to the abstract ideas of claim 1.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitations:
“obtaining, based on at least the output, analytic data generated by a first large language model (LLM), the analytic data comprising: leading indicators from ingest data used by the inference model to generate the output; and emerging trends from the ingest data used by the inference model to identify the leading indicators”
“obtaining, using at least the analytic data and a set of question generation templates, the question generated by a second LLM”
As drafted, are additional elements that correspond to insignificant extra-solution activity. In particular, the additional elements are merely directed towards mere data gathering. See MPEP 2106.05(g). In addition, the recitation of additional elements in claim 1 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 1 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 6,
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 6 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: Please see the analysis of claim 5. The limitations of claim 6 are only additional elements to the abstract ideas of claim 5.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitations:
“feeding first ingest data into the first LLM, the first ingest data comprising: inference model ingest data used by the inference model to generate the output; the output; and a set of queries comprising questions to be answered by the first LLM, the questions being based on the inference model ingest data and the output”
“obtaining, as output from the first LLM, the analytic data”
As drafted, are additional elements that correspond to insignificant extra-solution activity. In particular, the additional elements are merely directed towards mere data gathering. See MPEP 2106.05(g). In addition, the recitation of additional elements in claim 5 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 5 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing/feeding data). Furthermore, the “obtaining …”, “providing …”, and “feeding …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 7,
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 7 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitation:
“wherein the question is usable to identify facts to establish a causal relationship between at least a portion of the output and at least a portion of the analytic data”
As drafted, under its broadest reasonable interpretations, covers mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass using the question to identify facts to establish a causal relationship between a portion of the output and a portion of the analytic data (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can identify facts to establish a causal relationship between portions of the output and analytic data using the question).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The recitation of additional elements in claim 6 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …”, “providing …”, and “feeding …” limitations of claim 6 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing/feeding data). Furthermore, the “obtaining …”, “providing …”, and “feeding …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 8,
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 8 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“comparing, …, the manual insights to the insights to obtain a similarity score, the similarity score indicating a degree of similarity between the insights and the manual insights”
“making a third determination, based on the similarity score and success criteria, regarding whether the insights are acceptable, the insights being considered acceptable when the similarity score meets the success criteria”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass comparing the manual insights to the insights to obtain a similarity score indicating a degree of similarity between the insights and the manual insights (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can obtain a similarity score indicating a degree of similarity between the insights and manual insights by comparing the insights to the manual insights to obtain the similarity score); and making a third determination regarding whether the insights are acceptable based on the similarity score and success criteria, the insights being acceptable when the similarity score meets the success criteria (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, based on the similarity score and success criteria, make a third determination of if the insights are acceptable, the insights being acceptable when the similarity score meets the success criteria).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitation:
“using a fourth LLM”
As drafted, is an additional element that amounts to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). The limitation:
“obtaining manual insights, the manual insights being generated by a subject matter expert (SME) and being based on the question and the contextual data”
As drafted, is an additional element that corresponds to insignificant extra-solution activity. In particular, the additional element is merely directed towards mere data gathering. See MPEP 2106.05(g). In addition, the recitation of additional elements in claim 1 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 1 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 9,
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 9 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“wherein the level of success indicates an extent to which the similarity score meets the success criteria”
As drafted, is part of the abstract idea of claim 8 of making a determination regarding whether the insights are acceptable based on a level of success of the insights. The limitation of claim 9 further limits the limitation of claim 8 by further defining what the level of success comprises. The above limitation in the context of this claim encompasses making a determination regarding whether the insights are acceptable based on a level of success of the insights, the level of success indicated the extent that the similarity score meets the success criteria (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, based on a level of success of the insights indicating the extent to which the similarity score meets the success criteria, make a determination regarding whether the insights are acceptable).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The recitation of additional elements in claim 8 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 8 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 10,
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 10 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: Please see the analysis of claim 9. The limitations of claim 10 are only additional elements to the abstract ideas of claim 9.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitations:
“in the first instance of the first determination in which the insights are acceptable: applying reinforced learning to the second LLM using at least the question to increase a likelihood of questions generated by the second LLM at future points in time being usable to obtain insights that meet the success criteria”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). In addition, the recitation of additional elements in claim 8 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 8 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model, generic reinforcement learning, and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 11,
Claim 11 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 11 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“wherein the contextual data comprises economic report data”
As drafted, is part of the abstract idea of claim 1 of using the question and contextual data to obtain insights. The limitation of claim 11 further limits the limitation of claim 1 by further defining what the contextual data comprises. The above limitation in the context of this claim encompasses using the question and contextual data comprising economic report data to obtain insights intended to provide a response to the question (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can obtain insights intended to provide a response to the question using the question and contextual data comprising economic report data).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The recitation of additional elements in claim 1 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 1 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 12,
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 12 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“wherein the output comprises a prediction for a condition impacting a business at a future point in time”
As drafted, is part of the abstract idea of claim 1 of using the question to establish a level of confidence in the output. The limitation of claim 12 further limits the limitation of claim 1 by further defining what the output comprises. The above limitation in the context of this claim encompasses using the question to establish a level of confidence in the output comprising a prediction for a condition impacting a business at a future point in time (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can establish a level of confidence in the output using the question, the output comprising a prediction for a condition impacting a business at a future time point).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The recitation of additional elements in claim 1 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 1 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 13,
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 13 is directed to a method, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“wherein the condition impacting the business at the future point in time is a change in availability of supply of a product from a supplier”
As drafted, is part of the abstract idea of claim 12 of using the question to establish a level of confidence in the output comprising a prediction for a condition impacting a business at a future time point. The limitation of claim 13 further limits the limitation of claim 12 by further defining what the condition comprises. The above limitation in the context of this claim encompasses using the question to establish a level of confidence in the output comprising a prediction for a condition impacting a business at a future point in time comprising a change in availability of supply of a product from a supplier (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can establish a level of confidence in the output using the question, the output comprising a prediction for a change in availability of supply of a product from a supplier).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The recitation of additional elements in claim 1 of a generic inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 1 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 14,
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 14 is directed to a non-transitory machine-readable medium, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“the question being usable to establish a level of confidence in the output”
“obtaining, using at least the question and contextual data, insights intended to provide a response to the question”
“making a first determination, based on a level of success of the insights, regarding whether the insights are acceptable”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass using the question to establish a level of confidence in the output (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can establish a level of confidence in the output using the question); using the question and contextual data to obtain insights intended to provide a response to the question (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can obtain insights intended to provide a response to the question using the question and contextual data); and making a first determination regarding whether the insights are acceptable based on a level of success of the insights (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, based on a level of success of the insights, make a first determination regarding whether the insights are acceptable).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitations:
“a processor”
“an inference model”
“a second large language model (LLM)”
“the insights being generated by a third LLM”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). The limitations:
“obtaining a question, … the question being generated by a second large language model (LLM)”
“in a first instance of the first determination in which the insights are acceptable: providing the insights to a downstream consumer for use in interpreting the output”
As drafted, are additional elements that correspond to insignificant extra-solution activity. In particular, the additional elements are merely directed towards mere data gathering. See MPEP 2106.05(g). Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic processor, inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 15,
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 15 is directed to a non-transitory machine-readable medium, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“in a second instance of the first determination in which the insights are not acceptable: obtaining updated insights”
“making a second determination regarding whether the updated insights are acceptable”
“in a first instance of the second determination in which the updated insights are not acceptable: continuing to iteratively modify the updated insights until the modified updated insights are acceptable”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass obtaining updated insights in a second instance of the first determination wherein the insights are not acceptable (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, when the insights are not acceptable, obtain updated insights): making a second determination regarding whether the updated insights are acceptable (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can make a second determination on if the updated insights are acceptable); and iteratively modifying the updated insights until the modified updated insights are acceptable in a first instance of the second determination wherein the updated insights are not acceptable (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, when the updated insights are not acceptable, iteratively modify the updated insights until the modified updated insights are acceptable).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The recitation of additional elements in claim 14 of a generic processor, inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 14 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic processor, inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 16,
Claim 16 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 16 is directed to a non-transitory machine-readable medium, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“modifying the question to obtain an updated question”
“using the updated question as input … to generate the updated insights”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass modifying the question to obtain and updated question (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can obtain an updated question by modifying the question); and using the updated question to generated updated insights (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can generate updated insights using the updated question).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitation:
“the third LLM”
As drafted, is an additional element that amounts to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). In addition, the recitation of additional elements in claim 15 of a generic processor, inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 15 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic processor, inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 17,
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 17 is directed to a non-transitory machine-readable medium, which is directed to an article of manufacture, one of the statutory categories.
Step 2A Prong One Analysis: Please see the analysis of claim 15. The limitations of claim 17 are only additional elements to the abstract ideas of claim 15.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitation:
“providing instructions to the third LLM, the instructions indicating that a portion of the contextual data is not to be used to generate the updated insights”
As drafted, is an additional element that corresponds to insignificant extra-solution activity. In particular, the additional element is merely directed towards mere data gathering. See MPEP 2106.05(g). In addition, the recitation of additional elements in claim 15 of a generic processor, inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 15 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic processor, inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 18,
Claim 18 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 18 is directed to a data processing system, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“the question being usable to establish a level of confidence in the output”
“obtaining, using at least the question and contextual data, insights intended to provide a response to the question”
“making a first determination, based on a level of success of the insights, regarding whether the insights are acceptable”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass using the question to establish a level of confidence in the output (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can establish a level of confidence in the output using the question); using the question and contextual data to obtain insights intended to provide a response to the question (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can obtain insights intended to provide a response to the question using the question and contextual data); and making a first determination regarding whether the insights are acceptable based on a level of success of the insights (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, based on a level of success of the insights, make a first determination regarding whether the insights are acceptable).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitations:
“a processor”
“a memory coupled to the processor to store instructions”
“an inference model”
“a second large language model (LLM)”
“the insights being generated by a third LLM”
As drafted, are additional elements that amount to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). The limitations:
“obtaining a question, … the question being generated by a second large language model (LLM)”
“in a first instance of the first determination in which the insights are acceptable: providing the insights to a downstream consumer for use in interpreting the output”
As drafted, are additional elements that correspond to insignificant extra-solution activity. In particular, the additional elements are merely directed towards mere data gathering. See MPEP 2106.05(g). Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic processor, memory, inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 19,
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 19 is directed to a data processing system, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“in a second instance of the first determination in which the insights are not acceptable: obtaining updated insights”
“making a second determination regarding whether the updated insights are acceptable”
“in a first instance of the second determination in which the updated insights are not acceptable: continuing to iteratively modify the updated insights until the modified updated insights are acceptable”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass obtaining updated insights in a second instance of the first determination wherein the insights are not acceptable (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, when the insights are not acceptable, obtain updated insights): making a second determination regarding whether the updated insights are acceptable (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can make a second determination on if the updated insights are acceptable); and iteratively modifying the updated insights until the modified updated insights are acceptable in a first instance of the second determination wherein the updated insights are not acceptable (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can, when the updated insights are not acceptable, iteratively modify the updated insights until the modified updated insights are acceptable).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The recitation of additional elements in claim 18 of a generic processor, memory, inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 18 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic processor, memory, inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Regarding Claim 20,
Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 20 is directed to a data processing system, which is directed to a machine, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“modifying the question to obtain an updated question”
“using the updated question as input … to generate the updated insights”
As drafted, under their broadest reasonable interpretations, cover mental processes (concepts performed in the human mind (including an observation, evaluation, judgement, opinion)) but for the recitation of mere instructions to apply language (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)). The above limitations in the context of this claim encompass modifying the question to obtain and updated question (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can obtain an updated question by modifying the question); and using the updated question to generated updated insights (corresponds to evaluation and judgement; in particular, a human, with the assistance of pen and paper, can generate updated insights using the updated question).
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recites additional elements that are mere instructions to apply (See MPEP 2106.05(f)) or insignificant extra-solution activity (See MPEP 2106.05(g)). The limitation:
“the third LLM”
As drafted, is an additional element that amounts to no more than mere instructions to apply the exception for the abstract ideas. See MPEP 2106.05(f). In addition, the recitation of additional elements in claim 19 of a generic processor, memory, inference model and LLMs, as drafted, are reciting mere instructions to apply language such that it amounts to no more than mere instructions to apply the exceptions. Furthermore, the “obtaining …” and “providing …” limitations of claim 19 are additional elements that correspond to insignificant extra-solution activity as mere data gathering. Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, all of the additional elements are “mere instructions to apply an exception” (I.e. the additional elements describe a generic processor, memory, inference model and LLMs for applying the abstract ideas) or insignificant extra-solution activity (i.e. receiving/obtaining and transmitting/providing data). Furthermore, the “obtaining …” and “providing …” limitations are insignificant extra-solution activity that is well-understood, routine, and conventional according to MPEP 2106.05(d) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity… i. Receiving or transmitting data over a network). Mere instructions to apply an exception cannot provide an inventive concept. The claim is not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-4, 11, and 14-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ranga Prasad et al. (US 2025/0265468 A1) in view of Gardner et al. (US 12,008,332 B1).
Regarding Claim 1,
Ranga Prasad et al. teaches a method of interpreting an output generated by an inference model ([0020]: "a method utilizes a comprehensive computer-implemented approach for managing IT service incidents. It starts by receiving incident data, which is then processed for validation and classification using natural language processing. Insights and potential causes of the incident are generated using a Generative AI Inference Module. A probability score is calculated to assess the risk of escalation, while the effectiveness of resolution actions is validated against standards. Preventive actions are recommended based on these analyses, and a knowledge database is updated to improve future management processes" teaches a method for generating insights (e.g. interpreting) from a classification output (inference model output)), the method comprising:
making a first determination, based on a level of success of the insights, regarding whether the insights are acceptable (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives" teaches that the insights are analyzed to determine if the insights are actionable and aligned with the objective (e.g. determine if the insights are acceptable)); and
in a first instance of the first determination in which the insights are acceptable: providing the insights to a downstream consumer for use in interpreting the output (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that actionable and aligned insights (acceptable insights) are used for recommendations. [0079]: "FIG. 6 illustrates the Problem Prevention Recommendation Module (156), detailing its process from data mining of problem root causes (600) to generating preventive measures and recommendations (608). It starts with identifying root causes, then utilizes the Generative AI Inference Module (GAIM) to derive insights (602), leading to the identification of process gaps. Based on this analysis, it formulates recommendations (604) to address these gaps, aiming to prevent the recurrence of similar problems" teaches that the recommendations generated from the insights are provided for addressing the relevant inferences (e.g. insights are provided downstream for inference output interpretation). [0041]: "Generative AI and LLMs can be used in various aspects of this disclosure performing one or more various tasks, as desired, including: … Data Analysis and Insight Generation: Including trend analysis, pattern recognition, and generating predictions and forecasts based on historical data … Decision-Making Support: Providing insights that aid in making informed decisions" teaches that generated insights are provided to aid in making informed decisions (e.g. provided to downstream consumer for making informed decisions (i.e. for interpreting the output))).
Ranga Prasad et al. does not appear to explicitly teach obtaining a question, the question being usable to establish a level of confidence in the output and the question being generated by a second large language model (LLM); obtaining, using at least the question and contextual data, insights intended to provide a response to the question, the insights being generated by a third LLM.
However, Gardner et al. teaches obtaining a question, the question being usable to establish a level of confidence in the output and the question being generated by a second large language model (LLM) (Col. 14, lines 24-30: "The system may employ advanced machine learning techniques to optimize prompt engineering for precision control over the abstraction capabilities of large language models (LLMs). On innovation is a prompt engineering model that is trained to dynamically construct prompts tailored to the nuances of the input content and desired level of abstraction" teaches obtaining a prompt (question) from a prompt engineering model (e.g. second LLM). Col. 20, lines 34-39: "the prompts can be constructed to provide transparency into the reasoning behind added inferences and external data. Specifically, the prompts can instruct the models to include confidence values, source citations, and explanatory information on why particular inferences or data points were selected" teaches that the prompt (question) is usable to establish a confidence level in the inference output. Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model);
obtaining, using at least the question and contextual data, insights intended to provide a response to the question, the insights being generated by a third LLM (Col. 3, lines 42-47: "A prompt is automatically engineered for providing to the one or more LLMs. The prompt includes a reference to the first content item and the level of the abstraction for the first content item. A response to the prompt is received from the LLM. The response includes a second content item" teaches that the prompt and content item (contextual data) are provided to an LLM (e.g. third LLM) to generate a second content item (insights). Col. 6, lines 1-3: "the content summarization services 120 can enrich the representation of the content items 128 to include additional contextual details and derived knowledge" teaches the first content item being contextual data. Col. 18, lines 62-67: "Through sufficient training data, the system learns how to automatically construct prompts tailored to each zoom level that will elicit the desired LLM behavior and high-quality summaries with the appropriate balance of abstraction, conciseness, inferences, data integration, and insight" teaches that the LLM generates insights for responding to the prompt (question). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model).
Ranga Prasad et al. and Gardner et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate obtaining a question, the question being usable to establish a level of confidence in the output and the question being generated by a second large language model (LLM); obtaining, using at least the question and contextual data, insights intended to provide a response to the question, the insights being generated by a third LLM as taught by Gardner et al. to the disclosed invention of Ranga Prasad et al.
One of ordinary skill in the art would have been motivated to make this modification because "the innovations improve abstraction capabilities, provide configurable brevity, enable iterative summarization, and maintain data integrity" (Gardner et al. Col. 2, lines 16-19).
Regarding Claim 2,
Ranga Prasad et al. in view of Gardner et al. teaches the method of claim 1.
In addition, Ranga Prasad et al. further teaches further comprising: in a second instance of the first determination in which the insights are not acceptable: obtaining updated insights (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the insights are refined (updated) to ensure the insights are actionable and aligned with the objective (e.g. to ensure the insights are acceptable) (i.e. when the insights are not actionable and aligned (not acceptable) they are refined (updated)));
making a second determination regarding whether the updated insights are acceptable (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the refined insights (updated insights) are analyzed (validated) to determine if the insights are actionable and aligned with the objective (e.g. determine if the insights are acceptable)); and
in a first instance of the second determination in which the updated insights are not acceptable: continuing to iteratively modify the updated insights until the modified updated insights are acceptable (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the insights and refined insights are iteratively refined (modified) to ensure the insights are actionable and aligned with the objective (e.g. to ensure the insights are acceptable) (i.e. when the updated insights are not actionable and aligned (not acceptable) they are refined (modified) iteratively until they are acceptable)).
Regarding Claim 3,
Ranga Prasad et al. in view of Gardner et al. teaches the method of claim 2.
In addition, Gardner et al. further teaches wherein obtaining the updated insights comprises: modifying the question to obtain an updated question (Col. 4, lines 41-44: "the trained prompt engineering model dynamically constructs prompts customized to user-specified abstraction levels. Feedback loops continue refining prompts during usage by analyzing abstraction accuracy" teaches that the prompt (question) is refined (updated/modified). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model); and
using the updated question as input for the third LLM to generate the updated insights (Col. 2, lines 12-14: "The system may also be configured to engineer iterative prompts to have the LLM summarize its own prior outputs at increasing levels of abstraction" teaches that the iterative prompt (updated question) is used as input for the LLM to generate an updated output (e.g. updated insights). Col. 18, lines 62-67: "Through sufficient training data, the system learns how to automatically construct prompts tailored to each zoom level that will elicit the desired LLM behavior and high-quality summaries with the appropriate balance of abstraction, conciseness, inferences, data integration, and insight" teaches that the LLM generates insights for responding to the prompt (question) (e.g. updated prompts result in updated insights). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model).
Ranga Prasad et al. and Gardner et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein obtaining the updated insights comprises: modifying the question to obtain an updated question; and using the updated question as input for the third LLM to generate the updated insights as taught by Gardner et al. to the disclosed invention of Ranga Prasad et al.
One of ordinary skill in the art would have been motivated to make this modification because "the innovations improve abstraction capabilities, provide configurable brevity, enable iterative summarization, and maintain data integrity" (Gardner et al. Col. 2, lines 16-19).
Regarding Claim 4,
Ranga Prasad et al. in view of Gardner et al. teaches the method of claim 2.
In addition, Ranga Prasad et al. further teaches wherein obtaining the updated insights comprises: providing instructions to the third LLM, the instructions indicating that a portion of the contextual data is not to be used to generate the updated insights ([0070]-[0071]: "The Information Layer prepares and contextualizes the data, which is then analyzed by NLP to understand and interpret the language and semantics within the data. Following this, the LLM applies deep learning algorithms to generate insights and predictions based on the processed data, further refining the output for accuracy and relevance in ITSM applications … The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the insights are refined (updated) by operating the LLM through several iterations including refining the output for relevance, the LLM refining including preparing contextual data to generate the insights (i.e. the contextual data is refined (e.g. some is not used) to refine the insight outputs to be more relevant)).
Regarding Claim 11,
Ranga Prasad et al. in view of Gardner et al. teaches the method of claim 1.
In addition, Gardner et al. further teaches wherein the contextual data comprises economic report data (Col. 10, lines 53-65: "Many external data sources can provide contextual information to augment summaries: ... Financial Data: EDGAR filings, company profiles, macroeconomic indicators, and market data provide business context" teaches that the contextual information (contextual data) comprises economic report data).
Ranga Prasad et al. and Gardner et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the contextual data comprises economic report data as taught by Gardner et al. to the disclosed invention of Ranga Prasad et al.
One of ordinary skill in the art would have been motivated to make this modification to "improve generalization by regularizing the model to capture attributes necessary for high-quality prompts beyond just mimicking human demonstrations" (Gardner et al. Col. 15, lines 28-30).
Regarding Claim 14,
Ranga Prasad et al. teaches a non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for interpreting an output generated by an inference model ([0023]: "a system for IT service incident management can comprise a processor and memory that store and execute instructions for various operations. It starts with receiving incident data, then uses an Incident Validation and Classification Module with natural language processing to classify incidents. A Generative AI Inference Module generates insights, while a Problem Probability Calculation Module assesses escalation risks. An Incident Resolution Validation Module evaluates resolution efforts, and a Problem Prevention Recommendation Module suggests preventive measures. The system updates a knowledge database with insights and recommendations to aid in incident management" teaches a system comprising a processor and memory storing instructions for execution by the processor for generating insights (e.g. interpreting) from a classification output (inference model output). [0025]: "one or more various steps or processes disclosed herein can be implemented in whole or in part as computer-executable instructions (or as computer modules or in other computer constructs) stored on computer-readable media" teaches computer-readable media (machine-readable medium) storing computer-executable instructions for performing the disclosed processes), the operations comprising:
making a first determination, based on a level of success of the insights, regarding whether the insights are acceptable (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives" teaches that the insights are analyzed to determine if the insights are actionable and aligned with the objective (e.g. determine if the insights are acceptable)); and
in a first instance of the first determination in which the insights are acceptable: providing the insights to a downstream consumer for use in interpreting the output (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that actionable and aligned insights (acceptable insights) are used for recommendations. [0079]: "FIG. 6 illustrates the Problem Prevention Recommendation Module (156), detailing its process from data mining of problem root causes (600) to generating preventive measures and recommendations (608). It starts with identifying root causes, then utilizes the Generative AI Inference Module (GAIM) to derive insights (602), leading to the identification of process gaps. Based on this analysis, it formulates recommendations (604) to address these gaps, aiming to prevent the recurrence of similar problems" teaches that the recommendations generated from the insights are provided for addressing the relevant inferences (e.g. insights are provided downstream for inference output interpretation). [0041]: "Generative AI and LLMs can be used in various aspects of this disclosure performing one or more various tasks, as desired, including: … Data Analysis and Insight Generation: Including trend analysis, pattern recognition, and generating predictions and forecasts based on historical data … Decision-Making Support: Providing insights that aid in making informed decisions" teaches that generated insights are provided to aid in making informed decisions (e.g. provided to downstream consumer for making informed decisions (i.e. for interpreting the output))).
Ranga Prasad et al. does not appear to explicitly teach obtaining a question, the question being usable to establish a level of confidence in the output and the question being generated by a second large language model (LLM); obtaining, using at least the question and contextual data, insights intended to provide a response to the question, the insights being generated by a third LLM.
However, Gardner et al. teaches obtaining a question, the question being usable to establish a level of confidence in the output and the question being generated by a second large language model (LLM) (Col. 14, lines 24-30: "The system may employ advanced machine learning techniques to optimize prompt engineering for precision control over the abstraction capabilities of large language models (LLMs). On innovation is a prompt engineering model that is trained to dynamically construct prompts tailored to the nuances of the input content and desired level of abstraction" teaches obtaining a prompt (question) from a prompt engineering model (e.g. second LLM). Col. 20, lines 34-39: "the prompts can be constructed to provide transparency into the reasoning behind added inferences and external data. Specifically, the prompts can instruct the models to include confidence values, source citations, and explanatory information on why particular inferences or data points were selected" teaches that the prompt (question) is usable to establish a confidence level in the inference output. Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model);
obtaining, using at least the question and contextual data, insights intended to provide a response to the question, the insights being generated by a third LLM (Col. 3, lines 42-47: "A prompt is automatically engineered for providing to the one or more LLMs. The prompt includes a reference to the first content item and the level of the abstraction for the first content item. A response to the prompt is received from the LLM. The response includes a second content item" teaches that the prompt and content item (contextual data) are provided to an LLM (e.g. third LLM) to generate a second content item (insights). Col. 6, lines 1-3: "the content summarization services 120 can enrich the representation of the content items 128 to include additional contextual details and derived knowledge" teaches the first content item being contextual data. Col. 18, lines 62-67: "Through sufficient training data, the system learns how to automatically construct prompts tailored to each zoom level that will elicit the desired LLM behavior and high-quality summaries with the appropriate balance of abstraction, conciseness, inferences, data integration, and insight" teaches that the LLM generates insights for responding to the prompt (question). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model).
Ranga Prasad et al. and Gardner et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate obtaining a question, the question being usable to establish a level of confidence in the output and the question being generated by a second large language model (LLM); obtaining, using at least the question and contextual data, insights intended to provide a response to the question, the insights being generated by a third LLM as taught by Gardner et al. to the disclosed invention of Ranga Prasad et al.
One of ordinary skill in the art would have been motivated to make this modification because "the innovations improve abstraction capabilities, provide configurable brevity, enable iterative summarization, and maintain data integrity" (Gardner et al. Col. 2, lines 16-19).
Regarding Claim 15,
Ranga Prasad et al. in view of Gardner et al. teaches the non-transitory machine-readable medium of claim 14.
In addition, Ranga Prasad et al. further teaches further comprising: in a second instance of the first determination in which the insights are not acceptable: obtaining updated insights (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the insights are refined (updated) to ensure the insights are actionable and aligned with the objective (e.g. to ensure the insights are acceptable) (i.e. when the insights are not actionable and aligned (not acceptable) they are refined (updated)));
making a second determination regarding whether the updated insights are acceptable (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the refined insights (updated insights) are analyzed (validated) to determine if the insights are actionable and aligned with the objective (e.g. determine if the insights are acceptable)); and
in a first instance of the second determination in which the updated insights are not acceptable: continuing to iteratively modify the updated insights until the modified updated insights are acceptable (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the insights and refined insights are iteratively refined (modified) to ensure the insights are actionable and aligned with the objective (e.g. to ensure the insights are acceptable) (i.e. when the updated insights are not actionable and aligned (not acceptable) they are refined (modified) iteratively until they are acceptable)).
Regarding Claim 16,
Ranga Prasad et al. in view of Gardner et al. teaches the non-transitory machine-readable medium of claim 15.
In addition, Gardner et al. further teaches wherein obtaining the updated insights comprises: modifying the question to obtain an updated question (Col. 4, lines 41-44: "the trained prompt engineering model dynamically constructs prompts customized to user-specified abstraction levels. Feedback loops continue refining prompts during usage by analyzing abstraction accuracy" teaches that the prompt (question) is refined (updated/modified). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model); and
using the updated question as input for the third LLM to generate the updated insights (Col. 2, lines 12-14: "The system may also be configured to engineer iterative prompts to have the LLM summarize its own prior outputs at increasing levels of abstraction" teaches that the iterative prompt (updated question) is used as input for the LLM to generate an updated output (e.g. updated insights). Col. 18, lines 62-67: "Through sufficient training data, the system learns how to automatically construct prompts tailored to each zoom level that will elicit the desired LLM behavior and high-quality summaries with the appropriate balance of abstraction, conciseness, inferences, data integration, and insight" teaches that the LLM generates insights for responding to the prompt (question) (e.g. updated prompts result in updated insights). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model).
Ranga Prasad et al. and Gardner et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein obtaining the updated insights comprises: modifying the question to obtain an updated question; and using the updated question as input for the third LLM to generate the updated insights as taught by Gardner et al. to the disclosed invention of Ranga Prasad et al.
One of ordinary skill in the art would have been motivated to make this modification because "the innovations improve abstraction capabilities, provide configurable brevity, enable iterative summarization, and maintain data integrity" (Gardner et al. Col. 2, lines 16-19).
Regarding Claim 17,
Ranga Prasad et al. in view of Gardner et al. teaches the non-transitory machine-readable medium of claim 15.
In addition, Ranga Prasad et al. further teaches wherein obtaining the updated insights comprises: providing instructions to the third LLM, the instructions indicating that a portion of the contextual data is not to be used to generate the updated insights ([0070]-[0071]: "The Information Layer prepares and contextualizes the data, which is then analyzed by NLP to understand and interpret the language and semantics within the data. Following this, the LLM applies deep learning algorithms to generate insights and predictions based on the processed data, further refining the output for accuracy and relevance in ITSM applications … The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the insights are refined (updated) by operating the LLM through several iterations including refining the output for relevance, the LLM refining including preparing contextual data to generate the insights (i.e. the contextual data is refined (e.g. some is not used) to refine the insight outputs to be more relevant)).
Regarding Claim 18,
Ranga Prasad et al. teaches a data processing system, comprising: a processor; and a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for interpreting an output generated by an inference model ([0023]: "a system for IT service incident management can comprise a processor and memory that store and execute instructions for various operations. It starts with receiving incident data, then uses an Incident Validation and Classification Module with natural language processing to classify incidents. A Generative AI Inference Module generates insights, while a Problem Probability Calculation Module assesses escalation risks. An Incident Resolution Validation Module evaluates resolution efforts, and a Problem Prevention Recommendation Module suggests preventive measures. The system updates a knowledge database with insights and recommendations to aid in incident management" teaches a system (data processing system) comprising a processor and memory storing instructions for execution by the processor for generating insights (e.g. interpreting) from a classification output (inference model output)), the operations comprising:
making a first determination, based on a level of success of the insights, regarding whether the insights are acceptable (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives" teaches that the insights are analyzed to determine if the insights are actionable and aligned with the objective (e.g. determine if the insights are acceptable)); and
in a first instance of the first determination in which the insights are acceptable: providing the insights to a downstream consumer for use in interpreting the output (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that actionable and aligned insights (acceptable insights) are used for recommendations. [0079]: "FIG. 6 illustrates the Problem Prevention Recommendation Module (156), detailing its process from data mining of problem root causes (600) to generating preventive measures and recommendations (608). It starts with identifying root causes, then utilizes the Generative AI Inference Module (GAIM) to derive insights (602), leading to the identification of process gaps. Based on this analysis, it formulates recommendations (604) to address these gaps, aiming to prevent the recurrence of similar problems" teaches that the recommendations generated from the insights are provided for addressing the relevant inferences (e.g. insights are provided downstream for inference output interpretation). [0041]: "Generative AI and LLMs can be used in various aspects of this disclosure performing one or more various tasks, as desired, including: … Data Analysis and Insight Generation: Including trend analysis, pattern recognition, and generating predictions and forecasts based on historical data … Decision-Making Support: Providing insights that aid in making informed decisions" teaches that generated insights are provided to aid in making informed decisions (e.g. provided to downstream consumer for making informed decisions (i.e. for interpreting the output))).
Ranga Prasad et al. does not appear to explicitly teach obtaining a question, the question being usable to establish a level of confidence in the output and the question being generated by a second large language model (LLM); obtaining, using at least the question and contextual data, insights intended to provide a response to the question, the insights being generated by a third LLM.
However, Gardner et al. teaches obtaining a question, the question being usable to establish a level of confidence in the output and the question being generated by a second large language model (LLM) (Col. 14, lines 24-30: "The system may employ advanced machine learning techniques to optimize prompt engineering for precision control over the abstraction capabilities of large language models (LLMs). On innovation is a prompt engineering model that is trained to dynamically construct prompts tailored to the nuances of the input content and desired level of abstraction" teaches obtaining a prompt (question) from a prompt engineering model (e.g. second LLM). Col. 20, lines 34-39: "the prompts can be constructed to provide transparency into the reasoning behind added inferences and external data. Specifically, the prompts can instruct the models to include confidence values, source citations, and explanatory information on why particular inferences or data points were selected" teaches that the prompt (question) is usable to establish a confidence level in the inference output. Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model);
obtaining, using at least the question and contextual data, insights intended to provide a response to the question, the insights being generated by a third LLM (Col. 3, lines 42-47: "A prompt is automatically engineered for providing to the one or more LLMs. The prompt includes a reference to the first content item and the level of the abstraction for the first content item. A response to the prompt is received from the LLM. The response includes a second content item" teaches that the prompt and content item (contextual data) are provided to an LLM (e.g. third LLM) to generate a second content item (insights). Col. 6, lines 1-3: "the content summarization services 120 can enrich the representation of the content items 128 to include additional contextual details and derived knowledge" teaches the first content item being contextual data. Col. 18, lines 62-67: "Through sufficient training data, the system learns how to automatically construct prompts tailored to each zoom level that will elicit the desired LLM behavior and high-quality summaries with the appropriate balance of abstraction, conciseness, inferences, data integration, and insight" teaches that the LLM generates insights for responding to the prompt (question). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model).
Ranga Prasad et al. and Gardner et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate obtaining a question, the question being usable to establish a level of confidence in the output and the question being generated by a second large language model (LLM); obtaining, using at least the question and contextual data, insights intended to provide a response to the question, the insights being generated by a third LLM as taught by Gardner et al. to the disclosed invention of Ranga Prasad et al.
One of ordinary skill in the art would have been motivated to make this modification because "the innovations improve abstraction capabilities, provide configurable brevity, enable iterative summarization, and maintain data integrity" (Gardner et al. Col. 2, lines 16-19).
Regarding Claim 19,
Ranga Prasad et al. in view of Gardner et al. teaches the data processing system of claim 18.
In addition, Ranga Prasad et al. further teaches further comprising: in a second instance of the first determination in which the insights are not acceptable: obtaining updated insights (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the insights are refined (updated) to ensure the insights are actionable and aligned with the objective (e.g. to ensure the insights are acceptable) (i.e. when the insights are not actionable and aligned (not acceptable) they are refined (updated)));
making a second determination regarding whether the updated insights are acceptable (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the refined insights (updated insights) are analyzed (validated) to determine if the insights are actionable and aligned with the objective (e.g. determine if the insights are acceptable)); and
in a first instance of the second determination in which the updated insights are not acceptable: continuing to iteratively modify the updated insights until the modified updated insights are acceptable (Fig. 4; [0071]: "The Processing Layer (410) takes the insights and predictions generated by the Large Language Model (LLM) (408) and applies further analysis to refine and validate these outputs. It uses advanced algorithms to process the LLM's output, integrating it with existing data models and frameworks to ensure the generated insights are actionable and aligned with ITSM objectives … In other words, the Processing Layer takes the harmonized data from the previous stage and applies Natural Language Processing (NLP) to distill summaries and extract meaning from the text. This processed information is then advanced to a Large Language Model (LLM), which generates sophisticated inferences. The LLM operates through several iterations, each time refining and validating the results to ensure accuracy and relevance, thereby enhancing the decision-making process within IT service management through deep learning insights" teaches that the insights and refined insights are iteratively refined (modified) to ensure the insights are actionable and aligned with the objective (e.g. to ensure the insights are acceptable) (i.e. when the updated insights are not actionable and aligned (not acceptable) they are refined (modified) iteratively until they are acceptable)).
Regarding Claim 20,
Ranga Prasad et al. in view of Gardner et al. teaches the data processing system of claim 19.
In addition, Gardner et al. further teaches wherein obtaining the updated insights comprises: modifying the question to obtain an updated question (Col. 4, lines 41-44: "the trained prompt engineering model dynamically constructs prompts customized to user-specified abstraction levels. Feedback loops continue refining prompts during usage by analyzing abstraction accuracy" teaches that the prompt (question) is refined (updated/modified). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model); and
using the updated question as input for the third LLM to generate the updated insights (Col. 2, lines 12-14: "The system may also be configured to engineer iterative prompts to have the LLM summarize its own prior outputs at increasing levels of abstraction" teaches that the iterative prompt (updated question) is used as input for the LLM to generate an updated output (e.g. updated insights). Col. 18, lines 62-67: "Through sufficient training data, the system learns how to automatically construct prompts tailored to each zoom level that will elicit the desired LLM behavior and high-quality summaries with the appropriate balance of abstraction, conciseness, inferences, data integration, and insight" teaches that the LLM generates insights for responding to the prompt (question) (e.g. updated prompts result in updated insights). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model).
Ranga Prasad et al. and Gardner et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein obtaining the updated insights comprises: modifying the question to obtain an updated question; and using the updated question as input for the third LLM to generate the updated insights as taught by Gardner et al. to the disclosed invention of Ranga Prasad et al.
One of ordinary skill in the art would have been motivated to make this modification because "the innovations improve abstraction capabilities, provide configurable brevity, enable iterative summarization, and maintain data integrity" (Gardner et al. Col. 2, lines 16-19).
Claims 8-10 are rejected under 35 U.S.C. 103 as being unpatentable over Ranga Prasad et al. (US 2025/0265468 A1) in view of Gardner et al. (US 12,008,332 B1) and further in view of Hale et al. (US 2025/0298997 A1).
Regarding Claim 8,
Ranga Prasad et al. in view of Gardner et al. teaches the method of claim 1.
Ranga Prasad et al. in view of Gardner et al. does not appear to explicitly teach wherein making the first determination comprises: obtaining manual insights, the manual insights being generated by a subject matter expert (SME) and being based on the question and the contextual data; comparing, using a fourth LLM, the manual insights to the insights to obtain a similarity score, the similarity score indicating a degree of similarity between the insights and the manual insights; making a third determination, based on the similarity score and success criteria, regarding whether the insights are acceptable, the insights being considered acceptable when the similarity score meets the success criteria.
However, Hale et al. teaches wherein making the first determination comprises: obtaining manual insights, the manual insights being generated by a subject matter expert (SME) and being based on the question and the contextual data ([0072]: "the apparatus 200 includes means, such as training engine 212, or the like, for fine-tuning the annotation model based on a comparison of the one or more model-generated annotations and one or more ground-truth annotations from the historical call transcript in the validation subset. Upon obtaining the model-generated annotations for a given historical call transcript, the training engine 212 may retrieve the corresponding ground-truth annotations for the historical call transcript from the history recorder repository 106. The training engine 212 may employ one or more comparison techniques to compare the one or more model-generated annotations against the one or more ground-truth annotations" teaches obtaining ground-truth annotations (manual insights) corresponding to the annotations (insights) generated by the annotation model (LLM). [0004]: "the annotation model may prepare the call transcript and effectively set-the stage for future data analytics insights that may utilize the call transcript. In particular, the annotation model may be a large language model that is capable of consideration of terms included in the call transcript, the contextual information surrounding the terms, and a deeper analysis of the call transcript as a whole to determine the one or more annotations for the call transcript" teaches that the annotation model is an LLM that generates annotations for future data analytics insights based on contextual information (contextual data). [0067]: "the apparatus 200 includes means, such as training engine 212, or the like, for fine-tuning the base annotation model using the training subset, wherein the training subset comprises (a) annotated historical call transcripts, (b) each annotated call transcript is annotated with one or more ground-truth annotations, and (c) each ground-truth annotation corresponds to one or more attributes of interest. A ground-truth annotation refers to an annotation that has been manually labeled and/or verified by a subject matter expert. A ground-truth annotation serves as a reference against which the performance of a machine learning model is evaluated" teaches that the ground-truth annotations (manual insights) are generated by a subject matter expert);
comparing, using a fourth LLM, the manual insights to the insights to obtain a similarity score, the similarity score indicating a degree of similarity between the insights and the manual insights ([0072]: "The training engine 212 may employ one or more comparison techniques to compare the one or more model-generated annotations against the one or more ground-truth annotations. In some embodiments, the training engine 212 may use a loss function (e.g., cross-entropy loss) to quantify the dissimilarity between the model-generated annotations and the ground truth annotations. The loss value may then be used by the training engine 212 for backpropagation through the neural network … the training engine 212 may use comparison metrics such as precision, recall, F1 score, or accuracy for a detailed evaluation of the annotation model's performance. Precision refers to the ratio of true positive predictions to the sum of true positives and false positives. In other words, precision measures the accuracy of positive predictions made by the annotation model, indicating how many of the predicted positive instances are actually true positives. The recall comparison metric may be calculated as the ratio of true positive predictions to the sum of true positives and false negatives that measures the ability of the annotation model to capture all actual positive instances. The F1 score refers to the harmonic mean of precision and recall, providing a balanced measure of the annotation model's performance, which may especially be useful in scenarios with imbalanced class distributions. The accuracy comparison metric refers to the overall correctness of the model-generated annotations and is calculated as the ratio of correctly predicted instances to the total number of instances. Depending on the comparison metric used, the training engine 212 may classify a model-generated annotation as being a true positive, false positive, false negative, or a true negative. These comparison metrics may collectively offer insights into the strengths and weaknesses of the annotation model, guiding the refinement routine to improve the overall performance of the annotation model" teaches comparing the model-generated annotations (insights) to the ground-truth annotations (manual insights) to determine a similarity (similarity score) between the model-generated annotations (insights) and the ground-truth annotations (manual insights). [0004]: "the annotation model may prepare the call transcript and effectively set-the stage for future data analytics insights that may utilize the call transcript. In particular, the annotation model may be a large language model that is capable of consideration of terms included in the call transcript, the contextual information surrounding the terms, and a deeper analysis of the call transcript as a whole to determine the one or more annotations for the call transcript" teaches that the annotation model is an LLM that generates annotations for future data analytics insights based on contextual information (contextual data). [0067]: "the apparatus 200 includes means, such as training engine 212, or the like, for fine-tuning the base annotation model using the training subset, wherein the training subset comprises (a) annotated historical call transcripts, (b) each annotated call transcript is annotated with one or more ground-truth annotations, and (c) each ground-truth annotation corresponds to one or more attributes of interest. A ground-truth annotation refers to an annotation that has been manually labeled and/or verified by a subject matter expert. A ground-truth annotation serves as a reference against which the performance of a machine learning model is evaluated" teaches that the ground-truth annotations (manual insights) are generated by a subject matter expert);
making a third determination, based on the similarity score and success criteria, regarding whether the insights are acceptable, the insights being considered acceptable when the similarity score meets the success criteria ([0072]: "The training engine 212 may employ one or more comparison techniques to compare the one or more model-generated annotations against the one or more ground-truth annotations. In some embodiments, the training engine 212 may use a loss function (e.g., cross-entropy loss) to quantify the dissimilarity between the model-generated annotations and the ground truth annotations. The loss value may then be used by the training engine 212 for backpropagation through the neural network … the training engine 212 may use comparison metrics such as precision, recall, F1 score, or accuracy for a detailed evaluation of the annotation model's performance. Precision refers to the ratio of true positive predictions to the sum of true positives and false positives. In other words, precision measures the accuracy of positive predictions made by the annotation model, indicating how many of the predicted positive instances are actually true positives. The recall comparison metric may be calculated as the ratio of true positive predictions to the sum of true positives and false negatives that measures the ability of the annotation model to capture all actual positive instances. The F1 score refers to the harmonic mean of precision and recall, providing a balanced measure of the annotation model's performance, which may especially be useful in scenarios with imbalanced class distributions. The accuracy comparison metric refers to the overall correctness of the model-generated annotations and is calculated as the ratio of correctly predicted instances to the total number of instances. Depending on the comparison metric used, the training engine 212 may classify a model-generated annotation as being a true positive, false positive, false negative, or a true negative. These comparison metrics may collectively offer insights into the strengths and weaknesses of the annotation model, guiding the refinement routine to improve the overall performance of the annotation model" teaches using a comparison metric (success criteria) to determine, based on the similarity (similarity score) between the model-generated annotations (insights) and the ground-truth annotations (manual insights), the correctness of the annotations (insights) (e.g. annotations/insights are acceptable/correct when the similarity meets the comparison metric/criteria)).
Ranga Prasad et al., Gardner et al., and Hale et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein making the first determination comprises: obtaining manual insights, the manual insights being generated by a subject matter expert (SME) and being based on the question and the contextual data; comparing, using a fourth LLM, the manual insights to the insights to obtain a similarity score, the similarity score indicating a degree of similarity between the insights and the manual insights; making a third determination, based on the similarity score and success criteria, regarding whether the insights are acceptable, the insights being considered acceptable when the similarity score meets the success criteria as taught by Hale et al. to the disclosed invention of Ranga Prasad et al. in view of Gardner et al.
One of ordinary skill in the art would have been motivated to make this modification because the "embodiments rely on computational methods that obviate the use of manual methods of analysis to generate a targeted analytics report, thereby eliminating the potential for human error" (Hale et al. [0090]).
Regarding Claim 9,
Ranga Prasad et al. in view of Gardner et al. and further in view of Hale et al. teaches the method of claim 8.
In addition, Hale et al. further teaches wherein the level of success indicates an extent to which the similarity score meets the success criteria ([0072]: "The training engine 212 may employ one or more comparison techniques to compare the one or more model-generated annotations against the one or more ground-truth annotations. In some embodiments, the training engine 212 may use a loss function (e.g., cross-entropy loss) to quantify the dissimilarity between the model-generated annotations and the ground truth annotations. The loss value may then be used by the training engine 212 for backpropagation through the neural network … the training engine 212 may use comparison metrics such as precision, recall, F1 score, or accuracy for a detailed evaluation of the annotation model's performance. Precision refers to the ratio of true positive predictions to the sum of true positives and false positives. In other words, precision measures the accuracy of positive predictions made by the annotation model, indicating how many of the predicted positive instances are actually true positives. The recall comparison metric may be calculated as the ratio of true positive predictions to the sum of true positives and false negatives that measures the ability of the annotation model to capture all actual positive instances. The F1 score refers to the harmonic mean of precision and recall, providing a balanced measure of the annotation model's performance, which may especially be useful in scenarios with imbalanced class distributions. The accuracy comparison metric refers to the overall correctness of the model-generated annotations and is calculated as the ratio of correctly predicted instances to the total number of instances. Depending on the comparison metric used, the training engine 212 may classify a model-generated annotation as being a true positive, false positive, false negative, or a true negative. These comparison metrics may collectively offer insights into the strengths and weaknesses of the annotation model, guiding the refinement routine to improve the overall performance of the annotation model" teaches using a comparison metric (success criteria) to determine, based on the similarity (similarity score) between the model-generated annotations (insights) and the ground-truth annotations (manual insights), the level of correctness (level of success) of the annotations (insights) (e.g. the level of correctness (level of success) of the annotations depends on the extent to which the similarity meets the comparison metric/criteria)).
Ranga Prasad et al., Gardner et al., and Hale et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the level of success indicates an extent to which the similarity score meets the success criteria as taught by Hale et al. to the disclosed invention of Ranga Prasad et al. in view of Gardner et al.
One of ordinary skill in the art would have been motivated to make this modification because the "embodiments rely on computational methods that obviate the use of manual methods of analysis to generate a targeted analytics report, thereby eliminating the potential for human error" (Hale et al. [0090]).
Regarding Claim 10,
Ranga Prasad et al. in view of Gardner et al. and further in view of Hale et al. teaches the method of claim 9.
In addition, Gardner et al. further teaches further comprising: in the first instance of the first determination in which the insights are acceptable: applying reinforced learning to the second LLM using at least the question to increase a likelihood of questions generated by the second LLM at future points in time being usable to obtain insights that meet the success criteria (Col. 18, lines 46-67: "During training, the system learns how different prompts impact the LLM behavior and summary quality at each zoom level. The training process may involve one or more of the following: … Using reinforcement learning to associate prompt attributes with summary outcomes to determine optimal prompts; Through sufficient training data, the system learns how to automatically construct prompts tailored to each zoom level that will elicit the desired LLM behavior and high-quality summaries with the appropriate balance of abstraction, conciseness, inferences, data integration, and insight" teaches applying reinforcement learning to the LLM to increase the likelihood of constructed (generated) prompts (questions) at future points being optimal prompts for obtaining desired LLM behavior and high-quality insights (e.g. insights that meet the success criteria). Col. 7, lines 47-50: "For example, a single prompt could provide the text to the LLM for overall summarization, ask QA models for key details, have classifiers tag topics, retrievers augment with external data, and sentiment analysis score tone" teaches that the prompt may be a question for a QA LLM model).
Ranga Prasad et al., Gardner et al., and Hale et al. are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate further comprising: in the first instance of the first determination in which the insights are acceptable: applying reinforced learning to the second LLM using at least the question to increase a likelihood of questions generated by the second LLM at future points in time being usable to obtain insights that meet the success criteria as taught by Gardner et al. to the disclosed invention of Ranga Prasad et al. in view of Hale et al.
One of ordinary skill in the art would have been motivated to make this modification to "improve generalization by regularizing the model to capture attributes necessary for high-quality prompts beyond just mimicking human demonstrations" (Gardner et al. Col. 15, lines 28-30).
Claims 12 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Ranga Prasad et al. (US 2025/0265468 A1) in view of Gardner et al. (US 12,008,332 B1) and further in view of Kuan (US 2024/0362409 A1).
Regarding Claim 12,
Ranga Prasad et al. in view of Gardner et al. teaches the method of claim 1.
Ranga Prasad et al. in view of Gardner et al. does not appear to explicitly teach wherein the output comprises a prediction for a condition impacting a business at a future point in time.
However, Kuan teaches wherein the output comprises a prediction for a condition impacting a business at a future point in time (Fig. 1; Fig. 2; [0045]-[0047]: "The task classifier 206 employs a classifier prompt generation engine 208 to generate a classifier prompt 210 based on the user input 204, and the classifier prompt 210 is input to a classifier Large Language Model (“LLM”) 212. A Large Language Model is a model of the probability distribution over sequences of text, such as a deep learning neural network model that is pretrained on a large amount of text to understand written language. The classifier LLM 212 processes the classifier prompt 210 to generate the selected task 202 associated with the user input 204 … The selected task 202, for instance, may be one of descriptive analytics (e.g., generating metrics and reports describing the state of a business associated with the data), predictive analytics (e.g., training machine learning models to make predictions, such as propensity modeling and so forth), prescriptive analytics (e.g., training models that give recommendations on next-best-action for a business to take), cluster analysis (e.g., using machine learning to find groupings in the data, such as segmentation and so forth), correlation or association analysis (e.g., measuring association between variables), causal interference analysis (e.g., estimating causal effects), and so forth … generate a selected task 202 of “propensity modeling predictive analytics”, and the insight generation system 114, the insight presentation system 116, or the insight interaction system 118 perform processes associated with propensity modeling and predictive analytics to generate predicted future data" teaches that the classifier output (inference output) comprises a prediction of future data for analytics of a business (condition impacting a business) at a future time based on the input data associated with the business).
Ranga Prasad et al., Gardner et al., and Kuan are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the output comprises a prediction for a condition impacting a business at a future point in time as taught by Kuan to the disclosed invention of Ranga Prasad et al. in view of Gardner et al.
One of ordinary skill in the art would have been motivated to make this modification "to leverage machine learning or artificial intelligence to identify insights within a dataset and transform the insights into formats easily consumable by human users, and without any prior knowledge or identification of data structure, format, or sources. These techniques allow for real-time or near real-time analysis of a wide range of datasets with unknown contents, which is not possible using conventional data analysis techniques" (Kuan [0042]).
Regarding Claim 13,
Ranga Prasad et al. in view of Gardner et al. and further in view of Kuan teaches the method of claim 12.
In addition, Kuan further teaches wherein the condition impacting the business at the future point in time is a change in availability of supply of a product from a supplier (Fig. 1; Fig. 2; [0045]-[0047]: "The task classifier 206 employs a classifier prompt generation engine 208 to generate a classifier prompt 210 based on the user input 204, and the classifier prompt 210 is input to a classifier Large Language Model (“LLM”) 212. A Large Language Model is a model of the probability distribution over sequences of text, such as a deep learning neural network model that is pretrained on a large amount of text to understand written language. The classifier LLM 212 processes the classifier prompt 210 to generate the selected task 202 associated with the user input 204 … The selected task 202, for instance, may be one of descriptive analytics (e.g., generating metrics and reports describing the state of a business associated with the data), predictive analytics (e.g., training machine learning models to make predictions, such as propensity modeling and so forth), prescriptive analytics (e.g., training models that give recommendations on next-best-action for a business to take), cluster analysis (e.g., using machine learning to find groupings in the data, such as segmentation and so forth), correlation or association analysis (e.g., measuring association between variables), causal interference analysis (e.g., estimating causal effects), and so forth … generate a selected task 202 of “propensity modeling predictive analytics”, and the insight generation system 114, the insight presentation system 116, or the insight interaction system 118 perform processes associated with propensity modeling and predictive analytics to generate predicted future data" teaches that the classifier output (inference output) comprises a prediction of future data for analytics of a business (condition impacting a business) at a future time based on the input data associated with the business. [0036]: "The dataset 108 describes data stored in a database. In one example, the dataset 108 includes data collected by a business or organization and can take many forms such as data pertaining to customer information, sales data, financial data, inventory data, employee data, marketing data, website analytics data, supply chain data, production data, social media data, customer service data, risk management data, and so forth" teaches that the data associated with the business includes supply chain data (e.g. the future prediction may be data associated with a change in the supply change from a supplier)).
Ranga Prasad et al., Gardner et al., and Kuan are analogous to the claimed invention because they are directed towards machine learning LLM insight generation.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate wherein the condition impacting the business at the future point in time is a change in availability of supply of a product from a supplier as taught by Kuan to the disclosed invention of Ranga Prasad et al. in view of Gardner et al.
One of ordinary skill in the art would have been motivated to make this modification "to leverage machine learning or artificial intelligence to identify insights within a dataset and transform the insights into formats easily consumable by human users, and without any prior knowledge or identification of data structure, format, or sources. These techniques allow for real-time or near real-time analysis of a wide range of datasets with unknown contents, which is not possible using conventional data analysis techniques" (Kuan [0042]).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN J HALES whose telephone number is (571)272-0878. The examiner can normally be reached M-F 9:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BRIAN J HALES/Examiner, Art Unit 2125
/KAMRAN AFSHAR/Supervisory Patent Examiner, Art Unit 2125
Prosecution Insights