DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending and have been examined.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 6/23/2025 and 11/3/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement was considered and attached by the examiner.
Drawings
The drawings are objected to because Fig. 1B, Fig. 2, and Fig. 4 words in the block diagram cannot be read due to words being blurry. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
Claim 11: “an associative engine configured”, “a prompt construction module configured to”, “a large language model configured to”, and “a user interface configured to”.
Claim 12: “the associative engine is further configured to”.
Claim 13: “a secondary large language model configured to”.
Claim 14: “a visualization rendering module configured to”.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1 the limitations of “receiving, based on a user interaction with a user interface, a natural language query or a selection of a chart element; determining, based on the natural language query or the selection of the chart element, contextual metadata from an associative engine, wherein the contextual metadata comprises at least one of current selection state, hypercube data, data model relationships, or user interaction history; generating, based on the contextual metadata and one or more insight templates, a prompt for a large language model, wherein the one or more insight templates are incorporated as components within the prompt rather than as direct output; causing, based on the prompt, the large language model to generate a natural language insight response that combines factual content from the contextual metadata with flexible narrative generation; and causing, based on the natural language insight response, the natural language insight response to be output at the user interface”, as drafted, are processes that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. More specifically, the mental process of a human reading a question or a selection of a question, thinking of contextual elements of the question, thinking of a related question that utilizes a template, thinking of a response to the question, and writing the response on paper using a pen or pencil. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the --Mental Processes-- grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
Regarding claim 11 the limitations of “an associative engine configured to store data in-memory and provide contextual metadata comprising current selection states and data relationships ;a prompt construction module configured to generate prompts for a large language model based on user queries and the contextual metadata from the associative engine, wherein insight templates are incorporated as prompt components rather than deterministic output generators; a large language model configured to process the prompts and generate natural language insights that combine template-derived facts with flexible narrative generation; and a user interface configured to receive user interactions and display the natural language insights generated by the large language model”, as drafted, are processes that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. More specifically, the mental process of a human reading a selected question, thinking of contextual elements of the question, thinking of a related question that utilizes a template, thinking of a response to the question, and writing the response on paper using a pen or pencil. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the --Mental Processes-- grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
Regarding claim 16 the limitations of “receiving, based on a user selection of a visualization element in a user interface, a request for insight generation about the selected visualization element; determining, based on the selected visualization element, chart metadata and underlying data from an associative engine operating in-memory; generating, based on the chart metadata and underlying data, a structured prompt for a large language model, wherein the structured prompt includes contextual information about the visualization and template-based analytical findings as prompt components; causing, based on the structured prompt, the large language model to generate a natural language explanation of the selected visualization element; and causing, based on the natural language explanation, the natural language explanation to be output at the user interface alongside the selected visualization element”, as drafted, are processes that, under broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components. More specifically, the mental process of a human reading a selected question, thinking of contextual elements of the question, thinking of a related question that utilizes a template, thinking of a response to the question that includes visual elements, and writing the response on paper using a pen or pencil. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the --Mental Processes-- grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
This judicial exception is not integrated into a practical application because the additional element of large language model does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional element of using generalized computer components to read a question or a selection of a question, think of contextual elements of the question, think of a related question that utilizes a template, think of a response to the question, and write the response on paper using a pen or pencil amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claims are not patent eligible.
With respect to claim 2, the claim recites “wherein the associative engine operates in-memory and provides sub-second response times for retrieving the contextual metadata”, which reads on a human analyzing a question with sub-second response time. No additional limitations are present.
With respect to claim 3, the claim recites “wherein the contextual metadata further comprises excluded data values that are not associated with current user selections”, which reads on a human analyzing a question with information not present in the question. No additional limitations are present.
With respect to claim 4, the claim recites “wherein the natural language insight response includes information about the excluded data values to provide comprehensive analytical context”, which reads on a human thinking of a response with information not present in the question. No additional limitations are present.
With respect to claim 5, the claim recites “generating, based on the prompt, a secondary prompt using a secondary large language model to produce structured draft findings; and providing the structured draft findings to the large language model for generating the natural language insight response”, which reads on a human thinking of a second question to produce another response. No additional limitations are present.
With respect to claim 6, the claim recites “wherein the structured draft findings comprise factual observations and visualization recommendations”, which reads on a human thinking of a second response to include factual observations and visualization recommendations. No additional limitations are present.
With respect to claim 7, the claim recites “wherein the one or more insight templates comprise narrative structure patterns and analytical logic rules that guide content generation without dictating specific output format”, which reads on a human utilizing templates with narrative structure patterns and analytical logic rules when thinking of a question. No additional limitations are present.
With respect to claim 8, the claim recites “generating, based on the natural language insight response, a visualization specification; and rendering, based on the visualization specification and data from the associative engine, a chart or graph for display with the natural language insight response”, which reads on a human thinking of responses that include chart or graph visual elements and writing the chart or graph on paper using a pen or pencil. No additional limitations are present.
With respect to claim 9, the claim recites “wherein the visualization specification is determined by the large language model based on the type of analysis identified in the contextual metadata”, which reads on a human determining visual chart or graph from analysis of the question. No additional limitations are present.
With respect to claim 10, the claim recites “wherein the selection of the chart element triggers extraction of chart-specific metadata including underlying data points, dimensions, measures, and current filters affecting the chart”, which reads on a human thinking of a chart or graph from selected question that includes information about data points, dimensions, measures, and current filters. No additional limitations are present.
With respect to claim 12, the claim recites “wherein the associative engine is further configured to provide excluded data values that are not associated with current user selections, enabling the large language model to generate insights about data relationships beyond filtered results”, which reads on a human analyzing a question with information not present in the question and thinking of a response with information not present in the question. No additional limitations are present.
With respect to claim 13, the claim recites “further comprising a secondary large language model configured to generate structured draft findings based on the prompts, wherein the large language model processes the structured draft findings to produce the natural language insights”, which reads on a human thinking of a response that includes a structured draft finding. No additional limitations are present.
With respect to claim 14, the claim recites “wherein the structured draft findings comprise factual observations, key statistics, and visualization recommendations that guide the large language model in generating contextually relevant narratives”, which reads on a human thinking of a response that includes factual observations, key statistics, and visualization recommendations. No additional limitations are present.
With respect to claim 15, the claim recites “further comprising a visualization rendering module configured to generate charts or graphs based on visualization specifications provided by the large language model and data retrieved from the associative engine”, which reads on a human think about charts or graphs as a response. No additional limitations are present.
With respect to claim 17, the claim recites “wherein the chart metadata comprises chart type, data dimensions, measures, and current filters applied to the selected visualization element”, which reads on a human thinking of a question that includes chart type, data dimensions, measures, and current filters. No additional limitations are present.
With respect to claim 18, the claim recites “wherein the underlying data comprises historical comparison data and statistical measures that provide context for the selected visualization element”, which reads on a human thinking of historical comparison data and statistical measures that provide context when thinking of a response that includes visual element. No additional limitations are present.
With respect to claim 19, the claim recites “wherein the natural language explanation includes identification of anomalies or patterns in the underlying data that are relevant to the selected visualization element”, which reads on a human thinking of a response that includes anomalies or patterns in the underlying data. No additional limitations are present.
With respect to claim 20, the claim recites “wherein the template-based analytical findings comprise pre-computed statistical analyses and trend identifications that are incorporated as factual components within the structured prompt”, which reads on a human thinking of a response that includes statistical analyses and trend identifications. No additional limitations are present.
These claims further do not remedy the judicial exception being integrated into a practical application and further fail to include additional elements that are sufficient to amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 3-20 are rejected under 35 U.S.C. 103 as being unpatentable over Thomas et al. (U.S. PG Pub No. 20240386058), hereinafter Thomas, in view of Kuan (U.S. PG Pub No. 20240362409).
Regarding claim 1 Thomas teaches:
A method comprising: (P0041, Method of operation for an LLM integration with an application environment.)
receiving, based on a user interaction with a user interface, a natural language query or a selection of a chart element; (P0036, As illustrated in operational environment, property settings for the chart are presented to the user in chart menu which may be surfaced in user experience in response to the user making a selection to view the chart properties from an application menu, right-clicking the chart, or by other user input.; P0037, The user enters a natural language input in task pane.)
determining, based on the natural language query or the selection of the chart element, contextual metadata from an associative engine, wherein the contextual metadata comprises at least one of current selection state, hypercube data, data model relationships, or user interaction history; (P0028, Chat history adds to the contextual information.; P0037, Application service creates a prompt based on the user's statement along with contextual information of the chart indicating that the FY22 data series is currently selected.)
generating, based on the contextual metadata and one or more insight templates, a prompt for a large language model, wherein the one or more insight templates are incorporated as components within the prompt rather than as direct output; ()
causing, based on the prompt, the large language model to generate a natural language insight response that combines factual content from the contextual metadata with flexible narrative generation; and (P0060, LLM generates a reply according to the prompt and sends the reply to prompt engine.)
causing, based on the natural language insight response, the natural language insight response to be output at the user interface. (P0028, The application displays in the user interface of the application a task pane or a chat interface by which the application can receive user-supplied natural language inputs and display responses to the inputs based on the replies generated by the LLM.)
Thomas does not specifically teach:
causing, based on the prompt, the large language model to generate a natural language insight response that combines factual content from the contextual metadata with flexible narrative generation; and
Kuan, however, teaches:
causing, based on the prompt, the large language model to generate a natural language insight response that combines factual content from the contextual metadata with flexible narrative generation; and (P0073, The presentation LLM is a trained machine learning model configured to receive a presentation prompt and generate an insight description. In implementations, The insight description presents insights learned from the data in a natural language format with context allowing the data insights to be easily consumed by a user.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to generate a natural language insight response from the large language model that combines factual content with flexible narrative generation. It would have been obvious to combine the references because by utilizing machine learning to locate insightful relationships in the data, new insights are discovered compared to conventional techniques by eliminating the need for manually designed processes that target already-known insightful relationships. Kuan P0006.
Regarding claim 3 Thomas in view of Kuan teach claim 1.
Thomas further teaches:
wherein the contextual metadata further comprises excluded data values that are not associated with current user selections. (P0050, Contextual information includes an external data source supplying the data on which a chart is based.)
Regarding claim 4 Thomas in view of Kuan teach claim 3.
Thomas does not specifically teach:
wherein the natural language insight response includes information about the excluded data values to provide comprehensive analytical context.
Kuan, however, teaches:
wherein the natural language insight response includes information about the excluded data values to provide comprehensive analytical context. (P0067, In the ongoing example with a user input of “what is the churn rate of customers with fiber internet?”, a selected task is generated (e.g., as described above with respect to FIG. 2) as “descriptive analytics”, and the code prompt generation engine utilizes the database semantics module to identify portions of the database that may pertain to the user input, including data associated with an “internet_product_type” field in a “customer_monthly” table. The code prompt generation engine incorporates the knowledge gained from the database semantics module in generating the code prompt. For example, the user input of “what is the churn rate of customers with fiber internet?” is transformed into a code prompt of “generate SQL code to extract an average monthly count from customer_monthly where internet_product_type=‘fiber’ for each of entities that are current customers and entities that are not current customers.” The code prompt is then input to the code LLM, which generates the code to be applied upon the database. The data extraction system executes the code on the database to extract the data. The code includes any suitable code for responding to the user input, and may include code that does not execute upon the database. For example, SQL code executed upon the database may return a value of “2.28%”, and the code includes additional code to reformat or add structure or context to the returned value, such that the data may include “The churn rate of customers with fiber internet is 2.28%.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have response including information about excluded data values. It would have been obvious to combine the references because data needs to be extracted to be processed to generate a chart that visually represents the extracted data. Kuan P0005.
Regarding claim 5 Thomas in view of Kuan teach claim 1.
Thomas does not specifically teach:
generating, based on the prompt, a secondary prompt using a secondary large language model to produce structured draft findings; and providing the structured draft findings to the large language model for generating the natural language insight response.
Kuan, however, teaches:
generating, based on the prompt, a secondary prompt using a secondary large language model to produce structured draft findings; and providing the structured draft findings to the large language model for generating the natural language insight response. (P0047, The task classifier may generate a selected task of “descriptive analytics”, and the insight generation system, the insight presentation system, or the insight interaction system perform processes associated with descriptive analytics.; P0062, The selected task determines a configuration of the data extraction system with respect to the user input. For instance, the data extraction system may select a code generation module … that includes a respective code prompt generation engine and code LLM that are tailored for the particular selected task. For example, a first code generation module corresponding to a selected task of “descriptive analytics” includes a first code prompt generation engine and code LLM that are each created or trained to optimize tasks pertaining to descriptive analytics, while a second code generation module corresponding to a selected task of “causal inference analysis” includes a second code prompt generation engine and code LLM that are each created or trained to optimize tasks pertaining to causal inference analysis.; P0067, The code includes any suitable code for responding to the user input 508, and may include code that does not execute upon the database 504. For example, SQL code executed upon the database 504 may return a value of “2.28%”, and the code includes additional code to reformat or add structure or context to the returned value, such that the data may include “The churn rate of customers with fiber internet is 2.28%.”.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize a second large language model to produce structured draft findings. It would have been obvious to combine the references because data needs to be extracted and analyzed before insight can be generated. Kuan P0118-P0121.
Regarding claim 6 Thomas in view of Kuan teach claim 5.
Kuan further teaches:
wherein the structured draft findings comprise factual observations and visualization recommendations. (P0067, The code includes any suitable code for responding to the user input 508, and may include code that does not execute upon the database 504. For example, SQL code executed upon the database 504 may return a value of “2.28%”, and the code includes additional code to reformat or add structure or context to the returned value, such that the data may include “The churn rate of customers with fiber internet is 2.28%.”.; P0080, The presentation LLM generates chart recommendations corresponding to the insight descriptions. The presentation LLM (concurrently with generating the insight descriptions, or responsive to a subsequent input of the insight descriptions, and so forth) generates a recommended chart type for data associated with the insight description.; P0080, The presentation LLM generates chart recommendations corresponding to the insight descriptions. The presentation LLM (concurrently with generating the insight descriptions, or responsive to a subsequent input of the insight descriptions, and so forth) generates a recommended chart type for data associated with the insight description.; P0116, In implementations, the classifier LLM of FIG. 2, the code LLM of FIG. 5, and the presentation LLM of FIG. 7 are each independent LLMs trained individually for their respective tasks. In other implementations, the classifier LLM, the code LLM, and the presentation LLM are trained as a single LLM trained to perform all of the tasks of the individual LLMs.)
Regarding claim 7 Thomas in view of Kuan teach claim 1.
Thomas further teaches:
wherein the one or more insight templates comprise narrative structure patterns and analytical logic rules that guide content generation without dictating specific output format. (P0058, The prompt template includes rules which govern the manner in which LLM is to respond to the prompt and illustrative examples of output formatting to guide LLM in formatting its reply.; P0063, Prompt engine selects a prompt template based on a type of inquiry and configures a prompt according to the template. In the prompt, prompt engine includes tasks, instructions, or rules applicable to generating a reply to the input, such as tasking LLM to identify whether the user is trying to interact with a representation of chart (“Prop_Tree”) and a rule to return the reply in a particular output format which is suitable for parsing (“JSON object array”). The tasks, instructions, and rules included in the prompt may be predetermined according to the prompt template or according to the type of inquiry. The prompt also includes the user's natural language input (“User_Msg (1)”).)
Regarding claim 8 Thomas in view of Kuan teach claim 1.
Thomas does not specifically teach:
generating, based on the natural language insight response, a visualization specification; and
rendering, based on the visualization specification and data from the associative engine, a chart or graph for display with the natural language insight response.
Kuan, however, teaches:
generating, based on the natural language insight response, a visualization specification; and (P0080, The presentation LLM generates chart recommendations corresponding to the insight descriptions. The presentation LLM (concurrently with generating the insight descriptions, or responsive to a subsequent input of the insight descriptions, and so forth) generates a recommended chart type for data associated with the insight description. For instance, the chart type is selected from a set of available chart types, such as a bar chart, a donut chart, a table with a heat map, a column chart, a scatter chart, a pie chart, a histogram, and so forth.)
rendering, based on the visualization specification and data from the associative engine, a chart or graph for display with the natural language insight response. (P0083, The chart generation system processes the chart type and the chart data to generate a graphical representation as the insight chart.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to render a visual specification a chart or graph for display. It would have been obvious to combine the references because visual representation of the chart data is visually appealing and easy to understand. Kuan P0083.
Regarding claim 9 Thomas in view of Kuan teach claim 8.
Kuan further teaches:
wherein the visualization specification is determined by the large language model based on the type of analysis identified in the contextual metadata. (P0080, The presentation LLM generates chart recommendations corresponding to the insight descriptions. The presentation LLM (concurrently with generating the insight descriptions, or responsive to a subsequent input of the insight descriptions, and so forth) generates a recommended chart type for data associated with the insight description.)
Regarding claim 10 Thomas in view of Kuan teach claim 1.
Thomas further teaches:
wherein the selection of the chart element triggers extraction of chart-specific metadata including underlying data points, dimensions, measures, and current filters affecting the chart. (Fig 1., Chart format.; P0025, The application includes in the prompt a representation of the chart to which the user input refers. The format of the representation is a data structure which includes chart design and format properties which define the chart. The chart properties may be organized in a hierarchical fashion, such as a property tree. For example, the chart title may be a property which includes multiple categories, such as the font, location, visibility, text box formatting, and so on. The font of the chart title may include multiple categories such as the font name, size, style, word art effects (e.g., shadow, reflection, etc.), and so on.; P0040, The user may also use a chart design engine to create the chart.)
Regarding claim 11 Thomas teaches:
A system comprising: (P0056, FIG. 3 illustrates system architecture including application, workbook data, and LLM .)
an associative engine configured to store data in-memory and provide contextual metadata comprising current selection states and data relationships; (P0078, Computing device includes … processing system, storage system.; P0028, Chat history adds to the contextual information.; P0037, Application service creates a prompt based on the user's statement along with contextual information of the chart indicating that the FY22 data series is currently selected.)
a prompt construction module configured to generate prompts for a large language model based on user queries and the contextual metadata from the associative engine, wherein insight templates are incorporated as prompt components rather than deterministic output generators; (P0058, To configure the prompt, prompt engine identifies a prompt template for configuring a prompt relating to a data visualization such as a chart. … Using the selected template, prompt engine configures a prompt to include the user input and contextual information from workbook data. Contextual information included in the prompt may include a history of user inputs and replies from LLM and at least a portion of workbook data, such as a data representation of the subject chart (e.g., property tree of the subject chart).; P0060, Having configured a prompt, prompt engine submits the prompt to LLM.)
a large language model configured to process the prompts and generate natural language insights that combine template-derived facts with flexible narrative generation; and (P0060, LLM generates a reply according to the prompt and sends the reply to prompt engine.)
a user interface configured to receive user interactions and display the natural language insights generated by the large language model. (P0028, The application displays in the user interface of the application a task pane or a chat interface by which the application can receive user-supplied natural language inputs and display responses to the inputs based on the replies generated by the LLM.)
Thomas does not specifically teach:
a large language model configured to process the prompts and generate natural language insights that combine template-derived facts with flexible narrative generation; and
Kuan, however, teaches:
a large language model configured to process the prompts and generate natural language insights that combine template-derived facts with flexible narrative generation; and (P0073, The presentation LLM is a trained machine learning model configured to receive a presentation prompt and generate an insight description. In implementations, The insight description presents insights learned from the data in a natural language format with context allowing the data insights to be easily consumed by a user.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to generate a natural language insight response from the large language model that combines derived facts with flexible narrative generation. It would have been obvious to combine the references because by utilizing machine learning to locate insightful relationships in the data, new insights are discovered compared to conventional techniques by eliminating the need for manually designed processes that target already-known insightful relationships. Kuan P0006.
Regarding claim 12 Thomas in view of Kuan teach claim 11.
Thomas further teaches:
wherein the associative engine is further configured to provide excluded data values that are not associated with current user selections, enabling the large language model to generate insights about data relationships beyond filtered results. (P0050, Contextual information includes an external data source supplying the data on which a chart is based.)
Thomas does not specifically teach:
wherein the associative engine is further configured to provide excluded data values that are not associated with current user selections, enabling the large language model to generate insights about data relationships beyond filtered results.
Kuan, however, teaches:
wherein the associative engine is further configured to provide excluded data values that are not associated with current user selections, enabling the large language model to generate insights about data relationships beyond filtered results. (P0067, In the ongoing example with a user input of “what is the churn rate of customers with fiber internet?”, a selected task is generated (e.g., as described above with respect to FIG. 2) as “descriptive analytics”, and the code prompt generation engine utilizes the database semantics module to identify portions of the database that may pertain to the user input, including data associated with an “internet_product_type” field in a “customer_monthly” table. The code prompt generation engine incorporates the knowledge gained from the database semantics module in generating the code prompt. For example, the user input of “what is the churn rate of customers with fiber internet?” is transformed into a code prompt of “generate SQL code to extract an average monthly count from customer_monthly where internet_product_type=‘fiber’ for each of entities that are current customers and entities that are not current customers.” The code prompt is then input to the code LLM, which generates the code to be applied upon the database. The data extraction system executes the code on the database to extract the data. The code includes any suitable code for responding to the user input, and may include code that does not execute upon the database. For example, SQL code executed upon the database may return a value of “2.28%”, and the code includes additional code to reformat or add structure or context to the returned value, such that the data may include “The churn rate of customers with fiber internet is 2.28%.”)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have response including information about excluded data values. It would have been obvious to combine the references because data needs to be extracted to be processed to generate a chart that visually represents the extracted data. Kuan P0005.
Regarding claim 13 Thomas in view of Kuan teach claim 11.
Thomas does not specifically teach:
further comprising a secondary large language model configured to generate structured draft findings based on the prompts, wherein the large language model processes the structured draft findings to produce the natural language insights.
Kuan, however, teaches:
further comprising a secondary large language model configured to generate structured draft findings based on the prompts, wherein the large language model processes the structured draft findings to produce the natural language insights. (P0047, The task classifier may generate a selected task of “descriptive analytics”, and the insight generation system, the insight presentation system, or the insight interaction system perform processes associated with descriptive analytics.; P0062, The selected task determines a configuration of the data extraction system with respect to the user input. For instance, the data extraction system may select a code generation module … that includes a respective code prompt generation engine and code LLM that are tailored for the particular selected task. For example, a first code generation module corresponding to a selected task of “descriptive analytics” includes a first code prompt generation engine and code LLM that are each created or trained to optimize tasks pertaining to descriptive analytics, while a second code generation module corresponding to a selected task of “causal inference analysis” includes a second code prompt generation engine and code LLM that are each created or trained to optimize tasks pertaining to causal inference analysis.; P0067, The code includes any suitable code for responding to the user input 508, and may include code that does not execute upon the database 504. For example, SQL code executed upon the database 504 may return a value of “2.28%”, and the code includes additional code to reformat or add structure or context to the returned value, such that the data may include “The churn rate of customers with fiber internet is 2.28%.”.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to utilize a second large language model to produce structured draft findings. It would have been obvious to combine the references because data needs to be extracted and analyzed before insight can be generated. Kuan P0118-P0121.
Regarding claim 14 Thomas in view of Kuan teach claim 13.
Kuan further teaches:
wherein the structured draft findings comprise factual observations, key statistics, and visualization recommendations that guide the large language model in generating contextually relevant narratives. (P0062, The selected task determines a configuration of the data extraction system with respect to the user input. For instance, the data extraction system may select a code generation module … that includes a respective code prompt generation engine and code LLM that are tailored for the particular selected task. For example, a first code generation module corresponding to a selected task of “descriptive analytics” includes a first code prompt generation engine and code LLM that are each created or trained to optimize tasks pertaining to descriptive analytics, while a second code generation module corresponding to a selected task of “causal inference analysis” includes a second code prompt generation engine and code LLM that are each created or trained to optimize tasks pertaining to causal inference analysis.; P0067, The code includes any suitable code for responding to the user input 508, and may include code that does not execute upon the database 504. For example, SQL code executed upon the database 504 may return a value of “2.28%”, and the code includes additional code to reformat or add structure or context to the returned value, such that the data may include “The churn rate of customers with fiber internet is 2.28%.”.; P0080, The presentation LLM generates chart recommendations corresponding to the insight descriptions. The presentation LLM (concurrently with generating the insight descriptions, or responsive to a subsequent input of the insight descriptions, and so forth) generates a recommended chart type for data associated with the insight description.; P0116, In implementations, the classifier LLM of FIG. 2, the code LLM of FIG. 5, and the presentation LLM of FIG. 7 are each independent LLMs trained individually for their respective tasks. In other implementations, the classifier LLM, the code LLM, and the presentation LLM are trained as a single LLM trained to perform all of the tasks of the individual LLMs.)
Regarding claim 15 Thomas in view of Kuan teach claim 11.
Thomas does not specifically teach:
further comprising a visualization rendering module configured to generate charts or graphs based on visualization specifications provided by the large language model and data retrieved from the associative engine.
Kuan, however, teaches:
further comprising a visualization rendering module configured to generate charts or graphs based on visualization specifications provided by the large language model and data retrieved from the associative engine. (P0080, The presentation LLM generates chart recommendations corresponding to the insight descriptions. The presentation LLM (concurrently with generating the insight descriptions, or responsive to a subsequent input of the insight descriptions, and so forth) generates a recommended chart type for data associated with the insight description. For instance, the chart type is selected from a set of available chart types, such as a bar chart, a donut chart, a table with a heat map, a column chart, a scatter chart, a pie chart, a histogram, and so forth.; P0083, The chart generation system processes the chart type and the chart data to generate a graphical representation as the insight chart.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to render a visual specification a chart or graph for display. It would have been obvious to combine the references because visual representation of the chart data is visually appealing and easy to understand. Kuan P0083.
Regarding claim 16 Thomas teaches:
A method comprising: (P0041, Method of operation for an LLM integration with an application environment.)
receiving, based on a user selection of a visualization element in a user interface, a request for insight generation about the selected visualization element; (P0050, The user has selected the data series for FY22 in the chart. Application service includes in the prompt an indication of the selected data series which provides contextual information for LLM service in responding to the user input.; P0037, Application service creates a prompt based on the user's statement along with contextual information of the chart indicating that the FY22 data series is currently selected.; P0040, A user may use Insights to generate a chart relating to a data table (not shown) in the spreadsheet environment, where the chart is generated by Insights based on the entire data table. The user may also use a chart design engine to create the chart. In addition to and in parallel with the interaction with Insights, the user may submit an input to task pane causing application service to generate and send a prompt to LLM service based on the input, with the prompt including context information relating to the chart or the data table.)
determining, based on the selected visualization element, chart metadata and underlying data from an associative engine operating in-memory; (P0037, Application service creates a prompt based on the user's statement along with contextual information of the chart indicating that the FY22 data series is currently selected.)
generating, based on the chart metadata and underlying data, a structured prompt for a large language model, wherein the structured prompt includes contextual information about the visualization and template-based analytical findings as prompt components; (P0037, Application service creates a prompt based on the user's statement along with contextual information of the chart indicating that the FY22 data series is currently selected.; P0058, To configure the prompt, prompt engine identifies a prompt template for configuring a prompt relating to a data visualization such as a chart. … Using the selected template, prompt engine configures a prompt to include the user input and contextual information from workbook data. Contextual information included in the prompt may include a history of user inputs and replies from LLM and at least a portion of workbook data, such as a data representation of the subject chart (e.g., property tree of the subject chart).; P0060, Having configured a prompt, prompt engine submits the prompt to LLM.)
causing, based on the natural language explanation, the natural language explanation to be output at the user interface alongside the selected visualization element. (P0028, The application displays in the user interface of the application a task pane or a chat interface by which the application can receive user-supplied natural language inputs and display responses to the inputs based on the replies generated by the LLM.; Fig. 5A, Spreadsheet and chart.)
Thomas does not specifically teach:
causing, based on the structured prompt, the large language model to generate a natural language explanation of the selected visualization element; and
Kuan, however, teaches:
causing, based on the structured prompt, the large language model to generate a natural language explanation of the selected visualization element; and (P0073, The presentation LLM is a trained machine learning model configured to receive a presentation prompt and generate an insight description. In implementations, The insight description presents insights learned from the data in a natural language format with context allowing the data insights to be easily consumed by a user.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to generate a natural language explanation from the large language model. It would have been obvious to combine the references because by utilizing machine learning to locate insightful relationships in the data, new insights are discovered compared to conventional techniques by eliminating the need for manually designed processes that target already-known insightful relationships. Kuan P0006.
Regarding claim 17 Thomas in view of Kuan teach claim 16.
Thomas further teaches:
wherein the chart metadata comprises chart type, data dimensions, measures, and current filters applied to the selected visualization element. (P0025, The application includes in the prompt a representation of the chart to which the user input refers. The format of the representation is a data structure which includes chart design and format properties which define the chart. The chart properties may be organized in a hierarchical fashion, such as a property tree. For example, the chart title may be a property which includes multiple categories, such as the font, location, visibility, text box formatting, and so on. The font of the chart title may include multiple categories such as the font name, size, style, word art effects (e.g., shadow, reflection, etc.), and so on.; P0040, The user may also use a chart design engine to create the chart.)
Regarding claim 18 Thomas in view of Kuan teach claim 17.
Thomas does not specifically teach:
wherein the underlying data comprises historical comparison data and statistical measures that provide context for the selected visualization element.
Kuan, however, teaches:
wherein the underlying data comprises historical comparison data and statistical measures that provide context for the selected visualization element. (P0047, In the above example with a user input of “what is the churn rate of customers with fiber internet?”, the task classifier may generate a selected task of “descriptive analytics”, and the insight generation system, the insight presentation system, or the insight interaction system perform processes associated with descriptive analytics to describe historical data.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have data comprising historical comparison data and statistical measures. It would have been obvious to combine the references because comparison of data and statistical measures are a known technique that yields a predictable result of performing descriptive analytics on the data. Kuan P0047.
Regarding claim 19 Thomas in view of Kuan teach claim 18.
Thomas does not specifically teach:
wherein the natural language explanation includes identification of anomalies or patterns in the underlying data that are relevant to the selected visualization element.
Kuan, however, teaches:
wherein the natural language explanation includes identification of anomalies or patterns in the underlying data that are relevant to the selected visualization element. (P0037, The insight generation system, for instance, may be used to analyze data within the dataset to find insightful patterns and trends between different variables in the dataset. The identified insightful patterns and trends may then be processed by the insight presentation system or the insight interaction system to generate natural language descriptions or other media illustrating or describing the identified insightful patterns and trends as the insight output.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to identify anomalies or patterns in data. It would have been obvious to combine the references because comparison of data is a known technique that yields a predictable result of performing descriptive analytics on the data. Kuan P0047.
Regarding claim 19 Thomas in view of Kuan teach claim 16.
Thomas does not specifically teach:
wherein the template-based analytical findings comprise pre-computed statistical analyses and trend identifications that are incorporated as factual components within the structured prompt.
Kuan, however, teaches:
wherein the template-based analytical findings comprise pre-computed statistical analyses and trend identifications that are incorporated as factual components within the structured prompt. (P0065, The prompt generation engine of FIG. 4 as described above, using curated prompts or curated prompt templates that are curated specifically for the desired code prompt generation engine.; P0062, A first code generation module corresponding to a selected task of “descriptive analytics” includes a first code prompt generation engine and code LLM.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have template find statistical analysis and trend identification. It would have been obvious to combine the references because statistical analysis and trend identification of data is a known technique that yields a predictable result of performing descriptive analytics on the data. Kuan P0047.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Thomas in view of Kuan and further view of Dasher et al. (U.S. PG Pub No. 20250085131), hereinafter Dasher.
Regarding claim 2 Thomas in view of Kuan teach claim 1.
Thomas does not specifically teach:
wherein the associative engine operates in-memory and provides sub-second response times for retrieving the contextual metadata.
Kuan, however, teaches:
wherein the associative engine operates in-memory and provides sub-second response times for retrieving the contextual metadata. (P0042, These techniques allow for real-time or near real-time analysis of a wide range of datasets with unknown contents, which is not possible using conventional data analysis techniques. To do so, the data insight generation and presentation techniques described herein utilize an insight generation system to interpret data in a dataset and generate insights based on the dataset.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to quickly retrieve contextual metadata. It would have been obvious to combine the references because quick retrieval improves user experience. Kuan P0042.
Thomas in view of Kuan does not specifically teach:
wherein the associative engine operates in-memory and provides sub-second response times for retrieving the contextual metadata.
Dasher, however, teaches:
wherein the associative engine operates in-memory and provides sub-second response times for retrieving the contextual metadata. (P0234, As used herein, the terms “real-time,” “real time,” “simultaneous,” “substantially on-demand,” and the like are understood to be nearly instantaneous but may include delay due to practical limits of the system. Such delays may be on the order of milliseconds or microseconds, depending on the application and nature of the processing. Relatively longer delays (e.g., greater than a millisecond) may result due to communication or processing delays, particularly in remote and cloud computing environments.)
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have sub-second response time. It would have been obvious to combine the references because real-time processing provides enhanced user experience. Dasher P0053.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL WONSUK CHUNG whose telephone number is (571)272-1345. The examiner can normally be reached Monday - Friday (7am-4pm)[PT].
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PIERRE-LOUIS DESIR can be reached at (571)272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL W CHUNG/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
