DETAILED ACTION
Claims 1-20 are pending, of which claims 1, 16, and 20 are independent.
Claims 1, 16, and 20 are amended. Claims 1-20 are rejected.
Notice of AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 14 is objected to as similar language has been incorporated into parent independent claim 1.
Statutory Review under 35 USC § 101
Claims 1-15 are directed towards a method and have been reviewed.
Claims 1-15 appear to remain directed to patent-eligible subject matter under 35 U.S.C. 101 as the judicial exception is integrated into a practical application as per (Revised) Step 2A, Prong Two of the patent subject matter eligibility determination.
Specifically, the claims recite additional elements demonstrating that the claim as a whole integrates the exception into a practical application. The claims have been evaluated to ensure that the claims reflect the disclosed improvement: the claims are drawn to generating the visualization object to be displayed based on query parameters identified for the target dataset and based on a visualization type, which is realized as providing data in a form best suited for the target dataset, see also the instant specification FIGs. 6 and ¶ 0095 and ¶ 0098 showing chart generation in forms including a line chart, a bar chat, etc.
As the generated visualization to be presented is dictated by the properties of the target dataset, these additional claim elements improve the functioning of a computer or any other technology or technical fields, thus integrating the abstract exception into a practical application.
Claims 16-19 are directed toward a system and have been reviewed.
Claims 16-19 initially appear to remain statutory, as the system includes hardware (processing system) as disclosed in ¶ 0118 of the applicant’s specification, “The computer system 900 includes a processing system including a processor 901. The processor 901 may be a general-purpose single- or multi-chip microprocessor (e.g., an Advanced Reduced Instruction Set Computer (RISC) Machine (ARM)), a special-purpose microprocessor (e.g., a digital signal processor (DSP)), a microcontroller, a programmable gate array, etc. The processor 901 may be referred to as a central processing unit (CPU) and may cause computer-implemented instructions to be performed.”
Claims 16-19 also appear to remain directed to patent-eligible subject matter.
Claim 20 is directed towards a method and has been reviewed.
Claim 20 appears to remain directed to patent-eligible subject matter as the judicial exception is integrated into a practical application as per (Revised) Step 2A, Prong Two of the patent subject matter eligibility determination.
Specifically, the claims recite additional elements demonstrating that the claim as a whole integrates the exception into a practical application. The claims have been evaluated to ensure that the claims reflect the disclosed improvement: the claims are drawn to determining that a user query was previously received, then using a generative AI model to generate an updated user query to include context information from a previous instance of the user query, which shows an improved user query.
These additional claim elements improve the functioning of a computer or any other technology or technical fields, thus integrating the abstract exception into a practical application.
Response to Arguments
35 U.S.C. 103
Applicant’s arguments, see pp11-12, filed 02/27/2026, with respect to the rejection(s) of claim(s) 1, 16, and 20 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made of claims 1 and 16 under 35 U.S.C. 103 as being unpatentable over Lee in view of Shachaf in further view of newly incorporated references Doshi and Stremmel and of claim 20 under 35 U.S.C. 103.
The dependent claims remain rejected by virtue of their dependence on rejected base claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-4, 9-10, 14-15; and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al., U.S. Patent Application Publication No. 2024/0386040 (filed May 13, 2024; hereinafter Lee) in view of Shachaf et al., U.S. Patent No. 12,353,407 (filed January 10, 2024; hereinafter Shachaf) in further view of Doshi et al., U.S. Patent Application Publication No. 2025/0278643 (filed February 29, 2024; hereinafter Doshi) in further view of Stremmel et al., U.S. Patent Application Publication No. 2025/0348708 (filed May 7, 2024; hereinafter Stremmel)
Regarding claim 1, Lee teaches:
A computer-implemented method for using one or more generative artificial intelligence (AI) models to generate visualizations and text insights from large datasets, comprising: (Lee ¶ 0061: use the second model 116 (which can be trained to cross-reference metadata in different portions of inputs and relate together data elements) to generate output reports; Lee FIG. 9, ¶ 0148: At step 910, a recommendation or other response is output from the at least one AI model (e.g., from the fine-tuned AI model from step 902) responsive to the input query; Lee FIG. 7, ¶ 0119-0121, notably ¶ 0119: data (e.g., completions) provided to the client device 304, such as reports, insights, and action items)
generating, by a data analytics system, a database query prompt based on a user query and a target dataset, (Lee FIG. 9, ¶ 0138-0139: At step 902, at least one AI model is fine-tuned using building domain data including information regarding equipment types, equipment parameters, and output conditions … At step 904, an interactive interface is provided. The interactive interface is configured to guide input (by a user) of sufficient information for crafting an appropriate input query for the fine-tuned AI model)
wherein the database query prompt includes query parameters identified for the target dataset, a dynamic example, (Lee describes instructing the model to generate the query in ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904, for example based on the input of the relevant equipment type, the relevant equipment parameter, the relevant output condition, and the request type [shows being based on parameters as claimed] ... the input query is generated using a machine learning model, for example using first generative AI model; see being based on a claimed dynamic example in Lee ¶ 0139-0141, such as ¶ 0141: The at least one generative AI model can then prompt the user for such information, either directly (e.g., “What model of chiller are you referring to?”) or by asking questions to narrow down the set of possible types (e.g., “What color is the chiller?”; “How big is the chiller?”; etc.); see also Lee ¶ 0148, "The recommendation or other response can provide different information depending on the request type obtained from the user")
providing the database query prompt to a generative AI model … to generate the database query based on: the query parameters identified for the target dataset; (Lee FIG. 9, ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904 ... the input query can be generated by selecting a prompt template (e.g., based on the request type) and filling in the prompt template with the information collected from the user in step 904. In some embodiments, the input query is generated using a machine learning model, for example using first generative AI model trained to generate a prompt predicted to cause a second generative AI model (or other large language model) to provide a relevant, accurate, coherent, etc. result)
executing, by the data analytics system, the database query to: obtain the selected data from the target dataset; (Lee FIG. 9, ¶ 0147: At step 908, the input query from step 906 is provided as an input to at least one AI model (e.g., the fine-tuned AI model from step 902))
…
generating data attributes and corresponding attribute causes based on analyzing the selected data; (Lee shows attribute causes in ¶ 0053: This can enable the models 104, 116 to detect patterns of usage (e.g., spikes; troughs; seasonal or other temporal patterns) or other information that may be useful for determining causes of issues or causes of service requests, or predict future issues, such as to allow the models 104, 116 to be trained using information indicative of causes of issues across multiple items of equipment; Lee also shows attribute causes in ¶ 0061-0063: provide the received responses to at least one of the second model 116 or a root cause detection function (e.g., algorithm, model, data structure mapping inputs to candidate causes, etc.) to determine a prediction of a cause of the issue of the item of equipment and/or solutions; Lee also shows attribute causes associated with analysis in ¶ 0128: At 815, a cause of the fault condition can be identified ... the cause is detected using a function that includes one or more algorithms, tables, simulations, or machine learning models described herein)
Lee also teaches a plain-language insight summary. (Lee FIG. 7, ¶ 0119: a human expert to guide the user through the data (e.g., completions) provided to the client device 304, such as reports, insights, and action items; a human expert to review and/or provide feedback for revising insights, guidance, and recommendations before being presented by the application session 308; a human expert to adjust and/or validate insights or recommendations before they are viewed or used for actions by the user)
Lee does not expressly disclose:
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples; and
a visualization type for generating a visualization object from selected data obtained from a database query;
providing the database query prompt to a generative AI model directing the generative AI model to generate the database query based on: … the visualization type for generating the visualization object, wherein the database query is generated by the generative AI model to obtain data from the target dataset compatible with the visualization type;
generate the visualization object from the selected data according to the visualization type;
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text;
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes;
However, Shachaf teaches the following:
wherein the database query prompt includes: a visualization type for generating a visualization object from selected data obtained from a database query; (Shachaf col. 6, lines 20-58: an input text or string, which may for example be received from a user as a prompt [shows database query prompt] or insight request ... Embodiments may generate, by a large language model (LLM), a query based on the wrapped prompt [shows a database query] ... a desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database based on the query, and to create a graph displaying the information described in the user's insight request or prompt [shows a visualization type for generating a visualization object])
providing the database query prompt to a generative AI model directing the generative AI model to generate the database query based on: … the visualization type for generating the visualization object, (Shachaf FIG. 2, col. 5, lines 16-42: Some nonlimiting example for GenAI model that may be used in some embodiments of the invention may be, e.g., the GPT3.5 or GPT4.0 LLMs by OpenAI, although additional or alternative GenAI models and techniques (including, e.g., various open source LLMs known in the art) may be used in different embodiments of the invention; Shachaf col. 6, lines 20-58: Embodiments may wrap a text prompt, where the wrapped prompt may include, for example, database structure information. In some embodiments, the server may “wrap” or contextualize the received request with a description string outlining the required output format from a GenAI model. In the context of the present document, “wrapping” or contextualizing may refer to embedding an input text or string, which may for example be received from a user as a prompt or insight request, in additional pieces of text or strings—for example in order to standardize the input text before it is being further input into a GenAI or LLM component such as for example described herein [shows providing the database query prompt to a generative AI model] ... Embodiments may generate, by a large language model (LLM), a query based on the wrapped prompt [shows directing the generative AI model to generate the database query] ... a desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database based on the query, and to create a graph displaying the information described in the user's insight request or prompt [shows being based on a visualization type for generating the visualization object])
Shachaf also teaches to generate the database query based on: the query parameters identified for the target dataset; (Shachaf FIG. 2, col. 6, line 59-col. 7, line 3: The server may maintain or store predetermined or “constant” or predefined strings that correspond to, e.g., database structure information and may describe database structures, table names, column names, and their respective types, and the like [relevant to query parameters based on para. 108 of the instant specification "the query parameters include a metric, a time range, and a data location..." and para. 0064 "the database search tool searches the target dataset 304 to identify which metrics, filters, and columns would be most useful to answer the data query"]. In some embodiments, the server may send a plurality of strings, such as for example a constant string detailing the database structure (which may also be referred to herein as “schemas” or “database information”) which may be included in, or attached to, a wrapped or contextualized prompt—to a GenAI model 204)
Shachaf further teaches:
wherein the database query is generated by the generative AI model to obtain data from the target dataset compatible with the visualization type; (Shachaf FIG. 2, col. 5, lines 16-42: although additional or alternative GenAI models and techniques (including, e.g., various open source LLMs known in the art) may be used in different embodiments of the invention; Shachaf col. 6, lines 20-58: Embodiments may wrap a text prompt, where the wrapped prompt may include, for example, database structure information. In some embodiments, the server may “wrap” or contextualize the received request with a description string outlining the required output format from a GenAI model ... Embodiments may generate, by a large language model (LLM), a query based on the wrapped prompt [shows a database query generated by the generative AI model] ... a desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database based on the query, and to create a graph displaying the information described in the user's insight request or prompt [shows obtaining data from a target dataset compatible with the visualization type])
generate the visualization object from the selected data according to the visualization type; (Shachaf FIG. 2, col. 7, lines 33-44: extract or output data entries or data items from the database based on the query, and/or to retrieve the relevant data or result data matching or corresponding to the user's insight request as further described herein. As mentioned and further demonstrated herein, an output of a query may include data selected from two columns, which will later be used as the xAxis and yAxis in a chart or a graph; see also relevant Shachaf FIG. 12, col. 15, lines 12-26: The web application or analytics portal may process the response or JSON file and plot a chart 1220 based on the axis properties as follows: the chart type may be set to “type” in the response, which in this case may be a “bar” chart type; the x-axis may be set to “xaxis”, which in this case may be “desktop_application_long_name”; the y-axis may be set to “yaxis”, which in this case may be “total_used_time”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the generative AI query generation of Lee with the generative AI query generation of Shachaf.
In addition, both of the references (Lee and Shachaf) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI-based query generation.
Motivation to do so would be to improve the functioning of Lee performing query generation with the functioning of similar reference Shachaf also performing query generation but with improved wrapping and recontextualizing techniques.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to improve data analysis technologies by providing a user friendly, natural language based approach for producing database queries (Shachaf col. 22, line 58-col. 23, line 7).
Lee in view of Shachaf does not expressly disclose:
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples;
…
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text;
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes;
However, Doshi teaches:
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text; (Doshi FIGs. 2-4, ¶ 0060: AI guide service 314 is further configured to identify instructions based on the extracted information elements (e.g., compressed information 327) and add the instructions to the prompt 328. Such instructions may include instructions to guide the LLM 330 to perform the instructed task and generate the GenAI explanation. For example, an instruction may instruct the LLM to generate a human-readable explanation of the knowledge engine explanations 324, for example, text summarization, e.g., “Based on the information provided, answer the question, ‘Why is my refund not the same as last year?’”, or “Based on the information provided, answer the question, ‘Why is my refund $X?’” [shows plain-language insight summary prompt] ... Other example instructions may instruct the LLM to output the explanation in a particular format, provide examples (e.g., few-shot demonstrations of input and output), and others; see also Doshi ¶ 0024: aspects described herein utilize GenAI models to transform complex, domain-specific application outputs (e.g., a tax calculation for a particular state) into a clear and concise explanation suitable for non-expert users)
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes; (Doshi FIGs. 2-3, ¶ 0063: The GenAI explanation 332 may comprise a human-understandable explanation [shows plain-language insight summary] of the application output, in particular, a domain- and user-specific explanation of an application output (e.g., 105 in FIGS. 1 and 205 in FIG. 2) … the GenAI explanation 332 may be an explanation of a user's tax refund amount, e.g., 206 in FIG. 2; see also Doshi FIG. 2, ¶ 0043-0044: GenAI explanation 206 describes a user-specific explanation of the user's estimated tax refund ... the GenAI explanation 206 comprises a human understandable text output to provide a clear and concise explanation)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the generative AI query generation of Lee as modified with the generative AI output generation of Doshi.
In addition, both of the references (Lee as modified and Doshi) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI-based data retrieval.
Motivation to do so would be to improve the functioning of Lee as modified guiding language models (Lee ¶ 0129-0130) with the functioning of similar reference Doshi also guiding language models but with the improvement of reducing hallucinations as seen in Doshi ¶ 0061.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to use relevant domain knowledge to improve a GenAI model's ability to generate a meaningful GenAI explanation (Doshi ¶ 0028-0029).
Lee in view of Shachaf and Doshi does not expressly disclose:
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples;
However, Stremmel addresses this by teaching:
wherein the database query prompt includes: a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples; (Stremmel ¶ 0071, "the mistake-based selection mechanism is configured to provide, via one or more generative model prompts, a plurality of reference questions from the reference dataset to an LLM to generate a response output (e.g., an answer span) for the plurality of reference questions ... the plurality of reference questions may be provided as input to the LLM using any of a plurality of prompt templates, such as no-shot template (e.g., zero-shot prompt), random selection, or the like. The mistake-based selection approach identifies hard examples by including examples in the prompt which the LLM failed to answer. These examples may be sourced from the reference dataset [shows a corresponding example from a stored collection of examples] and not the test data to avoid overfitting. This may require first applying the LLM to the reference dataset, optionally using other approaches to select in-context examples"; see also Stremmel ¶ 0059, "an “in-context example selection mechanism” refers to a prompt engineering subroutine configured to intelligently filter and select in-context examples for a generative model query ... Examples of in-context example selection mechanisms include nearest neighbor-based selection mechanism, mistake-based selection mechanism, question-context lexical overlap selection mechanism, question-answer lexical overlap selection mechanism, or the like [relevant to context type]"; see also Stremmel ¶ 0103: the reference dataset 410 may include a plurality of annotated question-answer pairs that are related to the particular prediction domain. As one example, in a healthcare domain, the reference dataset 410 may include annotated question-answer pairs from one or more healthcare domain fields [shows context type], such as radiology, primary care, dermatology, and/or the like. In some embodiments, an annotated question-answer pair includes a reference question, a reference answer, and a reference document context (e.g., document context))
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the large language model fine-tuning of Lee as modified with the large language model improvement of Stremmel.
In addition, both of the references (Lee as modified and Stremmel) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI improvement techniques.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to ground answers in supporting evidence to prevent hallucinations that are prevalent in LLMs, thereby improving the reliability of LLM outputs (Stremmel ¶ 0001).
Regarding claim 3, Lee in view of Shachaf and Doshi and Stremmel teaches:
providing the database query prompt instructs the generative AI model to generate the database query based on the query parameters identified for the target dataset, the dynamic example, and the visualization type; and (Lee describes instructing the model to generate the query in ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904, for example based on the input of the relevant equipment type, the relevant equipment parameter, the relevant output condition, and the request type [shows being based on parameters and a visualization type as claimed] ... the input query is generated using a machine learning model, for example using first generative AI model; see being based on a claimed dynamic example in Lee ¶ 0139-0141, such as ¶ 0141: The at least one generative AI model can then prompt the user for such information, either directly (e.g., “What model of chiller are you referring to?”) or by asking questions to narrow down the set of possible types (e.g., “What color is the chiller?”; “How big is the chiller?”; etc.); see visualization type as claimed in Lee ¶ 0139, "The interactive interface is configured to guide input (by a user) of sufficient information for crafting an appropriate input query for the fine-tuned AI model. In some embodiments, such information includes" and Lee ¶ 0148, "The recommendation or other response can provide different information depending on the request type obtained from the user")
the database query includes instructions for the data analytics system to identify the selected data within the target dataset (Lee FIG. 9, ¶ 0147-0148: At step 908, the input query from step 906 is provided as an input to at least one AI model (e.g., the fine-tuned AI model from step 902). Step 908 can include prompting the fine-tuned AI model (e.g., model 116) to generate response to the input query. At step 910, a recommendation or other response is output from the at least one AI model (e.g., from the fine-tuned AI model from step 902) responsive to the input query. The recommendation or other response can provide different information depending on the request type obtained from the user, and can relate to the other domain-specific details input by the user (e.g., the equipment type, the parameter of interest, the output condition of interest, etc.))
and generate the visualization object from the selected data based on the visualization type. (Lee ¶ 0139: At step 904, an interactive interface is provided. The interactive interface is configured to guide input (by a user) of sufficient information for crafting an appropriate input query for the fine-tuned AI model ... such information includes ... a request type; see also relevant Lee ¶ 0148: At step 910, a recommendation or other response is output from the at least one AI model (e.g., from the fine-tuned AI model from step 902) responsive to the input query [shows generation of a visualization object]. The recommendation or other response can provide different information depending on the request type obtained from the user [shows being based on the visualization type])
Regarding claim 4, Lee in view of Shachaf and Doshi and Stremmel teaches all the features with respect to claim 1 above including:
further comprising determining the dynamic example by: comparing the user query to a set of user queries… (Lee ¶ 0094-0097, see notably ¶ 0097: the completion evaluator 324 evaluates the completions by comparing the completions with corresponding data from the data repository 204. For example, the completion evaluator 324 can identify data of the data repository 204 having similar text as the prompts and/or completions (e.g., using any of various natural language processing algorithms), and determine whether the data of the completions is within a range of expected data represented by the data of the data repository 204)
Stremmel teaches to identify a correlated user query. (Stremmel ¶ 0079: a machine learning classification model may be previously trained to classify an input question into a predefined category using a reference dataset associated with the prediction domain; Stremmel FIG. 4, ¶ 0100-0103: the input question 404 is a data entity that describes a request for information associated with an input document 406. The input question, for example, may include text inputs that define a natural language query ... the reference dataset 410 may include a plurality of annotated question-answer pairs that are related to the particular prediction domain. As one example, in a healthcare domain, the reference dataset 410 may include annotated question-answer pairs from one or more healthcare domain fields)
Stremmel further teaches identifying an example associated with the correlated user query; and selecting the example as the dynamic example to include in the database query prompt. (Stremmel ¶ 0071, "the mistake-based selection mechanism is configured to provide, via one or more generative model prompts, a plurality of reference questions from the reference dataset to an LLM to generate a response output (e.g., an answer span) for the plurality of reference questions ... the plurality of reference questions may be provided as input to the LLM using any of a plurality of prompt templates, such as no-shot template (e.g., zero-shot prompt), random selection, or the like. The mistake-based selection approach identifies hard examples by including examples in the prompt which the LLM failed to answer. These examples may be sourced from the reference dataset and not the test data to avoid overfitting. This may require first applying the LLM to the reference dataset, optionally using other approaches to select in-context examples"; see also Stremmel ¶ 0103: the reference dataset 410 may include a plurality of reference data objects, each reflective of an annotated question-answer pair for a prediction domain that may be used as a training entry and/or as an in-context prompt example for one or more different machine learning models of the present disclosure ... the reference dataset 410 may include a plurality of annotated question-answer pairs that are related to the particular prediction domain)
Regarding claim 9, Lee in view of Shachaf and Doshi and Stremmel teaches:
wherein generating the database query prompt includes validating the query parameters and automatically updating a query parameter that does not initially pass validation. (Lee FIG. 5, ¶ 0105-0112, see ¶ 0105: the system 200 can include one or more data filters 500 (e.g., data validators); see Lee ¶ 0111: the system 200 can at least one of (i) modify the data or (ii) output an alert responsive to the data not satisfying the criteria of the respective data filter(s) 500. For example, the system 200 can modify the data by modifying one or more values of the data to be within the criteria of the data filters 500)
Regarding claim 10, Lee in view of Shachaf and Doshi and Stremmel teaches:
wherein the data analytics system includes a visualization generation tool to generate the visualization object from the selected data. (Lee FIG. 9, ¶ 0148: At step 910, a recommendation or other response is output from the at least one AI model (e.g., from the fine-tuned AI model from step 902) responsive to the input query; see this in light of Lee ¶ 0061: use the second model 116 (which can be trained to cross-reference metadata in different portions of inputs and relate together data elements) to generate output reports (e.g., the second model 116, having been configured with data that includes time information, can use timestamps of input from dictation and timestamps of when an image is taken, and place the image in the report in a target position or label based on time correlation))
Regarding claim 14, Lee in view of Shachaf and Doshi and Stremmel teaches all the features with respect to claim 1 above including:
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text; (Doshi FIGs. 2-4, ¶ 0060: AI guide service 314 is further configured to identify instructions based on the extracted information elements (e.g., compressed information 327) and add the instructions to the prompt 328. Such instructions may include instructions to guide the LLM 330 to perform the instructed task and generate the GenAI explanation. For example, an instruction may instruct the LLM to generate a human-readable explanation of the knowledge engine explanations 324, for example, text summarization, e.g., “Based on the information provided, answer the question, ‘Why is my refund not the same as last year?’”, or “Based on the information provided, answer the question, ‘Why is my refund $X?’” [shows plain-language insight summary prompt] ... Other example instructions may instruct the LLM to output the explanation in a particular format, provide examples (e.g., few-shot demonstrations of input and output), and others; see also Doshi ¶ 0024: aspects described herein utilize GenAI models to transform complex, domain-specific application outputs (e.g., a tax calculation for a particular state) into a clear and concise explanation suitable for non-expert users)
providing the plain-language insight summary prompt to the generative AI model; (Doshi FIGs. 2-3, ¶ 0063: The GenAI explanation 332 may comprise a human-understandable explanation [shows plain-language insight summary] of the application output, in particular, a domain- and user-specific explanation of an application output (e.g., 105 in FIGS. 1 and 205 in FIG. 2) … the GenAI explanation 332 may be an explanation of a user's tax refund amount, e.g., 206 in FIG. 2; see also Doshi FIG. 2, ¶ 0043-0044: GenAI explanation 206 describes a user-specific explanation of the user's estimated tax refund ... the GenAI explanation 206 comprises a human understandable text output to provide a clear and concise explanation)
Regarding claim 15, Lee in view of Shachaf and Doshi and Stremmel teaches all the features with respect to claim 1 above; Lee teaches:
wherein providing … the plain-language insight summary within the user interface includes: generating a custom-generated interactive interface that includes: … the plain-language insight summary; and displaying the interactive interface within a user interface of the data analytics system. (Lee FIG. 9, ¶ 0148: At step 910, a recommendation or other response is output from the at least one AI model (e.g., from the fine-tuned AI model from step 902) responsive to the input query ... the output provides a control action that can be taken with respect to the equipment (e.g., an adjustment of operating setpoints, etc.) [shows interactive interface]; see also an insight summary and visualization objects in Lee FIG. 7, ¶ 0119-0121, see first ¶ 0119: a human expert to guide the user through the data (e.g., completions) provided to the client device 304, such as reports, insights, and action items; a human expert to review and/or provide feedback for revising insights, guidance, and recommendations before being presented by the application session 308; a human expert to adjust and/or validate insights or recommendations before they are viewed or used for actions by the user [also shows interactive interface]; see also Lee ¶ 0120: the expert system 700 requests at least one of an identifier or a credential of a user of the client device 704 prior to providing the data to the client device 704 and/or requesting feedback regarding the data from the expert session 708; see visualization objects also within the reports of Lee ¶ 0061: use the second model 116 (which can be trained to cross-reference metadata in different portions of inputs and relate together data elements) to generate output reports (e.g., the second model 116, having been configured with data that includes time information, can use timestamps of input from dictation and timestamps of when an image is taken, and place the image in the report in a target position or label based on time correlation))
Shachaf teaches: wherein providing the visualization object … within the user interface includes: generating a custom-generated interactive interface that includes: the visualization object… (Shachaf FIG. 2, col. 7, lines 33-44: extract or output data entries or data items from the database based on the query, and/or to retrieve the relevant data or result data matching or corresponding to the user's insight request as further described herein. As mentioned and further demonstrated herein, an output of a query may include data selected from two columns, which will later be used as the xAxis and yAxis in a chart or a graph; see also relevant Shachaf FIG. 12, col. 15, lines 12-26: The web application or analytics portal may process the response or JSON file and plot a chart 1220 based on the axis properties as follows: the chart type may be set to “type” in the response, which in this case may be a “bar” chart type; the x-axis may be set to “xaxis”, which in this case may be “desktop_application_long_name”; the y-axis may be set to “yaxis”, which in this case may be “total_used_time”; see another example of a 'custom-generated visualization object' in Shachaf FIGs. 13-14, col. 16, lines 21-37: In use case #3, a user asks for the number of instances per hour for the top or first ranked routine as shown or described in a previous chart (such as for example the chart or plot of FIG. 13 herein) ... The user may thus use or include the relevant routine's name 1410 as shown in the relevant, previously generated chart in a corresponding free-text input or request)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the generative AI query generation of Lee with the generative AI query generation of Shachaf.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to mitigate undesirable loss of data or in overwriting existing datasets (Shachaf col. 22, line 58-col. 23, line 7).
Regarding claim 16, Lee teaches:
A system comprising: a processing system; and a computer memory comprising instructions that, when executed by the processing system, cause the system to perform operations of: (Lee ¶ 0153: The embodiments of the present disclosure may be implemented using existing computer processors, or by a special purpose computer processor for an appropriate system, incorporated for this or another purpose … such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code in the form of machine-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer or other machine with a processor)
generating, by a data analytics system, a database query prompt based on a user query and a target dataset, (Lee FIG. 9, ¶ 0138-0139: At step 902, at least one AI model is fine-tuned using building domain data including information regarding equipment types, equipment parameters, and output conditions … At step 904, an interactive interface is provided. The interactive interface is configured to guide input (by a user) of sufficient information for crafting an appropriate input query for the fine-tuned AI model)
wherein the database query prompt includes query parameters identified for the target dataset, a dynamic example, (Lee describes instructing the model to generate the query in ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904, for example based on the input of the relevant equipment type, the relevant equipment parameter, the relevant output condition, and the request type [shows being based on parameters as claimed] ... the input query is generated using a machine learning model, for example using first generative AI model; see being based on a claimed dynamic example in Lee ¶ 0139-0141, such as ¶ 0141: The at least one generative AI model can then prompt the user for such information, either directly (e.g., “What model of chiller are you referring to?”) or by asking questions to narrow down the set of possible types (e.g., “What color is the chiller?”; “How big is the chiller?”; etc.); see also Lee ¶ 0148, "The recommendation or other response can provide different information depending on the request type obtained from the user")
providing the database query prompt to a generative AI model … to generate the database query based on: the query parameters identified for the target dataset; (Lee FIG. 9, ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904 ... the input query can be generated by selecting a prompt template (e.g., based on the request type) and filling in the prompt template with the information collected from the user in step 904. In some embodiments, the input query is generated using a machine learning model, for example using first generative AI model trained to generate a prompt predicted to cause a second generative AI model (or other large language model) to provide a relevant, accurate, coherent, etc. result)
executing, by the data analytics system, the database query to: obtain the selected data from the target dataset; (Lee FIG. 9, ¶ 0147: At step 908, the input query from step 906 is provided as an input to at least one AI model (e.g., the fine-tuned AI model from step 902))
…
generating data attributes and corresponding attribute causes based on analyzing the selected data; (Lee shows attribute causes in ¶ 0053: This can enable the models 104, 116 to detect patterns of usage (e.g., spikes; troughs; seasonal or other temporal patterns) or other information that may be useful for determining causes of issues or causes of service requests, or predict future issues, such as to allow the models 104, 116 to be trained using information indicative of causes of issues across multiple items of equipment; Lee also shows attribute causes in ¶ 0061-0063: provide the received responses to at least one of the second model 116 or a root cause detection function (e.g., algorithm, model, data structure mapping inputs to candidate causes, etc.) to determine a prediction of a cause of the issue of the item of equipment and/or solutions; Lee also shows attribute causes associated with analysis in ¶ 0128: At 815, a cause of the fault condition can be identified ... the cause is detected using a function that includes one or more algorithms, tables, simulations, or machine learning models described herein)
Lee also teaches a plain-language insight summary. (Lee FIG. 7, ¶ 0119: a human expert to guide the user through the data (e.g., completions) provided to the client device 304, such as reports, insights, and action items; a human expert to review and/or provide feedback for revising insights, guidance, and recommendations before being presented by the application session 308; a human expert to adjust and/or validate insights or recommendations before they are viewed or used for actions by the user)
Lee does not expressly disclose:
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples; and
a visualization type for generating a visualization object from selected data obtained from a database query;
providing the database query prompt to a generative AI model directing the generative AI model to generate the database query based on: … the visualization type for generating the visualization object, wherein the database query is generated by the generative AI model to obtain data from the target dataset compatible with the visualization type;
generate the visualization object from the selected data according to the visualization type;
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text;
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes;
However, Shachaf teaches the following:
wherein the database query prompt includes: a visualization type for generating a visualization object from selected data obtained from a database query; (Shachaf col. 6, lines 20-58: an input text or string, which may for example be received from a user as a prompt [shows database query prompt] or insight request ... Embodiments may generate, by a large language model (LLM), a query based on the wrapped prompt [shows a database query] ... a desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database based on the query, and to create a graph displaying the information described in the user's insight request or prompt [shows a visualization type for generating a visualization object])
providing the database query prompt to a generative AI model directing the generative AI model to generate the database query based on: … the visualization type for generating the visualization object, (Shachaf FIG. 2, col. 5, lines 16-42: Some nonlimiting example for GenAI model that may be used in some embodiments of the invention may be, e.g., the GPT3.5 or GPT4.0 LLMs by OpenAI, although additional or alternative GenAI models and techniques (including, e.g., various open source LLMs known in the art) may be used in different embodiments of the invention; Shachaf col. 6, lines 20-58: Embodiments may wrap a text prompt, where the wrapped prompt may include, for example, database structure information. In some embodiments, the server may “wrap” or contextualize the received request with a description string outlining the required output format from a GenAI model. In the context of the present document, “wrapping” or contextualizing may refer to embedding an input text or string, which may for example be received from a user as a prompt or insight request, in additional pieces of text or strings—for example in order to standardize the input text before it is being further input into a GenAI or LLM component such as for example described herein [shows providing the database query prompt to a generative AI model] ... Embodiments may generate, by a large language model (LLM), a query based on the wrapped prompt [shows directing the generative AI model to generate the database query] ... a desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database based on the query, and to create a graph displaying the information described in the user's insight request or prompt [shows being based on a visualization type for generating the visualization object])
Shachaf also teaches to generate the database query based on: the query parameters identified for the target dataset; (Shachaf FIG. 2, col. 6, line 59-col. 7, line 3: The server may maintain or store predetermined or “constant” or predefined strings that correspond to, e.g., database structure information and may describe database structures, table names, column names, and their respective types, and the like [relevant to query parameters based on para. 108 of the instant specification "the query parameters include a metric, a time range, and a data location..." and para. 0064 "the database search tool searches the target dataset 304 to identify which metrics, filters, and columns would be most useful to answer the data query"]. In some embodiments, the server may send a plurality of strings, such as for example a constant string detailing the database structure (which may also be referred to herein as “schemas” or “database information”) which may be included in, or attached to, a wrapped or contextualized prompt—to a GenAI model 204)
Shachaf further teaches:
wherein the database query is generated by the generative AI model to obtain data from the target dataset compatible with the visualization type; (Shachaf FIG. 2, col. 5, lines 16-42: although additional or alternative GenAI models and techniques (including, e.g., various open source LLMs known in the art) may be used in different embodiments of the invention; Shachaf col. 6, lines 20-58: Embodiments may wrap a text prompt, where the wrapped prompt may include, for example, database structure information. In some embodiments, the server may “wrap” or contextualize the received request with a description string outlining the required output format from a GenAI model ... Embodiments may generate, by a large language model (LLM), a query based on the wrapped prompt [shows a database query generated by the generative AI model] ... a desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database based on the query, and to create a graph displaying the information described in the user's insight request or prompt [shows obtaining data from a target dataset compatible with the visualization type])
generate the visualization object from the selected data according to the visualization type; (Shachaf FIG. 2, col. 7, lines 33-44: extract or output data entries or data items from the database based on the query, and/or to retrieve the relevant data or result data matching or corresponding to the user's insight request as further described herein. As mentioned and further demonstrated herein, an output of a query may include data selected from two columns, which will later be used as the xAxis and yAxis in a chart or a graph; see also relevant Shachaf FIG. 12, col. 15, lines 12-26: The web application or analytics portal may process the response or JSON file and plot a chart 1220 based on the axis properties as follows: the chart type may be set to “type” in the response, which in this case may be a “bar” chart type; the x-axis may be set to “xaxis”, which in this case may be “desktop_application_long_name”; the y-axis may be set to “yaxis”, which in this case may be “total_used_time”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the generative AI query generation of Lee with the generative AI query generation of Shachaf.
In addition, both of the references (Lee and Shachaf) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI-based query generation.
Motivation to do so would be to improve the functioning of Lee performing query generation with the functioning of similar reference Shachaf also performing query generation but with improved wrapping and recontextualizing techniques.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to improve data analysis technologies by providing a user friendly, natural language based approach for producing database queries (Shachaf col. 22, line 58-col. 23, line 7).
Lee in view of Shachaf does not expressly disclose:
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples;
…
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text;
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes;
However, Doshi teaches:
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text; (Doshi FIGs. 2-4, ¶ 0060: AI guide service 314 is further configured to identify instructions based on the extracted information elements (e.g., compressed information 327) and add the instructions to the prompt 328. Such instructions may include instructions to guide the LLM 330 to perform the instructed task and generate the GenAI explanation. For example, an instruction may instruct the LLM to generate a human-readable explanation of the knowledge engine explanations 324, for example, text summarization, e.g., “Based on the information provided, answer the question, ‘Why is my refund not the same as last year?’”, or “Based on the information provided, answer the question, ‘Why is my refund $X?’” [shows plain-language insight summary prompt] ... Other example instructions may instruct the LLM to output the explanation in a particular format, provide examples (e.g., few-shot demonstrations of input and output), and others; see also Doshi ¶ 0024: aspects described herein utilize GenAI models to transform complex, domain-specific application outputs (e.g., a tax calculation for a particular state) into a clear and concise explanation suitable for non-expert users)
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes; (Doshi FIGs. 2-3, ¶ 0063: The GenAI explanation 332 may comprise a human-understandable explanation [shows plain-language insight summary] of the application output, in particular, a domain- and user-specific explanation of an application output (e.g., 105 in FIGS. 1 and 205 in FIG. 2) … the GenAI explanation 332 may be an explanation of a user's tax refund amount, e.g., 206 in FIG. 2; see also Doshi FIG. 2, ¶ 0043-0044: GenAI explanation 206 describes a user-specific explanation of the user's estimated tax refund ... the GenAI explanation 206 comprises a human understandable text output to provide a clear and concise explanation)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the generative AI query generation of Lee as modified with the generative AI output generation of Doshi.
In addition, both of the references (Lee as modified and Doshi) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI-based data retrieval.
Motivation to do so would be to improve the functioning of Lee as modified guiding language models (Lee ¶ 0129-0130) with the functioning of similar reference Doshi also guiding language models but with the improvement of reducing hallucinations as seen in Doshi ¶ 0061.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to use relevant domain knowledge to improve a GenAI model's ability to generate a meaningful GenAI explanation (Doshi ¶ 0028-0029).
Lee in view of Shachaf and Doshi does not expressly disclose:
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples;
However, Stremmel addresses this by teaching:
wherein the database query prompt includes: a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples; (Stremmel ¶ 0071, "the mistake-based selection mechanism is configured to provide, via one or more generative model prompts, a plurality of reference questions from the reference dataset to an LLM to generate a response output (e.g., an answer span) for the plurality of reference questions ... the plurality of reference questions may be provided as input to the LLM using any of a plurality of prompt templates, such as no-shot template (e.g., zero-shot prompt), random selection, or the like. The mistake-based selection approach identifies hard examples by including examples in the prompt which the LLM failed to answer. These examples may be sourced from the reference dataset [shows a corresponding example from a stored collection of examples] and not the test data to avoid overfitting. This may require first applying the LLM to the reference dataset, optionally using other approaches to select in-context examples"; see also Stremmel ¶ 0059, "an “in-context example selection mechanism” refers to a prompt engineering subroutine configured to intelligently filter and select in-context examples for a generative model query ... Examples of in-context example selection mechanisms include nearest neighbor-based selection mechanism, mistake-based selection mechanism, question-context lexical overlap selection mechanism, question-answer lexical overlap selection mechanism, or the like [relevant to context type]"; see also Stremmel ¶ 0103: the reference dataset 410 may include a plurality of annotated question-answer pairs that are related to the particular prediction domain. As one example, in a healthcare domain, the reference dataset 410 may include annotated question-answer pairs from one or more healthcare domain fields [shows context type], such as radiology, primary care, dermatology, and/or the like. In some embodiments, an annotated question-answer pair includes a reference question, a reference answer, and a reference document context (e.g., document context))
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the large language model fine-tuning of Lee as modified with the large language model improvement of Stremmel.
In addition, both of the references (Lee as modified and Stremmel) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI improvement techniques.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to ground answers in supporting evidence to prevent hallucinations that are prevalent in LLMs, thereby improving the reliability of LLM outputs (Stremmel ¶ 0001).
Regarding claim 17, Lee in view of Shachaf and Doshi and Stremmel teaches:
wherein the first generative AI model and the second generative AI model are different generative AI models. (Lee ¶ 0043: the first model 104 and the second model 116 each include generative AI machine learning models, such as LLMs (e.g., GPT-based LLMs) and/or diffusion models; Lee ¶ 0146: the input query is generated using a machine learning model, for example using first generative AI model trained to generate a prompt predicted to cause a second generative AI model (or other large language model) to provide a relevant, accurate, coherent, etc. result)
Regarding claim 18, Lee in view of Shachaf and Doshi and Stremmel teaches all the features with respect to claim 16 above including:
providing an interactive interface within the data analytics system that enables a selection… (Lee ¶ 0139-0145: At step 904, an interactive interface is provided. The interactive interface is configured to guide input (by a user) of sufficient information for crafting an appropriate input query for the fine-tuned AI model; see primarily Lee ¶ 0143, "the interactive interface includes drop-down menus or other guided selection widgets for accepting user selection of relevant types, parameters, conditions, requests, etc." and Lee ¶ 0144, "the drop-down menu or other list can be generated or filtered based on equipment, points, sensors, etc. available in a building management system or connected equipment system associated with the user")
receiving the user query within a text query field associated with the interactive interface. (Lee ¶ 0139-0145; see primarily ¶ 0143: the interactive interface includes drop-down menus or other guided selection widgets for accepting user selection of relevant types, parameters, conditions, requests, etc., to at least partially define the user query via structured input (while a remainder of the user query may be provided via unstructured natural language); see Lee ¶ 0144-0145: Such a process can involve allowing free-text inputs, and then associated the free-text input with an option from a list generated as described above (e.g., from the equipment associated with a user account; from fine-tuning of the at least one generative AI model))
Another embodiment of Lee teaches a selection of the target dataset. (Lee ¶ 0058: the model updater 108 can select data from the parts data source 112 for the product recommendation generator application 120, or select various combinations of data from the data sources 112 (e.g., engineering data, operational data, and service data) for the service recommendation generator application 120. The model updater 108 can apply various combinations of data from various data sources 112 to facilitate configuring the second model 116 for one or more applications 120)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the user interface of Lee as modified with the data source combinations of Lee.
Motivation to do so would be to improve the functioning of Lee as modified allowing queries over data sources with the functioning of Lee and its user interface allowing queries by use of menus or other widgets.
Claims 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over Lee et al., U.S. Patent Application Publication No. 2024/0386040 (filed May 13, 2024; hereinafter Lee) in view of Shachaf and Doshi and Stremmel in further view of Murphy et al., U.S. Patent Application Publication No. 2024/0291853 (filed February 23, 2024; hereinafter Murphy).
Regarding claim 11, Lee in view of Shachaf and Doshi and Stremmel teaches all the features with respect to claim 1 above including:
wherein generating the data attributes comprises using a data forecasting tool to determine overall trends, cyclical patterns, … within the selected data. (Lee ¶ 0024: The system can perform classification or other pattern recognition or trend detection operations; Lee ¶ 0053: This can enable the models 104, 116 to detect patterns of usage (e.g., spikes; troughs; seasonal or other temporal patterns) or other information that may be useful for determining causes of issues or causes of service requests, or predict future issues; Lee ¶ 0136: information including unstructured data (e.g., service reports) regarding items of equipment and entity engagement or disengagement (e.g., deals) can be correlated to identify patterns regarding ways that service can be performed to maintain or increase the likelihood of increasing performance of one or more items of equipment of the entity, completion of deals or of maintaining engagement with the entity)
Lee in view of Shachaf and Doshi and Stremmel does not expressly disclose determining outliers.
However, Murphy addresses this by teaching:
using a data forecasting tool to determine overall trends, cyclical patterns, and outliers within the selected data. (Murphy describes trends and patterns in at least ¶ 0238-0241, see primarily ¶ 0238: utilize 1458 artificial intelligence/machine learning (in the manner described above) to identify one or more patterns/trends within the platform information (e.g., log files); Murphy also describes patterns and outliers in at least ¶ 0470: A loop facilitates the sequential examination of collected data, enabling the AI system to methodically identify unusual patterns or signatures indicative of malicious activities)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the pattern recognition and trend detection of Lee as modified with the pattern identification and trend identification of Murphy.
In addition, both of the references (Lee as modified and Murphy) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI.
Motivation to do so would be to improve the functioning of Lee as modified performing analysis of data to derive patterns to determine causes of current issues or of future issues with the functioning of similar reference Murphy also performing analysis of data to derive patterns to identify current issues (through its malicious activity) but with the addition of patterns related to known issues.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to significantly improve the detection of complex and evolving cyber threats as seen in Murphy ¶ 0653.
Regarding claim 12, Lee in view of Shachaf and Doshi and Stremmel and Murphy teaches all the features with respect to claim 11 above including:
wherein generating the corresponding attribute causes includes correlating one or more events to the data attributes to determine attribute causes of the data attributes. (Lee ¶ 0024-0030, see first ¶ 0024: The system can facilitate automated, flexible customer report generation … The system can perform root cause prediction by being trained using data that includes indications of root causes of faults or errors, where the indications are labels for or otherwise associated with (unstructured or structure) data; see then Lee ¶ 0027, "accurately generating predictions of root cause, presenting solutions, or presenting instructions for repairing or inspecting the equipment to identify information that the system can use to detect root causes or other issues"; Lee ¶ 0053: This can enable the models 104, 116 to detect patterns of usage (e.g., spikes; troughs; seasonal or other temporal patterns) or other information that may be useful for determining causes of issues or causes of service requests, or predict future issues, such as to allow the models 104, 116 to be trained using information indicative of causes of issues across multiple items of equipment; Lee ¶ 0061-0063: provide the received responses to at least one of the second model 116 or a root cause detection function (e.g., algorithm, model, data structure mapping inputs to candidate causes, etc.) to determine a prediction of a cause of the issue of the item of equipment and/or solutions; Lee ¶ 0128-0129, see primarily ¶ 0128: At 815, a cause of the fault condition can be identified, such as by performing a root cause analysis ... at least one of an identifier of the equipment, the fault condition, user text or speech identifying the fault condition (e.g., notes from any of a variety of entities, such as a facility manager, on-site technician, etc.), or data regarding the equipment used to detect the fault condition can be applied as input to the function to enable the function to determine an indication of a cause of the fault condition)
Regarding claim 13, Lee in view of Shachaf and Doshi and Stremmel and Murphy teaches all the features with respect to claim 11 above including:
wherein generating the corresponding attribute causes includes using the generative AI model to determine attribute causes of the data attributes based on one or more events and the data attributes. (Lee describes the generative AI model in at least ¶ 0043: the first model 104 and the second model 116 each include generative AI machine learning models; Lee shows using it to determine attribute causes in ¶ 0053: This can enable the models 104, 116 to detect patterns of usage (e.g., spikes; troughs; seasonal or other temporal patterns) or other information that may be useful for determining causes of issues or causes of service requests, or predict future issues, such as to allow the models 104, 116 to be trained using information indicative of causes of issues across multiple items of equipment; Lee also shows using the model to determine attribute causes in ¶ 0061-0063: provide the received responses to at least one of the second model 116 or a root cause detection function (e.g., algorithm, model, data structure mapping inputs to candidate causes, etc.) to determine a prediction of a cause of the issue of the item of equipment and/or solutions)
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Shachaf and Doshi and Stremmel and Socher et al., U.S. Patent Application Publication No. 2024/0020538 (published January 18, 2024; hereinafter Socher).
Regarding claim 2, Lee in view of Shachaf and Doshi and Stremmel teaches all the features with respect to claim 1 above including:
identifying context information associated with a previous instance of the user query; and using the generative AI model to generate an updated user query to include the context information from the previous instance of the user query, (Shachaf FIGs. 13-14, col. 16, lines 21-37: a user may use the data from a previously generated chart to further extend or investigate the analytics or insights provided in that chart. FIG. 14 illustrates example use case #3 of an artificial intelligence based generation [shows using the generative AI model to generate a user query] of database queries according to some embodiments of the invention. In use case #3, a user asks for the number of instances per hour for the top or first ranked routine as shown or described in a previous chart (such as for example the chart or plot of FIG. 13 herein) ... The user may thus use or include the relevant routine's name 1410 as shown in the relevant, previously generated chart in a corresponding free-text input or request)
Lee in view of Shachaf and Doshi and Stremmel does not expressly disclose:
determining, by the data analytics system, that the user query was previously received;
wherein generating the database query prompt is based on the updated user query.
However, Socher addresses this by teaching:
determining, by the data analytics system, that the user query was previously received; (Socher FIG. 8, ¶ 0112-0114: At step 802, an input query is received by a search server via a data interface; Socher ¶ 0123: allowing searches performed for previous user inputs to be reused when it is determined that the current user input is sufficiently similar)
wherein generating the database query prompt is based on the updated user query. (Socher ¶ 0112-0115, see notably ¶ 0112: At step 802, an input query is received by a search server via a data interface. As shown in FIG. 4, according to some embodiments, this input query may include one or more of a natural language user input 402, user context 404, or other context 406; see relevant Socher ¶ 0123: The steps detailed above for method 800 may be repeated an indefinite number of times ... the generative AI system is able to maintain a history of the conversation ... allows the generative AI system to maintain context)
Socher also fortifies the teachings of:
identifying context information associated with a previous instance of the user query; and (Socher FIG. 8, ¶ 0112-0114, see first ¶ 0112: At step 802, an input query is received by a search server via a data interface; ¶ 0113: The generative AI system may also take into account user context when generating conversational responses to user queries; ¶ 0114: In additional to context related to the user, the generative AI system may further consider other context, such as information related to prior user interactions with the generative AI system ... other context 406 may include one or more of previous user queries, previous responses by the generative AI system, previous searches performed by the generative AI system; ¶ 0123: the generative AI system is able to maintain a history of the conversation. This allows the generative AI system to maintain context as the user asks questions or otherwise interacts with the generative AI system)
using the generative AI model to generate an updated user query to include the context information from the previous instance of the user query, (Socher FIG. 8, ¶ 0112-0115, see first ¶ 0112: At step 802, an input query is received by a search server via a data interface; ¶ 0113: The generative AI system may also take into account user context when generating conversational responses to user queries; see most notably Socher ¶ 0115: At step 804, the search server may convert the input query into a modified query ... user context and other context, such as user preferences, may also be incorporated into the modified query and corresponding token sequence)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the querying of Lee as modified with the querying of Socher.
In addition, both of the references (Lee as modified and Socher) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI.
Motivation to do so would be to improve the functioning of Lee as modified performing queries and displaying desired information with the functioning of similar reference Socher also performing queries and displaying desired information but with the addition of indefinite repeat execution (Socher ¶ 0123).
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to implement improved generative AI systems that reflect on time-varying information as seen in Socher ¶ 0021.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Shachaf and Socher and Doshi and Stremmel.
Regarding claim 20, Lee teaches:
A computer-implemented method for using one or more generative artificial intelligence (AI) models to generate visualizations and text insights from large datasets, comprising: (Lee ¶ 0061: use the second model 116 (which can be trained to cross-reference metadata in different portions of inputs and relate together data elements) to generate output reports; Lee FIG. 9, ¶ 0148: At step 910, a recommendation or other response is output from the at least one AI model (e.g., from the fine-tuned AI model from step 902) responsive to the input query; Lee FIG. 7, ¶ 0119-0121, notably ¶ 0119: data (e.g., completions) provided to the client device 304, such as reports, insights, and action items)
generating, by a data analytics system, a database query prompt based on … user query and a target dataset, (Lee FIG. 9, ¶ 0138-0139: At step 902, at least one AI model is fine-tuned using building domain data including information regarding equipment types, equipment parameters, and output conditions … At step 904, an interactive interface is provided. The interactive interface is configured to guide input (by a user) of sufficient information for crafting an appropriate input query for the fine-tuned AI model)
wherein the database query prompt includes query parameters identified for the target dataset, a dynamic example, (Lee describes instructing the model to generate the query in ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904, for example based on the input of the relevant equipment type, the relevant equipment parameter, the relevant output condition, and the request type [shows being based on parameters as claimed] ... the input query is generated using a machine learning model, for example using first generative AI model; see being based on a claimed dynamic example in Lee ¶ 0139-0141, such as ¶ 0141: The at least one generative AI model can then prompt the user for such information, either directly (e.g., “What model of chiller are you referring to?”) or by asking questions to narrow down the set of possible types (e.g., “What color is the chiller?”; “How big is the chiller?”; etc.); see also Lee ¶ 0148, "The recommendation or other response can provide different information depending on the request type obtained from the user")
providing the database query prompt to a generative AI model … to generate the database query based on: the query parameters identified for the target dataset; (Lee FIG. 9, ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904 ... the input query can be generated by selecting a prompt template (e.g., based on the request type) and filling in the prompt template with the information collected from the user in step 904. In some embodiments, the input query is generated using a machine learning model, for example using first generative AI model trained to generate a prompt predicted to cause a second generative AI model (or other large language model) to provide a relevant, accurate, coherent, etc. result)
executing, by the data analytics system, the database query to: obtain the selected data from the target dataset; (Lee FIG. 9, ¶ 0147: At step 908, the input query from step 906 is provided as an input to at least one AI model (e.g., the fine-tuned AI model from step 902))
…
generating data attributes and corresponding attribute causes based on analyzing the selected data; (Lee shows attribute causes in ¶ 0053: This can enable the models 104, 116 to detect patterns of usage (e.g., spikes; troughs; seasonal or other temporal patterns) or other information that may be useful for determining causes of issues or causes of service requests, or predict future issues, such as to allow the models 104, 116 to be trained using information indicative of causes of issues across multiple items of equipment; Lee also shows attribute causes in ¶ 0061-0063: provide the received responses to at least one of the second model 116 or a root cause detection function (e.g., algorithm, model, data structure mapping inputs to candidate causes, etc.) to determine a prediction of a cause of the issue of the item of equipment and/or solutions; Lee also shows attribute causes associated with analysis in ¶ 0128: At 815, a cause of the fault condition can be identified ... the cause is detected using a function that includes one or more algorithms, tables, simulations, or machine learning models described herein)
Lee also teaches a plain-language insight summary. (Lee FIG. 7, ¶ 0119: a human expert to guide the user through the data (e.g., completions) provided to the client device 304, such as reports, insights, and action items; a human expert to review and/or provide feedback for revising insights, guidance, and recommendations before being presented by the application session 308; a human expert to adjust and/or validate insights or recommendations before they are viewed or used for actions by the user)
Lee does not expressly disclose:
in response to a data analytics system determining that a user query associated with a target dataset was previously received, using a generative AI model to generate an updated user query to include context information from a previous instance of the user query;
a database query prompt based on the updated user query…
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples; and
a visualization type for generating a visualization object from selected data obtained from a database query;
providing the database query prompt to a generative AI model directing the generative AI model to generate the database query based on: … the visualization type for generating the visualization object, wherein the database query is generated by the generative AI model to obtain data from the target dataset compatible with the visualization type;
generate the visualization object from the selected data according to the visualization type;
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text;
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes;
However, Shachaf addresses this by teaching the following:
… using a generative AI model to generate an updated user query to include context information from a previous instance of the user query; (Shachaf FIGs. 13-14, col. 16, lines 21-37: a user may use the data from a previously generated chart to further extend or investigate the analytics or insights provided in that chart. FIG. 14 illustrates example use case #3 of an artificial intelligence based generation [shows using the generative AI model to generate a user query] of database queries according to some embodiments of the invention. In use case #3, a user asks for the number of instances per hour for the top or first ranked routine as shown or described in a previous chart (such as for example the chart or plot of FIG. 13 herein) [shows an updated user query; shows using context information from a previous instance of the user query] ... The user may thus use or include the relevant routine's name 1410 as shown in the relevant, previously generated chart in a corresponding free-text input or request)
wherein the database query prompt includes … a visualization type for generating a visualization object from selected data obtained from a database query; (Shachaf col. 6, lines 20-58: an input text or string, which may for example be received from a user as a prompt [shows database query prompt] or insight request ... Embodiments may generate, by a large language model (LLM), a query based on the wrapped prompt [shows a database query] ... a desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database based on the query, and to create a graph displaying the information described in the user's insight request or prompt [shows a visualization type for generating a visualization object])
providing the database query prompt to a generative AI model directing the generative AI model to generate the database query based on: … the visualization type for generating the visualization object, (Shachaf FIG. 2, col. 5, lines 16-42: Some nonlimiting example for GenAI model that may be used in some embodiments of the invention may be, e.g., the GPT3.5 or GPT4.0 LLMs by OpenAI, although additional or alternative GenAI models and techniques (including, e.g., various open source LLMs known in the art) may be used in different embodiments of the invention; Shachaf col. 6, lines 20-58: Embodiments may wrap a text prompt, where the wrapped prompt may include, for example, database structure information. In some embodiments, the server may “wrap” or contextualize the received request with a description string outlining the required output format from a GenAI model. In the context of the present document, “wrapping” or contextualizing may refer to embedding an input text or string, which may for example be received from a user as a prompt or insight request, in additional pieces of text or strings—for example in order to standardize the input text before it is being further input into a GenAI or LLM component such as for example described herein [shows providing the database query prompt to a generative AI model] ... Embodiments may generate, by a large language model (LLM), a query based on the wrapped prompt [shows directing the generative AI model to generate the database query] ... a desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database based on the query, and to create a graph displaying the information described in the user's insight request or prompt [shows being based on a visualization type for generating the visualization object])
Shachaf also teaches to generate the database query based on: the query parameters identified for the target dataset; (Shachaf FIG. 2, col. 6, line 59-col. 7, line 3: The server may maintain or store predetermined or “constant” or predefined strings that correspond to, e.g., database structure information and may describe database structures, table names, column names, and their respective types, and the like [relevant to query parameters based on para. 108 of the instant specification "the query parameters include a metric, a time range, and a data location..." and para. 0064 "the database search tool searches the target dataset 304 to identify which metrics, filters, and columns would be most useful to answer the data query"]. In some embodiments, the server may send a plurality of strings, such as for example a constant string detailing the database structure (which may also be referred to herein as “schemas” or “database information”) which may be included in, or attached to, a wrapped or contextualized prompt—to a GenAI model 204)
Shachaf further teaches:
wherein the database query is generated by the generative AI model to obtain data from the target dataset compatible with the visualization type; (Shachaf FIG. 2, col. 5, lines 16-42: although additional or alternative GenAI models and techniques (including, e.g., various open source LLMs known in the art) may be used in different embodiments of the invention; Shachaf col. 6, lines 20-58: Embodiments may wrap a text prompt, where the wrapped prompt may include, for example, database structure information. In some embodiments, the server may “wrap” or contextualize the received request with a description string outlining the required output format from a GenAI model ... Embodiments may generate, by a large language model (LLM), a query based on the wrapped prompt [shows a database query generated by the generative AI model] ... a desired output from an LLM may be for example an SQL query including one or more SQL commands, which may for example select two columns from a database, which may be used to extract a plurality of data entries or data items from a database based on the query, and to create a graph displaying the information described in the user's insight request or prompt [shows obtaining data from a target dataset compatible with the visualization type])
generate the visualization object from the selected data according to the visualization type; (Shachaf FIG. 2, col. 7, lines 33-44: extract or output data entries or data items from the database based on the query, and/or to retrieve the relevant data or result data matching or corresponding to the user's insight request as further described herein. As mentioned and further demonstrated herein, an output of a query may include data selected from two columns, which will later be used as the xAxis and yAxis in a chart or a graph; see also relevant Shachaf FIG. 12, col. 15, lines 12-26: The web application or analytics portal may process the response or JSON file and plot a chart 1220 based on the axis properties as follows: the chart type may be set to “type” in the response, which in this case may be a “bar” chart type; the x-axis may be set to “xaxis”, which in this case may be “desktop_application_long_name”; the y-axis may be set to “yaxis”, which in this case may be “total_used_time”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the generative AI query generation of Lee with the generative AI query generation of Shachaf.
In addition, both of the references (Lee and Shachaf) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI-based query generation.
Motivation to do so would be to improve the functioning of Lee performing query generation with the functioning of similar reference Shachaf also performing query generation but with improved wrapping and recontextualizing techniques.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to improve data analysis technologies by providing a user friendly, natural language based approach for producing database queries (Shachaf col. 22, line 58-col. 23, line 7).
Lee in view of Shachaf does not expressly disclose using a generative AI model “in response to a data analytics system determining that a user query associated with a target dataset was previously received.”’
Lee in view of Shachaf further does not expressly disclose a database query prompt based on the updated user query…
Lee in view of Shachaf further does not expressly disclose:
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples; and
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text;
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes;
However, Socher teaches:
in response to a data analytics system determining that a user query associated with a target dataset was previously received, using a generative AI model to generate an updated user query to include context information from a previous instance of the user query; (Socher FIG. 8, ¶ 0112-0115, see first ¶ 0112: At step 802, an input query is received by a search server via a data interface; Socher ¶ 0113: The generative AI system may also take into account user context when generating conversational responses to user queries; see most notably Socher ¶ 0115: At step 804, the search server may convert the input query into a modified query ... user context and other context, such as user preferences, may also be incorporated into the modified query and corresponding token sequence)
Socher also teaches generating, by a data analytics system, a database query prompt based on the updated user query... (Socher ¶ 0116: At step 806, the search server determines one or more potential search objects in the modified query. This may be key words or phrases within the modified query that are identified to be of particular interest)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the querying of Lee as modified with the querying of Socher.
In addition, both of the references (Lee as modified and Socher) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI.
Motivation to do so would be to improve the functioning of Lee as modified performing queries and displaying desired information with the functioning of similar reference Socher also performing queries and displaying desired information but with the addition of indefinite repeat execution (Socher ¶ 0123).
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to implement improved generative AI systems that reflect on time-varying information as seen in Socher ¶ 0021.
Lee in view of Shachaf and Socher does not expressly disclose:
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples; and
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text;
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes;
However, Doshi teaches:
generating a plain-language insight summary prompt that instructs the generative AI model to convert the data attributes and the corresponding attribute causes into natural language text; (Doshi FIGs. 2-4, ¶ 0060: AI guide service 314 is further configured to identify instructions based on the extracted information elements (e.g., compressed information 327) and add the instructions to the prompt 328. Such instructions may include instructions to guide the LLM 330 to perform the instructed task and generate the GenAI explanation. For example, an instruction may instruct the LLM to generate a human-readable explanation of the knowledge engine explanations 324, for example, text summarization, e.g., “Based on the information provided, answer the question, ‘Why is my refund not the same as last year?’”, or “Based on the information provided, answer the question, ‘Why is my refund $X?’” [shows plain-language insight summary prompt] ... Other example instructions may instruct the LLM to output the explanation in a particular format, provide examples (e.g., few-shot demonstrations of input and output), and others; see also Doshi ¶ 0024: aspects described herein utilize GenAI models to transform complex, domain-specific application outputs (e.g., a tax calculation for a particular state) into a clear and concise explanation suitable for non-expert users)
providing the plain-language insight summary prompt to the generative AI model to generate a plain-language insight summary of the data attributes and the corresponding attribute causes; (Doshi FIGs. 2-3, ¶ 0063: The GenAI explanation 332 may comprise a human-understandable explanation [shows plain-language insight summary] of the application output, in particular, a domain- and user-specific explanation of an application output (e.g., 105 in FIGS. 1 and 205 in FIG. 2) … the GenAI explanation 332 may be an explanation of a user's tax refund amount, e.g., 206 in FIG. 2; see also Doshi FIG. 2, ¶ 0043-0044: GenAI explanation 206 describes a user-specific explanation of the user's estimated tax refund ... the GenAI explanation 206 comprises a human understandable text output to provide a clear and concise explanation)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the generative AI query generation of Lee as modified with the generative AI output generation of Doshi.
In addition, both of the references (Lee as modified and Doshi) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI-based data retrieval.
Motivation to do so would be to improve the functioning of Lee as modified guiding language models (Lee ¶ 0129-0130) with the functioning of similar reference Doshi also guiding language models but with the improvement of reducing hallucinations as seen in Doshi ¶ 0061.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to use relevant domain knowledge to improve a GenAI model's ability to generate a meaningful GenAI explanation (Doshi ¶ 0028-0029).
Lee in view of Shachaf and Socher and Doshi does not expressly disclose:
wherein the database query prompt includes:
a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples;
However, Stremmel addresses this by teaching:
wherein the database query prompt includes: a dynamic example determined by analyzing the user query to identify a context type and selecting, based on the context type, a corresponding example from a stored collection of examples; (Stremmel ¶ 0071, "the mistake-based selection mechanism is configured to provide, via one or more generative model prompts, a plurality of reference questions from the reference dataset to an LLM to generate a response output (e.g., an answer span) for the plurality of reference questions ... the plurality of reference questions may be provided as input to the LLM using any of a plurality of prompt templates, such as no-shot template (e.g., zero-shot prompt), random selection, or the like. The mistake-based selection approach identifies hard examples by including examples in the prompt which the LLM failed to answer. These examples may be sourced from the reference dataset [shows a corresponding example from a stored collection of examples] and not the test data to avoid overfitting. This may require first applying the LLM to the reference dataset, optionally using other approaches to select in-context examples"; see also Stremmel ¶ 0059, "an “in-context example selection mechanism” refers to a prompt engineering subroutine configured to intelligently filter and select in-context examples for a generative model query ... Examples of in-context example selection mechanisms include nearest neighbor-based selection mechanism, mistake-based selection mechanism, question-context lexical overlap selection mechanism, question-answer lexical overlap selection mechanism, or the like [relevant to context type]"; see also Stremmel ¶ 0103: the reference dataset 410 may include a plurality of annotated question-answer pairs that are related to the particular prediction domain. As one example, in a healthcare domain, the reference dataset 410 may include annotated question-answer pairs from one or more healthcare domain fields [shows context type], such as radiology, primary care, dermatology, and/or the like. In some embodiments, an annotated question-answer pair includes a reference question, a reference answer, and a reference document context (e.g., document context))
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the large language model fine-tuning of Lee as modified with the large language model improvement of Stremmel.
In addition, both of the references (Lee as modified and Stremmel) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI improvement techniques.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to ground answers in supporting evidence to prevent hallucinations that are prevalent in LLMs, thereby improving the reliability of LLM outputs (Stremmel ¶ 0001).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Shachaf and Doshi and Stremmel and Haq et al., U.S. Patent No. 12,105,729 (filed November 8, 2023; hereinafter Haq).
Regarding claim 5, Lee in view of Shachaf and Doshi and Stremmel teaches all the features with respect to claim 1 above but does not expressly disclose:
wherein generating the database query prompt includes:
determining whether the user query includes a time range;
based on the user query including a time range, including the time range in the database query prompt; and
based on the query not including a time range, including a default time range in the database query prompt.
However, Haq addresses this by teaching:
wherein generating the database query prompt includes: determining whether the user query includes a time range; based on the user query including a time range, including the time range in the database query prompt; and (Haq FIG. 8A, col. 16, line 57-col. 17, line 18: The time period for which search requests are shown in the scrollable listing 831 can be selected by the user; FIG. 16, col. 22, lines 39-62: The entry spaces can accept a direct entry of the dates or the user can use a from calendar icon 1321c and a to calendar icon 1321d to display calendars through which the user can select the from and to dates by clicking on a specific day for each)
based on the query not including a time range, including a default time range in the database query prompt. (Haq FIG. 8A, col. 16, line 57-col. 17, line 18: The time period for which search requests are shown in the scrollable listing 831 can be selected by the user or, if not selected, uses a predetermined system default time; FIG. 16, col. 22, lines 39-62: The entry spaces can accept a direct entry of the dates or the user can use a from calendar icon 1321c and a to calendar icon 1321d to display calendars through which the user can select the from and to dates by clicking on a specific day for each. If no date range is entered by the user, the system 200 defaults to search without a time limitation, so it searches all data stored in the system)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the querying of Lee as modified with the querying of Haq.
In addition, both of the references (Lee as modified and Haq) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to provide users with the ability to precisely control their search parameters to achieve the needed targeted results as seen in Haq col. 22, lines 47-49.
Claims 6-7 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Shachaf and Doshi and Stremmel and Dharnidharka et al., U.S. Patent Application Publication No. 2025/0021767 (filed July 31, 2023; hereinafter Dharnidharka).
Regarding claim 6, Lee in view of Shachaf and Doshi and Stremmel teaches all the features with respect to claim 1 above including:
wherein the query parameters include a metric … for data within the target dataset that corresponds to the user query. (Lee describes query parameters including at least one metric in ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904, for example based on the input of the relevant equipment type, the relevant equipment parameter, the relevant output condition, and the request type as shown in the example of FIG. 9)
Lee in view of Shachaf and Doshi and Stremmel does not expressly disclose wherein the query parameters include a time range, and a data location.
However, Dharnidharka addresses this by teaching:
wherein the query parameters include … a time range, and a data location for data within the target dataset that corresponds to the user query. (Dharnidharka teaches a time range in FIG. 10, ¶ 0117-0120, see notably ¶ 0120: the query 1066 includes a time indicator. The time indicator limits searching to events that have timestamps described by the indicator; see next Dharnidharka ¶ 0125: During a first phase of the map process 1070, the search peer 1064 identifies buckets that have events that are described by the time indicator in the search query 1066. As noted above, a bucket contains events whose timestamps fall within a particular range of time; see also Dharnidharka ¶ 0124 regarding the claimed 'data location': the search query 1066 may specify which indexes to search, and the search head 1062 will send the query 1066 to the search peers that have those indexes)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the querying of Lee as modified with the querying of Dharnidharka.
In addition, both of the references (Lee as modified and Dharnidharka) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI.
Motivation to do so would be to improve the functioning of Lee as modified performing queries and displaying desired information with the functioning of similar reference Socher also performing queries and displaying desired information but with the ability to allow distribution of the search workload across different hardware resources (Dharindharka ¶ 0124).
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to improve utilization of resources, utilizing user input to provide the natural language descriptions of search queries and then automating the prompting of several trained LLMs to obtain generative AI translations as seen in Dharnidharka ¶ 0022.
Regarding claim 19, Lee in view of Shachaf and Doshi and Stremmel teaches all the features with respect to claim 16 above including:
wherein generating the query parameters includes using the user query with a database search tool to identify a metric … for data within the target dataset that corresponds to the user query. (Lee describes query parameters including at least one metric in ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904, for example based on the input of the relevant equipment type, the relevant equipment parameter, the relevant output condition, and the request type as shown in the example of FIG. 9)
Lee in view of Shachaf and Doshi and Stremmel does not expressly disclose to identify a time range, and a data location.
However, Dharnidharka addresses this by teaching:
wherein generating the query parameters includes using the user query with a database search tool to identify a metric, a time range, and a data location for data within the target dataset that corresponds to the user query. (Dharnidharka teaches a time range in FIG. 10, ¶ 0117-0120, see notably ¶ 0120: the query 1066 includes a time indicator. The time indicator limits searching to events that have timestamps described by the indicator; see next Dharnidharka ¶ 0125: During a first phase of the map process 1070, the search peer 1064 identifies buckets that have events that are described by the time indicator in the search query 1066. As noted above, a bucket contains events whose timestamps fall within a particular range of time; see also Dharnidharka ¶ 0124 regarding the claimed 'data location': the search query 1066 may specify which indexes to search, and the search head 1062 will send the query 1066 to the search peers that have those indexes)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the querying of Lee as modified with the querying of Dharnidharka.
In addition, both of the references (Lee as modified and Dharnidharka) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI.
Motivation to do so would be to improve the functioning of Lee as modified performing queries and displaying desired information with the functioning of similar reference Socher also performing queries and displaying desired information but with the ability to allow distribution of the search workload across different hardware resources (Dharindharka ¶ 0124).
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to improve utilization of resources, utilizing user input to provide the natural language descriptions of search queries and then automating the prompting of several trained LLMs to obtain generative AI translations as seen in Dharnidharka ¶ 0022.
Regarding claim 7, Lee in view of Shachaf and Doshi and Stremmel in view of Dharnidharka teaches all the features with respect to claim 6 above.
Lee teaches:
wherein generating the query parameters includes using the user query with a database search tool to identify the metric… (Lee describes query parameters including at least one metric in ¶ 0146: At step 906, an input query is generated based on the user inputs collected in step 904, for example based on the input of the relevant equipment type, the relevant equipment parameter, the relevant output condition, and the request type as shown in the example of FIG. 9)
Dharnidharka teaches using the user query with a database search tool to identify … the data location from the target dataset. (Dharnidharka FIG. 10, ¶ 0123-0124 teaches 'data location': Upon receiving the search query 1066, the search head 1062 directs the query 1066 to one or more search peers, such as the search peer 1064 illustrated in FIG. 10 ... the search query 1066 may specify which indexes to search, and the search head 1062 will send the query 1066 to the search peers that have those indexes)
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Shachaf and Doshi and Stremmel and Dharnidharka in further view of Bischof et al., U.S. Patent Application Publication No. 2024/0256588 (filed January 29, 2024; hereinafter Bischof).
Regarding claim 8, Lee in view of Shachaf and Doshi and Stremmel and Dharnidharka teaches all the features with respect to claim 7 above including:
wherein using the database search tool includes searching metadata information of the target dataset to identify the metric and the data location from the target dataset, (Dharnidharka ¶ 0111: the indexer 932 can manage event indexes, which impose minimal structure on stored data and can accommodate any type of data. As another example, the indexer 932 can manage metrics indexes, which use a highly structured format to handle the higher volume and lower latency demands associated with metrics data; Dharnidharka FIG. 10, ¶ 0123-0125: Upon receiving the search query 1066, the search head 1062 directs the query 1066 to one or more search peers, such as the search peer 1064 illustrated in FIG. 10 ... search system 1060 may include different search peers for different purposes (e.g., one has an index storing a first type of data or from a first data source while a second has an index storing a second type of data or from a second data source) [interpreted as relevant to 'metadata information'] ... the search query 1066 may specify which indexes to search, and the search head 1062 will send the query 1066 to the search peers that have those indexes [shows identified 'data location'])
Shachaf teaches:
wherein the metadata information includes a column name, a column description, (Shachaf col. 6, lines 59-col. 7, line 3: The server may maintain or store predetermined or “constant” or predefined strings that correspond to, e.g., database structure information and may describe database structures, table names, column names, and their respective types, and the like.)
Lee in view of Shachaf and Doshi and Stremmel and Dharnidharka does not expressly disclose:
wherein the metadata information includes … a column query of a column associated with the data within the target dataset that corresponds to the user query
However, Bischof teaches:
wherein the metadata information includes a column name, a column description, and a column query of a column associated with the data within the target dataset that corresponds to the user query (Bischof ¶ 0045 shows involvement of a user query: Priming module 210 also determines, using the user graph structure, profile information of the user, from which priming module 210 determines a likely intent of the user that narrows the universe of what is queried in the natural language query ... Priming module 210 also determines, using the data warehouse graph, one or more likely columns within the data warehouse that the user intends to act on as part of the query ... a profile reflects that a user queries specific columns or types of columns frequently (or has done so in at least a most recent threshold number of days). The priming module 210 may determine from this profile information at least a threshold likelihood that a user intends to perform a similar semantic search (e.g., to query columns of the type typically searched) [types of columns in this passage is interpreted as addressing 'column description']; Bischof shows at least column queries in ¶ 0045: Priming module 210 also determines, using the data warehouse graph, one or more likely columns within the data warehouse that the user intends to act on as part of the query … a profile reflects that a user queries specific columns [interpreted as addressing 'column name'] or types of columns frequently (or has done so in at least a most recent threshold number of days); Bischof is also interpreted as showing column names in ¶ 0055: priming module 210 may prime generative AI model 140 with the context of which data warehouse to query, what columns within that data warehouse to query)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the querying of Lee as modified with the querying of Bischof.
In addition, both of the references (Lee as modified and Bischof) disclose features that are directed to analogous art, and they are directed to the same field of endeavor, such as generative AI.
Motivation to do so would be to improve the functioning of Lee as modified performing queries with the functioning of similar reference Bischof also performing queries but with the improvement of determining likely user intent and user preferences.
Motivation to do so would also be the teaching, suggestion, or motivation for a person of ordinary skill in the art to mitigate inefficiencies and waste of network and compute resources as seen in Bischof ¶ 0002-0003.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Meltsner et al., U.S. Patent Application Publication No. 2025/0225428 (filed January 5, 2024), "Content Generation With Machine Learning-Augmented Summarization"; see Meltsner ¶ 0072-0073, "The system generates a prompt that includes a subset of data components determined to be relevant for inclusion in a summary report ... the generative AI model takes the prompt derived from the selected data components and transforms it into a comprehensive narrative that makes the data more comprehensible for human users. The generative AI model operates by utilizing techniques, such as deep learning, to construct a coherent summary report that captures the essence of the information within the data components" and Meltsner FIG. 4, ¶ 0128, "generates a human-readable natural language summary report ... The resulting summary report encapsulates key findings, such as performance metrics for different departments over time, resource optimization trends, and market trends, in a human-readable manner. It also provides key insights into workforce productivity"; relevant to at least the independent claim limitations involving a plain-language insight summary prompt and a plain-language insight summary of data attributes and corresponding attribute causes.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEDIDIAH P FERRER whose telephone number is (571)270-7695. The examiner can normally be reached Monday-Friday 12:00pm-8:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kavita Stanley can be reached at (571)272-8352. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.P.F/Examiner, Art Unit 2153 April 27, 2026
/KAVITA STANLEY/Supervisory Patent Examiner, Art Unit 2153
Prosecution Insights