Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The Action is responsive to the Request for Continued Examination, Amendments, and Remarks filed on 8/7/2026. Claims 1-9 and 11-19 are pending claims. Claim 1 is written in independent form. Claim 10 was previously cancelled. Claims 17-19 are newly added.
Priority
Applicant’s claim for benefit of prior-filed provisional applications 63/505,233 (filing date 5/31/2023) and 63/520,027 (filing date 8/16/2023) under 35. U.S.C. 119(e) or under 35 U.S.C. 120, 121, or 365(c) is acknowledged.
It is noted that the added limitations to Independent Claim 1 included in the amendments filed on 3/19/2026 are supported in at least paragraph [0021] of the cited provisional 63/520,027 (filing date 8/16/2023), but support could not be found in the cited provisional 63/505,233 (filing date 5/31/2023).
It is further noted that the added limitations to Independent Claim 1 included in amendments filed on 8/7/2026 are supported in at least paragraphs [0021], [0046], [0060], [00 and [0092] of the cited provisional 63/520,027 (filing date 8/16/2023), but support could not be found in the cited provisional 63/505,233 (filing date 5/31/2023). Therefore, Independent Claim 1 and Claims 2-9 and 11-19, which depend upon Claim 1, are being examined with an effective filing date of 8/16/2023 corresponding to provisional application 63/520,027.
Claim Interpretation
Claim 18 recites the term “programmatic interface” with Claim 19, depending upon Claim 18, reciting that “the programmatic interface comprises an application programming interface (API)”. Examiner notes that while the term “programmatic interface” is not explicitly recited in Applicant’s Specification, Applicant appears to be using it interchangeably with an application programming interface (API) which is recited in Paragraphs [0068] and [0079] of Applicant’s Specification (and Paragraphs [0066] and [0077] of the provisional application 63,520,027 dated 8/16/2023) and is understood as being a programmatic interface. Applicant is suggested to use API instead of “programmatic interface” to be consistent with the terminology explicitly used in the Specification.
Claim Objections
Claim 1 is objected to because of the following informalities:
Claim 1 recites “providing, to the LLM and using the selected one or more tools and based on the request, the information about the one or more properties of the one or more data object types using the selected one or more tools;” which states “using the selected one or more tools” twice related to “providing”. The limitation is understood as reciting “providing, to the LLM and using the selected one or more tools and based on the request, the information about the one or more properties of the one or more data object types
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 18-19 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement.
Dependent Claim 18 contains the subject matter “wherein a selected tool of the one or more tools comprises a programmatic interface…” which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.It is not clearly stated in the written description any recitation of a selected tool, let alone any tool, that “comprises a programmatic interface”.It is noted that Applicant does not provide supporting paragraphs for any of the amendments filed with the corresponding Remarks dated 8/7/2026. Upon further review of Applicant’s Specification and corresponding provisional applications, the only reference to any type of programmatic interface is reciting the use of an API in Paragraphs [0068] and [0079] of the Present Specification and Paragraphs [0066] and [0077] of the Specification corresponding to the Provisional Application 63/520,027. However, the references to an API were not found to support the claimed “selected tool…comprising a programmatic interface” as recited in Dependent Claim 18.For purposes of compact prosecution and based on Paragraphs [0068] and [0079] of the Present Specification and Paragraphs [0066] and [0077] of the Specification corresponding to the Provisional Application 63/520,027, the claim is being interpreted as reciting “wherein a selected tool of the one or more tools uses a programmatic interface…”.
Dependent Claim 19 inherits the deficiencies of its parent claim(s) and is therefore being rejected under 112(a) based upon the same reason(s) stated for its parent claim(s).
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 18-19 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claim 18, the limitation, as interpreted following the 112(a) rejection above, “wherein a selected tool of the one or more tools uses a programmatic interface…” renders the claims indefinite because when analyzed in light of the parent claim 1 reciting “receiving…a user selection of one or more tools”, it is unclear whether the “a selected tool of the one or more tools” of Claim 18 is referring to the “user selection of one or more tools” of the parent claim 1 or is referring to an unrelated/new selected tool of the same one or more tools recited in the parent claim 1.For purposes of compact prosecution and when combined with the interpretation following the 112(a) rejection above, the limitation is being understood as “wherein [[a]] the selected tool of the one or more tools uses a programmatic interface…”.
Dependent Claim 19 inherits the deficiencies of its parent claim(s) and is therefore being rejected under 112(b) based upon the same reason(s) stated for its parent claim(s).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-9 and 11-16 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature Maddigan et al., "Chat2VIS: Fine-Tuning Data Visualisations using Multilingual Natural Language Text and Pre-Trained Large Language Models", March 2023, arXiv.2303.14292, hereinafter referred to as Maddigan, and further in view of Maschmeyer et al. (U.S. Pre-Grant Publication No. 2024/0256764, hereinafter referred to as Maschmeyer) and Devaux et al. (U.S. Pre-Grant Publication No. 2024/0370691, hereinafter referred to as Devaux).
Regarding Claim 1:
Maddigan teaches a computer-implemented method for referencing a data set via a large language model, the computer-implemented method comprising, by one or more hardware processors executing program instructions:
Receiving, via a user interface, a natural language query,
Maddigan teaches “using the interface illustrated in Fig. 11, a user enters a request in the form of natural language text” (Page 4 Section 3.2).
Receiving indications of one or more data object types, wherein each of the one or more data object types is associated with a respective one or more properties;
Maddigan teaches receiving one or more data object types with the user request by teaching a request to “plot the outcome” with the indication of one or more data object types being the “outcome” (Page 4 Section 3.2 & Fig. 4(a) on Page 5). Maddigan further teaches that the data object types are associated with one or more properties by teaching in Fig. 4(b) “the column ‘Outcome’ has categorical values ‘Mismatch’, ‘Match’, ‘Error’”.
Receiving references to one or more data sets, wherein the one or more data sets are each associated with at least a respective data object type of the one or more data object types;
Maddigan teaches receiving a reference to “use a dataframe called df from data_file.csv “ (Fig. 4(b)) thereby teaching receiving a reference to one or more data sets associated with a data object type (columns ‘Code”, ‘Outcome’, ‘Difficulty’, ‘Database’).
Transmitting a prompt the large language model (“LLM”),
Maddigan teaches “Chat2VIS engineers the prompt, submits it to the chosen LLMs, formats the returned script, and renders the visualization” (Page 4 Section 3.2) thereby teaching transmitting a prompt to an LLM.
the prompt comprising at least: the natural language query, the indications of the one or more data object types, and the references to the one or more data sets; and
Maddigan teaches an engineered prompt in Fig. 4(b) comprising the natural language query “Plot the outcome”, the indications of the one or more data object types “outcome”, and the references to the data_file.csv data set.
Wherein the references to the one or more data sets each comprise respective unique identifiers; and
Maddigan teaches an engineered prompt in Fig. 4(b) comprising the references to the data_file.csv data set, which name of the data set is understood as being a unique identifier to the data set and when there is only one data set, there is only one reference, and thus only one unique identifier.
Receiving, from the LLM, a response to the prompt,
Wherein the response comprises indications of: at least a first reference to a first data set of the one or more data sets, and a structured query to be applied to the first data set,
Maddigan teaches “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a structured query to be executed on the particular data set data_file.csv to render the visualization in Fig. 4(e).
Wherein at least a portion of the structured query is generated by the LLM based on the information about the one or more data object types,
Maddigan teaches “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a structured query to be executed on the particular data set data_file.csv and includes information about the one or more data object types by teaching the use of “Outcome” in the return script.
Wherein the structured query is executable to obtain a subset of data of the first data set indicated by the first reference, and
Maddigan teaches “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a query executable on the particular data set data_file.csv to obtain a subset of the data from the data_file.csv to be used to render the visualization in Fig. 4(e).
Wherein at least a portion of the subset of data is responsive to the natural language query;
Maddigan teaches an engineered prompt in Fig. 4(b) comprising the natural language query “Plot the outcome” and “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a query executable on the particular data set data_file.csv to obtain a subset of the data from the data_file.csv to be used to render the visualization in Fig. 4(e), the subset of data being responsive to the natural language query because the natural language query was requesting to “plot the outcome” and the data shown in Figure 4e is a subset of the dataset including “outcome” data.
Executing the structured query on the first data set, using the first reference; and
Maddigan teaches “Chat2VIS engineers the prompt, submits it to the chosen LLMs, formats the returned script, and renders the visualization” (Page 4 Section 3.2) and “the newly-created script is executed to render the requested visualization, as pictured in Fig. 4(e)” (Page 5 Section 3.2) thereby teaching executing the script/query on the data set, using the reference, and obtaining a subset of data of the data set for visualization.
Obtaining, in response to executing the structured query on the first data set using the first reference, the subset of data of the first data set comprising the portion of the subset of data responsive to the natural language query.
Maddigan teaches an engineered prompt in Fig. 4(b) comprising the natural language query “Plot the outcome” and “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a query executable on the particular data set data_file.csv to obtain a subset of the data from the data_file.csv to be used to render the visualization in Fig. 4(e), the subset of data being responsive to the natural language query because the natural language query was requesting to “plot the outcome” and the data shown in Figure 4e is a subset of the dataset including “outcome” data.
Maddigan explicitly teaches all of the elements of the claimed invention as recited above except:
Based on the prompt, receiving, from the LLM, a request for information about one or more properties of the one or more data object types;
Providing, to the LLM and using the selected one or more tools and based on the request, the information about the one or more properties of the one or more data object types;
Wherein at least a portion of the structured query is generated by the LLM based on the information about the one or more properties of the one or more data object types,
Receiving a selection of one or more tools, wherein the selection of one or more tools identifies a set of instructions for a large language model (“LLM”);
the prompt comprising the set of instructions,
However, in the related field of endeavor of an interface for building a prompt to an LLM, Maschmeyer teaches:
Receiving a selection of one or more tools, wherein the selection of one or more tools identifies a set of instructions for a large language model (“LLM”);
Maschmeyer teaches “the object attribute(s) to be included in the object description may be selected” (Para. [0147]) and “a prompt to a LLM is generated (e.g., by the prompt generator 500). The LLM (which may be a generative pre-trained transformer LLM, such as GPT-3 or ChatGPT) is prompted to generate a description of the object. The generated prompt includes the object attribute(s) obtained at the operation 702 (and optionally selected at the operation 704) to include in the generated description” (Para. [0148]) thereby teaching instructions associated with the selected object attributes to be included in the prompt of what to be included in the generated output/description from the LLM.
the prompt comprising the set of instructions,
Maschmeyer teaches “the object attribute(s) to be included in the object description may be selected” (Para. [0147]) and “a prompt to a LLM is generated (e.g., by the prompt generator 500). The LLM (which may be a generative pre-trained transformer LLM, such as GPT-3 or ChatGPT) is prompted to generate a description of the object. The generated prompt includes the object attribute(s) obtained at the operation 702 (and optionally selected at the operation 704) to include in the generated description” (Para. [0148]) thereby teaching instructions included in the prompt of what to be included in the generated output/description from the LLM.
Thus, it would have been obvious to one of ordinary skill in the art, having the teachings of Maschmeyer and Maddigan at the time that the claimed invention was effectively filed, to have modified the fine-tuning data visualizations using multilingual natural language text and pre-trained large language models, as taught by Maddigan, with the systems and methods for using selected object attributes when generating a prompt to an LLM, as taught by Maschmeyer,.
One would have been motivated to make such combination because Maschmeyer teaches obtaining instructions for generating a prompt based both on obtained/received attributes and user-selected attributes (Paras. [0146] – [0148] & Fig. 4) and it would have been obvious to a person having ordinary skill in the art that adding the capability for a user to select attributes in combination with other methods of including attributes in a prompt would improve the user’s ability to personalize the prompt to the LLM, thus improving the personalization of the output from the LLM.
Maschmeyer and Maddigan explicitly teach all of the elements of the claimed invention as recited above except:
Based on the prompt, receiving, from the LLM, a request for information about one or more properties of the one or more data object types;
Providing, to the LLM and using the selected one or more tools and based on the request, the information about the one or more properties of the one or more data object types;
Wherein at least a portion of the structured query is generated by the LLM based on the information about the one or more properties of the one or more data object types,
However, in the related field of endeavor of generating a structured query using an LLM, Devaux teaches:
Based on the prompt, receiving, from the LLM, a request for information about one or more properties of the one or more data object types;
Devaux teaches, based on an initial prompt 504-1 to an LLM, “message 504-2 is generated by LLM engine 120 and sent via collaborator platform 104 to device 116-1 with the content “Sure! Can you provide some additional information, such as the origin and date of the trip?” Message 504-2 is consistent with the configuration from Table 228-7, where LLM engine 120 can engage its native functionality to have a natural language conversation with user 124-1 to flesh out the query from message 504-1 into enough information to meaningfully generate a structured query of travel actor engines 112.” (Paras. [0096 – [0098] & Figure 6).Maschmeyer further teaches other types of additional information being about one or more data object types by teaching “A prompt can include one or more examples of the desired output, which provides the LLM with additional information to enable the LLM to better generate output according to the desired output. Additionally or alternatively, the examples included in a prompt may provide inputs (e.g., example inputs) corresponding to/as may be expected to result in the desired outputs provided” (Para. [0082]) and “for each object, data about one or more object attributes (e.g., object name, object size, object type, object features, etc.). Object attribute(s) for a given object may for example, be stored in a lookup table that can be referenced using the name of the object, a unique identifier (e.g., identification number) of the object, etc” (Para. [0089])It is noted that Paragraphs [0023] and [0072] of the present Specification recite that “An example response from an LLM can include a request for one or more samples of data from one or more data sets (e.g., one or more data objects, data object types, properties, property types, or the like), a request for one or more additional unique identifiers, and/or a request including suggestions for narrowing search results requested by the user.” (Spec - Para. [0023]) and “Data Object (or “Object,” “Data Item,” “Item,” and/or the like) can be a data container for information representing a specific thing in the world that has a number of definable properties. For example, a data object can represent an entity such as a person, a place, an organization, a market instrument, or other noun.” (Spec – Para. [0072])
Therefore, Devaux teaches based on the prompt about a particular noun “trip”, receiving a request for more information about one or more definable properties of the “trip”, the particular properties being requested not included in the original prompt.
Providing, to the LLM and using the selected one or more tools and based on the request, the information about the one or more properties of the one or more data object types;
Devaux teaches, based on an initial prompt 504-1 to an LLM, a “message 504-2 is generated by LLM engine 120 and sent via collaborator platform 104 to device 116-1 with the content “Sure! Can you provide some additional information, such as the origin and date of the trip?”” and “user 124-1, via device 116-1, responds to the message 504-2 with the necessary additional information in the form of message 504-3 with the natural language text “I would like to travel the 12 of April from Nice”. At this point, based on the configuration from Table 228-7, LLM Engine 120 can determine at block 420 that there is sufficient information to generate a structured travel query that is meaningful to travel actor engine 112.” (Paras. [0096 – [0098] & Figure 6).
Wherein at least a portion of the structured query is generated by the LLM based on the information about the one or more properties of the one or more data object types,
Devaux teaches, based on an initial prompt 504-1 to an LLM, a “message 504-2 is generated by LLM engine 120 and sent via collaborator platform 104 to device 116-1 with the content “Sure! Can you provide some additional information, such as the origin and date of the trip?”” and “user 124-1, via device 116-1, responds to the message 504-2 with the necessary additional information in the form of message 504-3 with the natural language text “I would like to travel the 12 of April from Nice”. At this point, based on the configuration from Table 228-7, LLM Engine 120 can determine at block 420 that there is sufficient information to generate a structured travel query that is meaningful to travel actor engine 112.” (Paras. [0096 – [0098] & Figure 6).
Therefore, Devaux teaches the generation of the structured travel query by the LLM being based on the requested additional information/one or more properties (“such as the origin and date”) of the noun “trip”.
Thus, it would have been obvious to one of ordinary skill in the art, having the teachings of Devaux, Maschmeyer, and Maddigan at the time that the claimed invention was effectively filed, to have modified the fine-tuning data visualizations using multilingual natural language text and pre-trained large language models, as taught by Maddigan, and the systems and methods for using selected object attributes when generating a prompt to an LLM, as taught by Maschmeyer, with the iterative natural language conversation via the LLM engine towards generating a structured query, as taught by Devaux.
One would have been motivated to make such combination because Devaux teaches “repeated iterations of queries with slight modifications, resulting in wasted network traffic congestion and drain on server resources” (Para. [0003]) and reducing the drain on computing resources of search engines and related systems (Para. [0051]) by “determining whether there is sufficient information to complete the structured travel query.” (Para. [0096]) and “LLM engine 120 can direct questions to user 124-1 and receive further input from user 124-1 until a fully structured travel query can be generated.” (Para. [0097]) where “LLM engine 120 can engage its native functionality to have a natural language conversation with user 124-1 to flesh out the query from message 504-1 into enough information to meaningfully generate a structured query” (Para. [0098]).
Regarding Claim 2:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the indications of one or more data object types are, at least one of:
Specified by user via the user interface,
A user specifying via the user interface the object type “outcome” (Page 4 Section 3.2).
Specified by the selection of one or more tools, or
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2).
Defined by an ontology.
Regarding Claim 3:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the indications of one or more data object types are specified by the selection of one or more tools, and
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2).
Wherein the one or more tools include at least one of:
Query objects,
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2).
Apply actions, or
Ontology functions.
Regarding Claim 4:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the one or more data sets comprise at least one of:
Data objects of a first data object type of the one or more data object types, or
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome” which is also a natural language word associated with the data set.
A natural language word associated with a first data set of the one or more data sets.
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome” which is also a natural language word associated with the data set.
Regarding Claim 5:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the references to one or more data sets are associated with a selected tool of the one or more tools.
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2). Therefore, Maddigan teaches associating the references to the dataset with the selected tool to query objects in the dataset.
Regarding Claim 6:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the selected tool includes at least one of: query objects, apply actions, or ontology functions.
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2).
Regarding Claim 7:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the one or more data sets include each of the one or more data object types and one or more data objects of the respective one or more data object types.
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome” which is also a natural language word associated with the data set.
Regarding Claim 8:
Devaux, Maschmeyer, and Maddigan further teach:
Receiving and/or obtaining
Example data objects of the one or more data object types,
Maddigan teaches a dataset given with the natural language query in Fig. 4(a) thereby teaching example data objects of the data object types “outcome”.
One or more instructions to use the references in the response as part of the structured query, or
Maddigan teaches in instructions to use the dataset and references to particular aspects of the data in the query to plot the Outcome Fig. 4(a).
One or more instructions to limit the response to the first data set,
Maddigan teaches in instructions to limit the data to the dataset given with the natural language query in Fig. 4(a).
Wherein the prompt further comprises at least one of: the example data objects of the one or more data object types, the one or more instructions to use the references in the response as part of the structured query, or the one or more instructions to limit the response to a first data set.
Maddigan teaches in instructions to limit the data to the dataset, comprising the example data objects of the one or more data object types and references to the data set, given with the natural language query in Fig. 4(a) which is then included in the engineered prompt “use a data frame called dc from data_file.csv with columns ‘Code’, ‘Outcome’, ‘Difficulty’, ‘Database’.” (Fig. 4(b))
Regarding Claim 9:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the example data objects of the one or more data object types, the one or more instructions to use the references in response as part of the structured query, or the one or more instructions to limit the response to the first data set, are associated with a selected tool of the one or more tools, and
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2). Therefore, Maddigan teaches associating the selected tool to query data objects with at least one of the example input data objects in the dataset corresponding to the natural language request, the instructions to use the dataset and references to particular aspects of the data in the query to plot the Outcome and the instructions to limit the data to the dataset given with the natural language query (Pages 4-5 Section 3.2).
Wherein the selected tool includes at least one of: query objects, apply actions, or ontology functions.
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2)
Regarding Claim 11:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the first data set includes data objects of a first data object type of the one or more data object types.
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome” which is also a natural language word associated with the data set.
Regarding Claim 12:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the first data set is selected by the LLM based at least in part on the first data set including data objects of the first data object type.
Maddigan teaches “a user enters a request in the form of natural language text in reference to a selected dataset. Chat2VIS engineers the prompt, submits it to the chosen LLMs, formats the returned script, and renders the visualisation.” (Page. 4 Section 3.2) thereby teaching the LLM selecting the alredy “selected dataset” based in part on the data set being compatible including data objects of the first data object type included in the user’s request/prompt.
Regarding Claim 13:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the unique identifiers comprise respective one or more natural language words.
Maddigan teaches receiving a reference to “use a dataframe called df from data_file.csv “ (Fig. 4(b)) thereby teaching the unique identifier comprising natural language words of at least “data” and “file”.
Regarding Claim 14:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein each data object type is associated with at least one data set,
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome”.
Wherein each data set comprises data objects, and
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome”.
Wherein the data objects associated with the one or more properties of a subset of the data object types are included in at least one of the data sets.
Maddigan teaches that the data objects of column “Outcome” are associated with one or more properties of a subset of the data object type “Outcome”, the properties being one of the categorical values: Mismatch, Match, and Error. (Maddigan - Fig. 4(b): “the column ‘Outcome’ has categorical values ‘Mismatch’, ‘Match’, ‘Error’”)
Regarding Claim 15:
Devaux, Maschmeyer, and Maddigan further teach:
One or more computer-readable storage mediums having program instructions embodied therewith; and
Maddigan teaches “Early NL2VIS systems were built on symbolic-based NLP approaches, relying on heuristic algorithms [6], rule-based architectures, and probabilistic grammer-based methods for translating NL queries. Although each technique displayed increasing accuracy, they required more computational resources.” (Page 2 Section 2) thereby teaching computational resources such as storage mediums with instructions embodied therewith and processors configured to execute the instructions.
One or more processors configured to execute the program instructions to cause the system to perform the computer-implemented method.
Maddigan teaches “Early NL2VIS systems were built on symbolic-based NLP approaches, relying on heuristic algorithms [6], rule-based architectures, and probabilistic grammer-based methods for translating NL queries. Although each technique displayed increasing accuracy, they required more computational resources.” (Page 2 Section 2) thereby teaching computational resources such as storage mediums with instructions embodied therewith and processors configured to execute the instructions.
Regarding Claim 16:
Devaux, Maschmeyer, and Maddigan further teach:
A computer program product comprising one or more computer-readable storage mediums having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform the computer-implemented method.
Maddigan teaches “Early NL2VIS systems were built on symbolic-based NLP approaches, relying on heuristic algorithms [6], rule-based architectures, and probabilistic grammer-based methods for translating NL queries. Although each technique displayed increasing accuracy, they required more computational resources.” (Page 2 Section 2) thereby teaching computational resources such as storage mediums with instructions embodied therewith and processors configured to execute the instructions.
Regarding Claim 17:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the prompt does not include the respective one or more properties of the one or more data object types.
Devaux teaches, based on an initial prompt 504-1 to an LLM, “message 504-2 is generated by LLM engine 120 and sent via collaborator platform 104 to device 116-1 with the content “Sure! Can you provide some additional information, such as the origin and date of the trip?” Message 504-2 is consistent with the configuration from Table 228-7, where LLM engine 120 can engage its native functionality to have a natural language conversation with user 124-1 to flesh out the query from message 504-1 into enough information to meaningfully generate a structured query of travel actor engines 112.” (Paras. [0096 – [0098] & Figure 6).Maschmeyer further teaches other types of additional information being about one or more data object types by teaching “A prompt can include one or more examples of the desired output, which provides the LLM with additional information to enable the LLM to better generate output according to the desired output. Additionally or alternatively, the examples included in a prompt may provide inputs (e.g., example inputs) corresponding to/as may be expected to result in the desired outputs provided” (Para. [0082]) and “for each object, data about one or more object attributes (e.g., object name, object size, object type, object features, etc.). Object attribute(s) for a given object may for example, be stored in a lookup table that can be referenced using the name of the object, a unique identifier (e.g., identification number) of the object, etc” (Para. [0089])It is noted that Paragraphs [0023] and [0072] of the present Specification recite that “An example response from an LLM can include a request for one or more samples of data from one or more data sets (e.g., one or more data objects, data object types, properties, property types, or the like), a request for one or more additional unique identifiers, and/or a request including suggestions for narrowing search results requested by the user.” (Spec - Para. [0023]) and “Data Object (or “Object,” “Data Item,” “Item,” and/or the like) can be a data container for information representing a specific thing in the world that has a number of definable properties. For example, a data object can represent an entity such as a person, a place, an organization, a market instrument, or other noun.” (Spec – Para. [0072])
Therefore, Devaux teaches based on the prompt about a particular noun “trip”, receiving a request for more information about one or more definable properties of the “trip”, the particular properties being requested not included in the original prompt.
Regarding Claim 18:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the selected tool of the one or more tools uses a programmatic interface, and
Devaux teaches “Collaboration platform 104 can also be incorporated into travel management engine 122. Collaboration platform 104, LLM Engine 120 and travel management engine 122 can be incorporated into a single platform. Likewise, the functionalities of collaboration platform 104 and/or travel management engine 122 and/or LLM engine 120 can be incorporated into one or more travel actor engines 112, either as a full hardware integration and/or a software integration via API calls.” (Para. [0182]) and “The teachings herein can be incorporated into conversational search by integration into a travel agent chatbot as source (API) of prices for indication purposes.” (Para. [0213]) thereby teaching the functionalities/tools of the system using an API, which is understood as being a programmatic interface.
Wherein providing to the LLM the information about the one or more properties of the one or more data object types comprises invoking an application programming interface (API) to retrieve the information about the one or more properties of the one or more data object types.
Devaux teaches “Collaboration platform 104 can also be incorporated into travel management engine 122. Collaboration platform 104, LLM Engine 120 and travel management engine 122 can be incorporated into a single platform. Likewise, the functionalities of collaboration platform 104 and/or travel management engine 122 and/or LLM engine 120 can be incorporated into one or more travel actor engines 112, either as a full hardware integration and/or a software integration via API calls.” (Para. [0182]) and the example of “The teachings herein can be incorporated into conversational search by integration into a travel agent chatbot as source (API) of prices for indication purposes.” (Para. [0213]) thereby teaching using an invoked API to retrieve the additional information that can subsequently be provided to an LLM.
Regarding Claim 19:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the programmatic interface comprises an application programming interface (API).
Devaux teaches “Collaboration platform 104 can also be incorporated into travel management engine 122. Collaboration platform 104, LLM Engine 120 and travel management engine 122 can be incorporated into a single platform. Likewise, the functionalities of collaboration platform 104 and/or travel management engine 122 and/or LLM engine 120 can be incorporated into one or more travel actor engines 112, either as a full hardware integration and/or a software integration via API calls.” (Para. [0182]) thereby teaching using an API as a programmatic interface.
Response to Amendment
Applicant’s Amendments, filed on 8/7/2026, are acknowledged.
In light of the Amendments filed on 8/7/2026, the claim objections to claims 1, 8, and 9 have been withdrawn.
Response to Arguments
On page 8 of the Remarks filed on 8/7/2026, Applicant argues that “Devaux does not teach or suggest transmitting a prompt, and, based on the prompt, receiving [a request for] information about one or more properties of the one or more data object types, as recited in amended Claim 1. In Devaux, the LLM engine asks the user to supply missing query parameters, namely, an origin ("from Nice") and a date ("the 12 of April"), that are needed to construct a "structured travel query." Devaux at [0096]-[0098]. Those parameters are not properties of data object types, as recited by amended Claim 1. Amended Claim 1, by contrast, expressly requires that, using the selected one or more tools and based on the request, providing... the information about the one or more properties of the one or more data object types.”
Applicant’s argument is not convincing because based on a review of Paragraphs [0023] and [0072] of Applicant’s specification and the Devaux reference, Devaux is understood as teaching the amended language similar to how the terminology it is described in Applicant’s specification. The amended language has been fully addressed in the rejection above including a detailed explanation of how Applicant’s specification relating to the claim language is taught by Devaux.
On page 8 of the Remarks filed on 8/7/2026, Applicant argues that “in Devaux, the additional information is supplied by the user typing another natural language message via the user interface, not by the system using a selected tool to look up and provide that information to the LLM. Devaux at [0097]-[0098]. Amended Claim 1 expressly requires that the information be provided "using the selected one or more tools," which are the same tools identified by the "user selection of one or more tools" received via the user interface earlier in the claim, and which identify the set of instructions for the LLM. Nothing in Devaux describes or suggests any such tool-based lookup or provision of information about data object type properties in response to an LLM-issued request.”Applicant’s argument is not convincing because the claims do not appear to reflect the scope being argued. The scope argued with respect to what Applicant’s states Devaux does not teach, “using a selected tool to look up and provide that information to the LLM”, is understood as obtaining the information using the selected one or more tools and then providing the obtained information, using the selected tool and based on the request, to the LLM. However, Claim 1 recites a scope of providing the information to the LLM using the selected one or more tools and based on the request. It is further noted the claims recite “selection of one or more tools identifies a set of instructions for a large language model” which is unrelated to accessing/obtaining extra information, separate from an LLM, using selected tools and then providing that information to the LLM.
On pages 8-9 of the Remarks filed on 8/7/2026, Applicant argues that “Maschmeyer does not cure these deficiencies of Maddigan and Devaux. The Office Action relies on Maschmeyer, at paragraphs [0147]-[0148], for teaching a selection of one or more tools that identifies a set of instructions for the LLM and a prompt comprising the set of instructions. Office Action at 8. As Applicant previously argued, however, Maschmeyer describes that its "prompt generator," a computing-system component, selects the object attributes to be included in the object description. Maschmeyer does not describe a user selection of a tool that identifies a set of instructions for the LLM. See Maschmeyer at [0141]-[0142] and [0145]-[0147]. Moreover, Maschmeyer's attribute selection occurs on the initial-prompt side. Maschmeyer does not describe or suggest any subsequent LLM-issued request for information about properties of data object types, nor any provision of such information to the LLM using selected tools in response to such a request.Applicant’s argument is moot because it was never stated in the previous rejection nor herein that Maschmeyer cured all of the argued deficiencies of Maddigan and Devaux. However, while not relied upon in the rejection herein, it is noted that Maschmeyer does recite “For functions that are not included within the commerce management engine 136, applications 142A-B provide a way to add features to the e-commerce platform 100 or individual online stores 138. For example, applications 142A-B may be able to access and modify data on a merchant's online store 138, perform tasks through the administrator 114, implement new flows for a merchant through a user interface (e.g., that is surfaced through extensions/API), and the like” (Para. [0177]), “For instance, the e-commerce platform 100 may provide API interfaces 140A-B to applications 142A-B which may connect to products and services external to the platform 100. The flexibility offered through use of applications and APIs (e.g., as offered for application development) enable the e-commerce platform 100 to better accommodate new and unique needs of merchants or to address specific use cases without requiring constant change to the commerce management engine 136. For instance, shipping services 122 may be integrated with the commerce management engine 136 through a shipping or carrier service API, thus enabling the e-commerce platform 100 to provide shipping service functionality without directly impacting code running in the commerce management engine 136.” (Para. [0179]), “Depending on the implementation, applications 142A-B may utilize APIs to pull data on demand (e.g., customer creation events, product change events, or order cancelation events, etc.) or have the data pushed when updates occur” (Para. [0180]), and “The association between a product listing and a channel may be modeled as a product publication and accessed by channel applications, such as via a product listing API. A product may have many attributes and/or characteristics, like size and color, and many variants that expand the available options into specific combinations of all the attributes, like a variant that is size extra-small and green, or a variant that is size large and blue.” (Para. [0184]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Non-Patent Literature Dang et al., "How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models", September 2022, arXiv:2209.01390 teaches prompting a generative model is a promising recent development that in principle enables end-users to creatively leverage zero-shot and few-shot learning to assign new tasks to an AI adhoc, simply by writing them down. However, for the majority of end-users writing effective prompts is currently largely a trial and error process. To address this, we discuss the key opportunities and challenges for interactive creative applications that use prompting as a new paradigm for Human-AI interaction.
Non-Patent Literature Sun et al., "SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL", May 26, 2023, arXiv:2306.00739 teaches prompting a generative model is a promising recent development that in principle enables end-users to creatively leverage zero-shot and few-shot learning to assign new tasks to an AI adhoc, simply by writing them down. However, for the majority of end-users writing effective prompts is currently largely a trial and error process. To address this, we discuss the key opportunities and challenges for interactive creative applications that use prompting as a new paradigm for Human-AI interaction.
Dong et al. (U.S. Pre-Grant Publication No. 2024/0311579) teaches techniques that may generate prompts for language models. The techniques include obtaining a first dataset and a second dataset and training a hierarchical virtual token generator (VTG) model to generate a large language model (LLM) input prompt. Training the hierarchical VTG includes training, based on the first dataset, a first VTG to output a first virtual token and training, based on the second dataset, a second VTG to output a second virtual token embedding. The generated LLM input prompt includes the first virtual token embedding and the second virtual token embedding.
Cheng et al. (U.S. Pre-Grant Publication No. 2020/0395007) teaches an action agent architecture in a scalable multi-service virtual assistant platform that can construct a fluid and dynamic dialogue by assembling responses to end user utterances from two kinds of agents, information agents and action agents. The virtual assistant platform comprises a plurality of action agents to perform two or more actions in response to parsed user input, contextual data, and/or an information value obtained from an information agent. The plurality of action agents are interrelated via at least one follow-up connection which interrelates any two of the action agents such that a second action agent follows-up with a second action after a first action agent completes a first action based on a trigger condition. The second action agent utilizes at least one information value set by the first action agent or a system state change triggered by the first action agent.
Procter et al. (U.S. Pre-Grant Publication No. 2024/0354320) teaches a thought object selection server receives a plurality of thought objects and the thought objects include text present in qualitative responses from plurality of user devices in a communication environments. The plurality of thought objects consists of M thought objects that are most recently seen thought objects and N thought objects that are least seen by the plurality of user devices. A prompt is provided to the LLM with the M thought objects, the N thought objects, and a request to identify diverse thought objects. LLM compares the M thought objects and the N thought objects to identify dissimilar one or more dissimilar thought objects from the N thought objects based on semantic distances. One thought object is selected from the dissimilar thought objects as a diverse thought object. In some cases, non-transformer based embedding tools may be used to identify diverse thought in the received thought objects.
Schaefer et al. (U.S. Pre-Grant Publication No. 2024/0311093) teaches an elidable text is constructed that prioritizes the content included in a prompt to a large language model having a fixed-size context window. The elidable text is generated from developer-generated instructions or automatically for source code within a source code editor. A source code editor may include a feature that selects certain lines of code as important or focused which are assigned a high-priority value. A changed line, a line of source code at a current cursor position, lines of source code at the beginning of a file and those that output data are considered focused lines. Non-focused lines are assigned a priority based on a distance from a focused line. The elidable text constrains the data included in a prompt to the context window size by replacing the lowest-valued lines of text and source code with a replacement string.The reference further teaches “the developer may select one or several source code statements as a focused line or the elidable text generator may select a line in the current cursor position as the focused line.” (Para. [0033]) and a method comprising “accessing a large language model to perform a task given a prompt that fits within a context window size; monitoring a source code program in a software development tool; selecting a focus line from the source code program; associating a high-priority value to the focus line; associating a priority value to a non-focused line based on a distance from the focus line; constructing a prompt to the large language model from select lines of the source code program; replacing the select lines of the prompt with a replacement string based on a corresponding priority value when size of the prompt exceeds size of the context window size; and applying the prompt to the large language model for performance of the task.” (Para. [0061]).
Bennet et al. (U.S. Pre-Grant Publication No. 2025/0005655) teaches “follow up information manually provided by the user in accordance with a type-2 prompt in which the LLM makes at least one additional information request from a user. For example, a user may provide “customer response data” when responding to the at least one additional LLM prompt “What is your question about product X?.”” (Para. [0031] and “The term “Type-2 clickable prompt button”, as referred to herein, may refer to an embodiment in which a large language model (LLM) makes at least one additional information request from a user to obtain further context information from the user.” (Para. [0033]).
Wu et al. (U.S. Pre-Grant Publication No. 2024/0419902) teaches “an LLM (or other language model type(s)) may retrieve and/or access map data or other information determined to be necessary to generate an output using one or more application programming interfaces (APIs) and/or plug-ins (e.g., third-party plug-ins). For example, in order to retrieve additional contextual information, additional map information, additional feature information, and/or other information not directly included in a prompt to the model, the system—using the LLM, in some embodiments—may generate one or more prompts or queries for one or more data sources (e.g., open street maps (OSM), wolfram alpha, a local map database, etc.), via one or more APIs or plug-ins, in order to obtain the additional information required (or deemed necessary) for responding to the initial query or prompt. Such an approach to querying additional resources may be recursive, in at least some embodiments, in that the system may continue to access one or more data sources via the API(s) and/or plug-ins until it is determined the necessary information has been obtained, or until no additional information is available” (Para. [0145]).
Siracusano et al. (U.S. Pre-Grant Publication No. 2024/0411994) teaches “The data acquisition module 308 can decide whether the information is enough for the LLM 312 to generate a response (e.g., an accurate or helpful response), or if the data acquisition module 308 is required to search for additional information” (Para. [0067]).
Shazeer et al. (U.S. Pre-Grant Publication No. 2022/0374608) teaches one or more machine-learned language models that generate intermediate textual analysis (e.g., including usage of structural tools such as APIs) in service of contextual text generation. For example, a computing system can obtain a contextual text string that includes one or more contextual text tokens. The computing system can process the contextual text string with the machine-learned language model to generate one or more intermediate text strings that include one or more intermediate text tokens. The computing system can process the one or more intermediate text strings with the machine-learned language model to generate an output text string comprising one or more output text tokens. The one or more intermediate text strings can include textual analysis of the contextual text string that supports the output text string.The reference further teaches “As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database; an application programming interface (API) call to request and receive additional information via the API;” (Para. [0025]) and “according to an aspect of the present disclosure, in some implementations, the one or more intermediate text tokens 16 can include at least one tool token that invokes use of a structural tool 15 to access additional information not included in the contextual text string 12. In some implementations, the structural tool 15 can include a database lookup to access additional information from a database. In some implementations, the structural tool 15 can include an application programming interface (API) call to request and receive additional information via the API.” (Para. [0043]).
Agarwal et al. (U.S. Pre-Grant publication No. 2025/0335508) teaches automatically detecting and responding to incomplete user queries. Embodiments include receiving a query from a user; generating a planning language representation of the query using a language processing machine learning model; determining that the query did not provide a particular item of information, wherein the particular item of information is used for executing an execution plan; generating the execution plan based on the planning language representation using an artificial intelligence (AI) planner, wherein the execution plan includes requesting that the user provide the one or more particular items of information; executing the execution plan, comprising requesting the particular item of information from the user; and generating a response to the user query based on the executing of the execution plan.
Fan et al. (U.S. Pre-Grant Publication No. 2025/0078823) teaches techniques for determining one or more responses associated with one or more components that are responsive to a user input are described. The system receives a user input and causes one or more components to generate one or more responses associated with the user input. The system determines one or more of the responses are responsive to the user input, causes one or more actions associated with the responses to be performed, and outputs a natural language summary of the one or more responses. If the system determines that none of the responses are responsive to the user input and/or an ambiguity exists with respect to the user input, the system can generate a request for additional information usable to resolve the ambiguity, which may be sent to another component of the system and/or output to the user that provided the user input.
Foreign Publication CN119513139A teaches an NL2SQL method and system based on LLM recursion recall and self-correct, the method comprises: according to the user query and the table element information of the database, driving the LLM to generate an initial recall table set, performing recursion expansion on the table in the SEED_SET, in the virtual database environment, according to the V-FK supplementary deletion table, using FIN-RECALL-SET and SQL-HINT to construct SQL-GEN-PROMPT, if execution fails, recording error, and correcting according to prompt, repeating the process until the maximum correction times is reached, if SQL query return result is empty, judging whether it is caused by the false filtering condition through NLI and vector retrieval, if it is determined that there is error in the filtering condition, then modifying the filtering value in the SQL, updating the query sentence and re-executing it, ensuring the SQL result to be accurate, sorting all the successfully executed SQL query sentences according to the quality, selecting the best query sentence as the final output, returning the SQL result needed by the user.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT F MAY whose telephone number is (571)272-3195. The examiner can normally be reached Monday-Friday 9:30am to 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached on 571-270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBERT F MAY/Examiner, Art Unit 2154 9/14/2026
Prosecution Insights