Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The Action is responsive to the Amendments and Remarks filed on 3/19/2026. Claims 1-9 and 11-16 are pending claims. Claim 1 is written in independent form. Claim 10 was previously cancelled.
Priority
Applicant’s claim for benefit of prior-filed provisional applications 63/505,233 (filing date 5/31/2023) and 63/520,027 (filing date 8/16/2023) under 35. U.S.C. 119(e) or under 35 U.S.C. 120, 121, or 365(c) is acknowledged. It is noted that the newly added limitations to Independent Claim 1 included in the amendments filed on 3/19/2026 are supported in at least paragraph [0021] of the cited provisional 63/520,027 (filing date 8/16/2023), but support could not be found in the cited provisional 63/505,233 (filing date 5/31/2023). Therefore, Independent Claim 1 and Claims 2-9 and 11-16, which depend upon Claim 1, are being examined with an effective filing date of 8/16/2023 corresponding to provisional application 63/520,027.
Claim Objections
Claims 1, 8 and 9 are objected to because of the following informalities:
Claim 1 appears to recite a typographical error in the newly added “providing” limitation by reciting “based on the prompt, receiving, from the LLM, a request for information”, “providing…the information about the one or more data objects types using the selected one or more tools”, and “receiving, from the LLM, a response to the prompt…” without specifying that the providing is being done to the LLM. Based on at least paragraph [0062] of the present Specification, the “providing” limitation is understood as reciting “Providing to the LLM, using the selected one or more tools and based on the request, the information about the one or more data object types using the selected one or more tools;”,
Claims 8 and 9 appears to recite a typographical error by reciting “the query” when Independent Claim 1 upon which the claims depend has been amended to recite “a structured query”. Claims 8 and 9 are understood as being intended to recite “the structured query” instead of “the query” to stay consistent with the amendment to the antecedent language in Claim 1.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-9 and 11-16 are rejected under 35 U.S.C. 103 as being unpatentable over Non-Patent Literature Maddigan et al., "Chat2VIS: Fine-Tuning Data Visualisations using Multilingual Natural Language Text and Pre-Trained Large Language Models", March 2023, arXiv.2303.14292, hereinafter referred to as Maddigan, and further in view of Maschmeyer et al. (U.S. Pre-Grant Publication No. 2024/0256764, hereinafter referred to as Maschmeyer) and Devaux et al. (U.S. Pre-Grant Publication No. 2024/0370691, hereinafter referred to as Devaux).
Regarding Claim 1:
Maddigan teaches a computer-implemented method for referencing a data set via a large language model, the computer-implemented method comprising, by one or more hardware processors executing program instructions:
Receiving, via a user interface, a natural language query,
Maddigan teaches “using the interface illustrated in Fig. 11, a user enters a request in the form of natural language text” (Page 4 Section 3.2).
Receiving indications of one or more data object types, wherein each of the one or more data object types is associated with a respective one or more properties;
Maddigan teaches receiving one or more data object types with the user request by teaching a request to “plot the outcome” with the indication of one or more data object types being the “outcome” (Page 4 Section 3.2 & Fig. 4(a) on Page 5). Maddigan further teaches that the data object types are associated with one or more properties by teaching in Fig. 4(b) “the column ‘Outcome’ has categorical values ‘Mismatch’, ‘Match’, ‘Error’”.
Receiving references to one or more data sets, wherein the one or more data sets are each associated with at least a respective data object type of the one or more data object types;
Maddigan teaches receiving a reference to “use a dataframe called df from data_file.csv “ (Fig. 4(b)) thereby teaching receiving a reference to one or more data sets associated with a data object type (columns ‘Code”, ‘Outcome’, ‘Difficulty’, ‘Database’).
Transmitting a prompt the large language model (“LLM”),
Maddigan teaches “Chat2VIS engineers the prompt, submits it to the chosen LLMs, formats the returned script, and renders the visualization” (Page 4 Section 3.2) thereby teaching transmitting a prompt to an LLM.
the prompt comprising at least: the natural language query, the indications of the one or more data object types, and the references to the one or more data sets; and
Maddigan teaches an engineered prompt in Fig. 4(b) comprising the natural language query “Plot the outcome”, the indications of the one or more data object types “outcome”, and the references to the data_file.csv data set.
Wherein the references to the one or more data sets each comprise respective unique identifiers; and
Maddigan teaches an engineered prompt in Fig. 4(b) comprising the references to the data_file.csv data set, which name of the data set is understood as being a unique identifier to the data set and when there is only one data set, there is only one reference, and thus only one unique identifier.
Receiving, from the LLM, a response to the prompt,
Wherein the response comprises indications of: at least a first reference to a first data set of the one or more data sets, and a structured query to be applied to the first data set.
Maddigan teaches “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a structured query to be executed on the particular data set data_file.csv to render the visualization in Fig. 4(e).
Wherein at least a portion of the structured query is generated by the LLM based on the information about the one or more data object types,
Maddigan teaches “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a structured query to be executed on the particular data set data_file.csv and includes information about the one or more data object types by teaching the use of “Outcome” in the return script.
Wherein the structured query is executable to obtain a subset of data of the first data set indicated by the first reference, and
Maddigan teaches “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a query executable on the particular data set data_file.csv to obtain a subset of the data from the data_file.csv to be used to render the visualization in Fig. 4(e).
Wherein at least a portion of the subset of data is responsive to the natural language query; and
Maddigan teaches an engineered prompt in Fig. 4(b) comprising the natural language query “Plot the outcome” and “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a query executable on the particular data set data_file.csv to obtain a subset of the data from the data_file.csv to be used to render the visualization in Fig. 4(e), the subset of data being responsive to the natural language query because the natural language query was requesting to “plot the outcome” and the data shown in Figure 4e is a subset of the dataset including “outcome” data.
Executing the structured query on the first data set, using the first reference; and
Maddigan teaches “Chat2VIS engineers the prompt, submits it to the chosen LLMs, formats the returned script, and renders the visualization” (Page 4 Section 3.2) and “the newly-created script is executed to render the requested visualization, as pictured in Fig. 4(e)” (Page 5 Section 3.2) thereby teaching executing the script/query on the data set, using the reference, and obtaining a subset of data of the data set for visualization.
Obtaining, in response to executing the structured query on the first data set using the first reference, the subset of data of the first data set comprising the portion of the subset of data responsive to the natural language query.
Maddigan teaches an engineered prompt in Fig. 4(b) comprising the natural language query “Plot the outcome” and “upon submission of the prompt to the selected LLMs, Fig. 4(c) details the returned script (red) – a continuation of the code section with in the engineered prompt” (Page 4 Section 3.2) where the return script is for a query executable on the particular data set data_file.csv to obtain a subset of the data from the data_file.csv to be used to render the visualization in Fig. 4(e), the subset of data being responsive to the natural language query because the natural language query was requesting to “plot the outcome” and the data shown in Figure 4e is a subset of the dataset including “outcome” data.
Maddigan explicitly teaches all of the elements of the claimed invention as recited above except:
Based on the prompt, receiving, from the LLM, a request for information about the one or more data object types;
Providing to the LLM, using the selected one or more tools and based on the request, the information about the one or more data object types using the selected one or more tools;
Receiving a selection of one or more tools, wherein the selection of one or more tools identifies a set of instructions for a large language model (“LLM”);
the prompt comprising the set of instructions,
However, in the related field of endeavor of an interface for building a prompt to an LLM, Maschmeyer teaches:
Receiving a selection of one or more tools, wherein the selection of one or more tools identifies a set of instructions for a large language model (“LLM”);
Maschmeyer teaches “the object attribute(s) to be included in the object description may be selected” (Para. [0147]) and “a prompt to a LLM is generated (e.g., by the prompt generator 500). The LLM (which may be a generative pre-trained transformer LLM, such as GPT-3 or ChatGPT) is prompted to generate a description of the object. The generated prompt includes the object attribute(s) obtained at the operation 702 (and optionally selected at the operation 704) to include in the generated description” (Para. [0148]) thereby teaching instructions associated with the selected object attributes to be included in the prompt of what to be included in the generated output/description from the LLM.
the prompt comprising the set of instructions,
Maschmeyer teaches “the object attribute(s) to be included in the object description may be selected” (Para. [0147]) and “a prompt to a LLM is generated (e.g., by the prompt generator 500). The LLM (which may be a generative pre-trained transformer LLM, such as GPT-3 or ChatGPT) is prompted to generate a description of the object. The generated prompt includes the object attribute(s) obtained at the operation 702 (and optionally selected at the operation 704) to include in the generated description” (Para. [0148]) thereby teaching instructions included in the prompt of what to be included in the generated output/description from the LLM.
Thus, it would have been obvious to one of ordinary skill in the art, having the teachings of Maschmeyer and Maddigan at the time that the claimed invention was effectively filed, to have modified the fine-tuning data visualizations using multilingual natural language text and pre-trained large language models, as taught by Maddigan, with the systems and methods for using selected object attributes when generating a prompt to an LLM, as taught by Maschmeyer,.
One would have been motivated to make such combination because Maschmeyer teaches obtaining instructions for generating a prompt based both on obtained/received attributes and user-selected attributes (Paras. [0146] – [0148] & Fig. 4) and it would have been obvious to a person having ordinary skill in the art that adding the capability for a user to select attributes in combination with other methods of including attributes in a prompt would improve the user’s ability to personalize the prompt to the LLM, thus improving the personalization of the output from the LLM.
Maschmeyer and Maddigan explicitly teach all of the elements of the claimed invention as recited above except:
Based on the prompt, receiving, from the LLM, a request for information about the one or more data object types;
Providing to the LLM, using the selected one or more tools and based on the request, the information about the one or more data object types using the selected one or more tools;
However, in the related field of endeavor of generating a structured query using an LLM, Devaux teaches:
Based on the prompt, receiving, from the LLM, a request for information about the one or more data object types;
Devaux teaches, based on an initial prompt 504-1 to an LLM, “message 504-2 is generated by LLM engine 120 and sent via collaborator platform 104 to device 116-1 with the content “Sure! Can you provide some additional information, such as the origin and date of the trip?” Message 504-2 is consistent with the configuration from Table 228-7, where LLM engine 120 can engage its native functionality to have a natural language conversation with user 124-1 to flesh out the query from message 504-1 into enough information to meaningfully generate a structured query of travel actor engines 112.” (Paras. [0096 – [0098] & Figure 6).Maschmeyer further teaches other types of additional information being about one or more data object types by teaching “A prompt can include one or more examples of the desired output, which provides the LLM with additional information to enable the LLM to better generate output according to the desired output. Additionally or alternatively, the examples included in a prompt may provide inputs (e.g., example inputs) corresponding to/as may be expected to result in the desired outputs provided” (Para. [0082]) and “for each object, data about one or more object attributes (e.g., object name, object size, object type, object features, etc.). Object attribute(s) for a given object may for example, be stored in a lookup table that can be referenced using the name of the object, a unique identifier (e.g., identification number) of the object, etc” (Para. [0089])It is noted that Paragraph [0023] of the present Specification recites that “An example response from an LLM can include a request for one or more samples of data from one or more data sets (e.g., one or more data objects, data object types, properties, property types, or the like), a request for one or more additional unique identifiers, and/or a request including suggestions for narrowing search results requested by the user.”
Providing to the LLM, using the selected one or more tools and based on the request, the information about the one or more data object types using the selected one or more tools;
Devaux teaches, based on an initial prompt 504-1 to an LLM, a “message 504-2 is generated by LLM engine 120 and sent via collaborator platform 104 to device 116-1 with the content “Sure! Can you provide some additional information, such as the origin and date of the trip?”” and “user 124-1, via device 116-1, responds to the message 504-2 with the necessary additional information in the form of message 504-3 with the natural language text “I would like to travel the 12 of April from Nice”. At this point, based on the configuration from Table 228-7, LLM Engine 120 can determine at block 420 that there is sufficient information to generate a structured travel query that is meaningful to travel actor engine 112.” (Paras. [0096 – [0098] & Figure 6).
Thus, it would have been obvious to one of ordinary skill in the art, having the teachings of Devaux, Maschmeyer, and Maddigan at the time that the claimed invention was effectively filed, to have modified the fine-tuning data visualizations using multilingual natural language text and pre-trained large language models, as taught by Maddigan, and the systems and methods for using selected object attributes when generating a prompt to an LLM, as taught by Maschmeyer, with the iterative natural language conversation via the LLM engine towards generating a structured query, as taught by Devaux.
One would have been motivated to make such combination because Devaux teaches “repeated iterations of queries with slight modifications, resulting in wasted network traffic congestion and drain on server resources” (Para. [0003]) and reducing the drain on computing resources of search engines and related systems (Para. [0051]) by “determining whether there is sufficient information to complete the structured travel query.” (Para. [0096]) and “LLM engine 120 can direct questions to user 124-1 and receive further input from user 124-1 until a fully structured travel query can be generated.” (Para. [0097]) where “LLM engine 120 can engage its native functionality to have a natural language conversation with user 124-1 to flesh out the query from message 504-1 into enough information to meaningfully generate a structured query” (Para. [0098]).
Regarding Claim 2:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the indications of one or more data object types are, at least one of:
Specified by user via the user interface,
A user specifying via the user interface the object type “outcome” (Page 4 Section 3.2).
Specified by the selection of one or more tools, or
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2).
Defined by an ontology.
Regarding Claim 3:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the indications of one or more data object types are specified by the selection of one or more tools, and
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2).
Wherein the one or more tools include at least one of:
Query objects,
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2).
Apply actions, or
Ontology functions.
Regarding Claim 4:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the one or more data sets comprise at least one of:
Data objects of a first data object type of the one or more data object types, or
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome” which is also a natural language word associated with the data set.
A natural language word associated with a first data set of the one or more data sets.
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome” which is also a natural language word associated with the data set.
Regarding Claim 5:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the references to one or more data sets are associated with a selected tool of the one or more tools.
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2). Therefore, Maddigan teaches associating the references to the dataset with the selected tool to query objects in the dataset.
Regarding Claim 6:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the selected tool includes at least one of: query objects, apply actions, or ontology functions.
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2).
Regarding Claim 7:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the one or more data sets include each of the one or more data object types and one or more data objects of the respective one or more data object types.
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome” which is also a natural language word associated with the data set.
Regarding Claim 8:
Devaux, Maschmeyer, and Maddigan further teach:
Receiving and/or obtaining
Example data objects of the one or more data object types,
Maddigan teaches a dataset given with the natural language query in Fig. 4(a) thereby teaching example data objects of the data object types “outcome”.
One or more instructions to use the references in the response as part of the query, or
Maddigan teaches in instructions to use the dataset and references to particular aspects of the data in the query to plot the Outcome Fig. 4(a).
One or more instructions to limit the response to the first data set,
Maddigan teaches in instructions to limit the data to the dataset given with the natural language query in Fig. 4(a).
Wherein the prompt further comprises at least one of: the example data objects of the one or more data object types, the one or more instructions to use the references in the response as part of the query, or the one or more instructions to limit the response to a first data set.
Maddigan teaches in instructions to limit the data to the dataset, comprising the example data objects of the one or more data object types and references to the data set, given with the natural language query in Fig. 4(a) which is then included in the engineered prompt “use a data frame called dc from data_file.csv with columns ‘Code’, ‘Outcome’, ‘Difficulty’, ‘Database’.” (Fig. 4(b))
Regarding Claim 9:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the example data objects of the one or more data object types, the one or more instructions to use the references in response as part of the query, or the one or more instructions to limit the response to the first data set, are associated with a selected tool of the one or more tools, and
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2). Therefore, Maddigan teaches associating the selected tool to query data objects with at least one of the example input data objects in the dataset corresponding to the natural language request, the instructions to use the dataset and references to particular aspects of the data in the query to plot the Outcome and the instructions to limit the data to the dataset given with the natural language query (Pages 4-5 Section 3.2).
Wherein the selected tool includes at least one of: query objects, apply actions, or ontology functions.
Maddigan teaches indicating data object types specified by selecting a tool to query objects based on the natural language request stating to “plot the outcome” and the dataset including named columns such as “outcome” (Page 4 Section 3.2)
Regarding Claim 11:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the first data set includes data objects of a first data object type of the one or more data object types.
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome” which is also a natural language word associated with the data set.
Regarding Claim 12:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the first data set is selected by the LLM based at least in part on the first data set including data objects of the first data object type.
Maddigan teaches “a user enters a request in the form of natural language text in reference to a selected dataset. Chat2VIS engineers the prompt, submits it to the chosen LLMs, formats the returned script, and renders the visualisation.” (Page. 4 Section 3.2) thereby teaching the LLM selecting the alredy “selected dataset” based in part on the data set being compatible including data objects of the first data object type included in the user’s request/prompt.
Regarding Claim 13:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein the unique identifiers comprise respective one or more natural language words.
Maddigan teaches receiving a reference to “use a dataframe called df from data_file.csv “ (Fig. 4(b)) thereby teaching the unique identifier comprising natural language words of at least “data” and “file”.
Regarding Claim 14:
Devaux, Maschmeyer, and Maddigan further teach:
Wherein each data object type is associated with at least one data set,
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome”.
Wherein each data set comprises data objects, and
Maddigan teaches the dataset in Fig. 4(a) having data objects of the data object type “outcome”.
Wherein the data objects associated with the one or more properties of a subset of the data object types are included in at least one of the data sets.
Maddigan teaches that the data objects of column “Outcome” are associated with one or more properties of a subset of the data object type “Outcome”, the properties being one of the categorical values: Mismatch, Match, and Error. (Maddigan - Fig. 4(b): “the column ‘Outcome’ has categorical values ‘Mismatch’, ‘Match’, ‘Error’”)
Regarding Claim 15:
Devaux, Maschmeyer, and Maddigan further teach:
One or more computer-readable storage mediums having program instructions embodied therewith; and
Maddigan teaches “Early NL2VIS systems were built on symbolic-based NLP approaches, relying on heuristic algorithms [6], rule-based architectures, and probabilistic grammer-based methods for translating NL queries. Although each technique displayed increasing accuracy, they required more computational resources.” (Page 2 Section 2) thereby teaching computational resources such as storage mediums with instructions embodied therewith and processors configured to execute the instructions.
One or more processors configured to execute the program instructions to cause the system to perform the computer-implemented method.
Maddigan teaches “Early NL2VIS systems were built on symbolic-based NLP approaches, relying on heuristic algorithms [6], rule-based architectures, and probabilistic grammer-based methods for translating NL queries. Although each technique displayed increasing accuracy, they required more computational resources.” (Page 2 Section 2) thereby teaching computational resources such as storage mediums with instructions embodied therewith and processors configured to execute the instructions.
Regarding Claim 16:
Devaux, Maschmeyer, and Maddigan further teach:
A computer program product comprising one or more computer-readable storage mediums having program instructions embodied therewith, the program instructions executable by one or more processors to cause the one or more processors to perform the computer-implemented method.
Maddigan teaches “Early NL2VIS systems were built on symbolic-based NLP approaches, relying on heuristic algorithms [6], rule-based architectures, and probabilistic grammer-based methods for translating NL queries. Although each technique displayed increasing accuracy, they required more computational resources.” (Page 2 Section 2) thereby teaching computational resources such as storage mediums with instructions embodied therewith and processors configured to execute the instructions.
Response to Amendment
Applicant’s Amendments, filed on 3/19/2026, are acknowledged and accepted.
Response to Arguments
In light of the Amendments and Remarks filed on 3/19/2026 and further review of the Application’s specification, the 101 rejection of claims 1-9 and 11-16 for being directed to an abstract idea without significantly more has been withdrawn. In particular, Applicant convincingly argues on Pages 9-13 of the Remarks that “the claims recite a practical application, and thus are not ‘directed to’ an abstract idea” (Remarks Page 9) because the additional elements “describe a specific solution to problem related to using an LLM to process a query on a large dataset. For example, Applicant’s specification at [0009] describes a problem with existing techniques for interacting with datasets when using an LLM…[and] further describes how the claims can address technological problems of interacting with a dataset when using an LLM, for example at paragraph [0010]-[0011]” (Remarks Pages 11-12) and that the practical application is “improving the ability of a user to interact with a dataset when using an LLM, for example by enabling the LLM to generate a relevant query without providing an entire dataset as input to the LLM as input, and allowing the LLM to adapt to different dataset formats or schemas.” (Remarks Page 13).
On page 15 of the Remarks filed on 3/19/2026, Applicant argues that “Maschmeyer's description of a prompt generator selecting attributes does not teach or suggest at least "a user selection of one or more tools, wherein the selection of one or more tools identifies a set of instructions for a large language model," as recited in Applicant's amended Claim 1. Further, Applicant submits that Maddigan, alone or in combination with Maschmeyer, do not teach or suggest at least "based on the prompt, receiving, from the LLM, a request for information about the one or more data object types," and "providing, using the selected one or more tools and based on the request, the information about the one or more data object types using the selected one or more tools," as recited in the amended Claim 1.”Applicant’s argument is convincing that neither Maschmeyer nor Maddigan teach the newly added limitations related to receiving a request for information and providing the information to the LLM. Therefore, a new grounds of rejection was necessitated by the amendments and is presented above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Non-Patent Literature Dang et al., "How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models", September 2022, arXiv:2209.01390 teaches prompting a generative model is a promising recent development that in principle enables end-users to creatively leverage zero-shot and few-shot learning to assign new tasks to an AI adhoc, simply by writing them down. However, for the majority of end-users writing effective prompts is currently largely a trial and error process. To address this, we discuss the key opportunities and challenges for interactive creative applications that use prompting as a new paradigm for Human-AI interaction.
Non-Patent Literature Sun et al., "SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL", May 26, 2023, arXiv:2306.00739 teaches prompting a generative model is a promising recent development that in principle enables end-users to creatively leverage zero-shot and few-shot learning to assign new tasks to an AI adhoc, simply by writing them down. However, for the majority of end-users writing effective prompts is currently largely a trial and error process. To address this, we discuss the key opportunities and challenges for interactive creative applications that use prompting as a new paradigm for Human-AI interaction.
Dong et al. (U.S. Pre-Grant Publication No. 2024/0311579) teaches techniques that may generate prompts for language models. The techniques include obtaining a first dataset and a second dataset and training a hierarchical virtual token generator (VTG) model to generate a large language model (LLM) input prompt. Training the hierarchical VTG includes training, based on the first dataset, a first VTG to output a first virtual token and training, based on the second dataset, a second VTG to output a second virtual token embedding. The generated LLM input prompt includes the first virtual token embedding and the second virtual token embedding.
Cheng et al. (U.S. Pre-Grant Publication No. 2020/0395007) teaches an action agent architecture in a scalable multi-service virtual assistant platform that can construct a fluid and dynamic dialogue by assembling responses to end user utterances from two kinds of agents, information agents and action agents. The virtual assistant platform comprises a plurality of action agents to perform two or more actions in response to parsed user input, contextual data, and/or an information value obtained from an information agent. The plurality of action agents are interrelated via at least one follow-up connection which interrelates any two of the action agents such that a second action agent follows-up with a second action after a first action agent completes a first action based on a trigger condition. The second action agent utilizes at least one information value set by the first action agent or a system state change triggered by the first action agent.
Procter et al. (U.S. Pre-Grant Publication No. 2024/0354320) teaches a thought object selection server receives a plurality of thought objects and the thought objects include text present in qualitative responses from plurality of user devices in a communication environments. The plurality of thought objects consists of M thought objects that are most recently seen thought objects and N thought objects that are least seen by the plurality of user devices. A prompt is provided to the LLM with the M thought objects, the N thought objects, and a request to identify diverse thought objects. LLM compares the M thought objects and the N thought objects to identify dissimilar one or more dissimilar thought objects from the N thought objects based on semantic distances. One thought object is selected from the dissimilar thought objects as a diverse thought object. In some cases, non-transformer based embedding tools may be used to identify diverse thought in the received thought objects.
Schaefer et al. (U.S. Pre-Grant Publication No. 2024/0311093) teaches an elidable text is constructed that prioritizes the content included in a prompt to a large language model having a fixed-size context window. The elidable text is generated from developer-generated instructions or automatically for source code within a source code editor. A source code editor may include a feature that selects certain lines of code as important or focused which are assigned a high-priority value. A changed line, a line of source code at a current cursor position, lines of source code at the beginning of a file and those that output data are considered focused lines. Non-focused lines are assigned a priority based on a distance from a focused line. The elidable text constrains the data included in a prompt to the context window size by replacing the lowest-valued lines of text and source code with a replacement string.The reference further teaches “the developer may select one or several source code statements as a focused line or the elidable text generator may select a line in the current cursor position as the focused line.” (Para. [0033]) and a method comprising “accessing a large language model to perform a task given a prompt that fits within a context window size; monitoring a source code program in a software development tool; selecting a focus line from the source code program; associating a high-priority value to the focus line; associating a priority value to a non-focused line based on a distance from the focus line; constructing a prompt to the large language model from select lines of the source code program; replacing the select lines of the prompt with a replacement string based on a corresponding priority value when size of the prompt exceeds size of the context window size; and applying the prompt to the large language model for performance of the task.” (Para. [0061]).
Bennet et al. (U.S. Pre-Grant Publication No. 2025/0005655) teaches “follow up information manually provided by the user in accordance with a type-2 prompt in which the LLM makes at least one additional information request from a user. For example, a user may provide “customer response data” when responding to the at least one additional LLM prompt “What is your question about product X?.”” (Para. [0031] and “The term “Type-2 clickable prompt button”, as referred to herein, may refer to an embodiment in which a large language model (LLM) makes at least one additional information request from a user to obtain further context information from the user.” (Para. [0033]).
Wu et al. (U.S. Pre-Grant Publication No. 2024/0419902) teaches “an LLM (or other language model type(s)) may retrieve and/or access map data or other information determined to be necessary to generate an output using one or more application programming interfaces (APIs) and/or plug-ins (e.g., third-party plug-ins). For example, in order to retrieve additional contextual information, additional map information, additional feature information, and/or other information not directly included in a prompt to the model, the system—using the LLM, in some embodiments—may generate one or more prompts or queries for one or more data sources (e.g., open street maps (OSM), wolfram alpha, a local map database, etc.), via one or more APIs or plug-ins, in order to obtain the additional information required (or deemed necessary) for responding to the initial query or prompt. Such an approach to querying additional resources may be recursive, in at least some embodiments, in that the system may continue to access one or more data sources via the API(s) and/or plug-ins until it is determined the necessary information has been obtained, or until no additional information is available” (Para. [0145]).
Siracusano et al. (U.S. Pre-Grant Publication No. 2024/0411994) teaches “The data acquisition module 308 can decide whether the information is enough for the LLM 312 to generate a response (e.g., an accurate or helpful response), or if the data acquisition module 308 is required to search for additional information” (Para. [0067]).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBERT F MAY whose telephone number is (571)272-3195. The examiner can normally be reached Monday-Friday 9:30am to 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached on 571-270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BORIS GORNEY/Supervisory Patent Examiner, Art Unit 2154
/ROBERT F MAY/Examiner, Art Unit 2154 5/27/2026
Prosecution Insights