DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims are not limited to tangible embodiments. In view of Applicant’s disclosure, specification paragraph [0205]-[0208], the “computer-readable media” is not limited to tangible embodiments, instead being defined as including both tangible embodiments (e.g., “storage media”) and intangible embodiments (e.g., “communication media”). As such, the claims are not limited to statutory subject matter and are therefore non-statutory.
Examiner suggests amending claim 8 to recite “non-transitory computer-readable storage media” instead of “computer-readable media”, which encompasses both tangible (i.e. storage media) and non-tangible (i.e. communication media (e.g. carrier waves)).
Allowable Subject Matter
Claims 1-7 and 15-20 allowed.
Claims 9-14 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding independent claims 1 and 15:
Prior art relied upon:
MUTHU et al (US 20250209372 A1)
The above prior art fails to teach:
“identifying, based on the evaluating, a subset of rejected examples from the set of training examples, wherein the subset of rejected examples include logical form queries that fail one or more tests; grouping the subset of rejected examples into at least one or more groups of rejected examples based on similarity between the subset of rejected examples; generating, by the generative model, instructions for each group of the one or more groups of rejected examples based on a pattern of error identified for each group of the one or more groups of rejected examples; and updating the baseline prompt to include the instructions for each group of the one or more groups of rejected examples”
MUTHU lacks any discussion of identifying, based on the evaluating, a subset of rejected examples from the set of training examples, wherein the subset of rejected examples include logical form queries that fail one or more tests; grouping the subset of rejected examples into at least one or more groups of rejected examples based on similarity between the subset of rejected examples; generating, by the generative model, instructions for each group of the one or more groups of rejected examples based on a pattern of error identified for each group of the one or more groups of rejected examples; and updating the baseline prompt to include the instructions for each group of the one or more groups of rejected examples. Instead, MUTHU discloses generating optimized machine learning model prompts. Embodiments include providing an input prompt to a child machine learning model that directs the child machine learning model to generate an output. Embodiments further include generating a parent model prompt comprising instructions to generate a score for the input prompt based on one or more scoring criteria, the input prompt, and the output of the child machine learning model. Embodiments further include providing the parent model prompt to a parent machine learning model. Embodiments further include generating, by a generative machine learning model, an optimized prompt for the child machine learning model based on the generated score for the input prompt. (See Abstract).
These features, together with the other limitations of the independent claims are novel and non-obvious over the prior art of record. The dependent claims 2-7 and 16-20 as being definite, enabling by the specification, and further limiting to the independent claim, would also be allowable.
Regarding independent claim 8:
Prior art relied upon:
MUTHU et al (US 20250209372 A1)
The above prior art fails to teach:
“identifying, based on the evaluating, a subset of rejected examples from the set of training examples, wherein the subset of rejected examples include logical form queries that fail one or more tests; v) grouping the subset of rejected examples into at least one or more groups of rejected examples based on similarity between the subset of rejected examples; vi) generating, by a generative model, instructions for each group of the one or more groups of rejected examples based on a pattern of error identified for each group of the one or more groups of rejected examples; vii) modifying the baseline prompt to include the instructions for each group of the one or more groups of rejected examples; and performing ii)-vii) with the updated baseline prompt until a performance threshold is exceeded”
MUTHU lacks any discussion of identifying, based on the evaluating, a subset of rejected examples from the set of training examples, wherein the subset of rejected examples include logical form queries that fail one or more tests; grouping the subset of rejected examples into at least one or more groups of rejected examples based on similarity between the subset of rejected examples; generating, by the generative model, instructions for each group of the one or more groups of rejected examples based on a pattern of error identified for each group of the one or more groups of rejected examples; and updating the baseline prompt to include the instructions for each group of the one or more groups of rejected examples. Instead, MUTHU discloses generating optimized machine learning model prompts. Embodiments include providing an input prompt to a child machine learning model that directs the child machine learning model to generate an output. Embodiments further include generating a parent model prompt comprising instructions to generate a score for the input prompt based on one or more scoring criteria, the input prompt, and the output of the child machine learning model. Embodiments further include providing the parent model prompt to a parent machine learning model. Embodiments further include generating, by a generative machine learning model, an optimized prompt for the child machine learning model based on the generated score for the input prompt. (See Abstract).
These features, together with the other limitations of the independent claims are novel and non-obvious over the prior art of record. The dependent claims 9-14 as being definite, enabling by the specification, and further limiting to the independent claim, would also be allowable.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Sairam Gurajada et al (“Effectiveness of Prompt Optimization in NL2SQL Systems”) - effective SQL generation, all an LLM needs is a static set of exemplars that capture the intricacies of the domain—offering performance comparable to retrieval-based approaches, while eliminating the need for inference-time retrieval. The key challenge lies in identifying this representative set of exemplars. To tackle this, we leverage prompt optimization techniques for exemplar selection in NL2SQL and demonstrate their effectiveness. The Proposer agent takes a Proposer prompt as input and generates an NL2SQL prompt comprising an instruction and a set of exemplars. The SQL Generator agent then evaluates the generated prompt on a validation set (sampled iteratively from the training data) and collects performance metrics including accuracy, as well as the correct and incorrect examples. This feedback is used to update the Proposer prompt. In subsequent iterations, the Proposer is guided to refine the NL2SQL prompt based on past performance, aiming to produce more informative exemplars and a better-suited instruction for improved SQL generation.
Arpit Bajgoti et al (“ASKSQL: Enabling cost-effective natural language to SQL conversion for enhanced analytics and search”) - Natural Language to SQL (NL2SQL) for database query and search has been a significant research focus in recent years. However, existing methods have predominantly concentrated on SQL query generation, overlooking critical aspects such as enterprise cost, latency, and the overall analytical search experience. This paper presents an end-to-end NL2SQL pipeline named ASKSQL that integrates optimized and adaptable query recommendation, entity-swapping module, and skeleton-based caching to enhance the search experience. The pipeline also incorporates an intelligent schema selector for efficiently handling large schema entity selection and a fast and scalable adapter-based query generator. The proposed pipeline emphasizes minimizing Large Language Model (LLM) costs by finding search patterns in previously requested or generated queries. The pipeline can also be tuned to adapt to trends and common patterns observed from the daily search analytics. Experimental results demonstrate an average increase in accuracy by 5.83% and an overall decrease in latency by 32.6% as the usage count of this search pipeline increases highlighting its effectiveness in improving the NL2SQL search experience.
Xiaohu Zhu et al (“Large Language Model Enhanced Text-to-SQL Generation: A Survey”) - Text-to-SQL translates natural language queries into Structured Query Language (SQL) commands, enabling users to interact with databases using natural language. Essentially, the text-to-SQL task is a text generation task and its development primarily dependent on changes in language models. Especially with the rapid development of Large Language Models (LLMs), the pattern of text-to-SQL has undergone significant changes. Existing survey work mainly focuses on rule-based and neural-based approaches, still lacking a survey of Text-to-SQL with LLMs. In this paper, we survey the large language model enhanced text-to-SQL generations, classifying them into prompt engineering, fine-tuning, pre-trained and Agent groups according to training strategies. And we also summarize datasets and evaluation metrics comprehensively. This survey could help people better understand the pattern, research status, and challenges of LLM-based text-to-SQL generations.
El-Haddad et al (US 20260140944 A1) - A computing system is configured to: (i) receive, from a client device, a natural-language query that is related to a construction project managed via a construction management software platform, (ii) generate a prompt for a generative AI model that comprises a request to transform the natural-language query into a structured query of a relational data store that is related to the construction project, (iii) pass the prompt to the generative AI model, (iv) receive, from the generative AI model, a response that comprises the structured query that is generated by the generative AI model, (v) perform a search in accordance with the structured query and thereby return search results comprising a dataset that is returned based on the structured query, (vi) based on the search results, generate a response to the natural-language query, and (vii) present, using the client device, the response to the user.
Yakovlev et al (US 20250284688 A1) - A database system generates a prompt for an LLM or other machine learning (ML) model to narrow the search space to highly relevant information about a database. A distinct instance of a classifier, a clustering algorithm, or a topic modeling model can be trained based on information from ML automation within the database system, respectively for each column or table in the database. Model instances can then be used during generative LLM inferencing to identify relevant sources of data to answer the user's question. Thus, the prompt generation combines ML automation and other ML models or an LLM for topic modeling and schema description.
Popescu et al (US 12393584 B2) - A set of values can be selected from a plurality of fields of a table in a database. At least one adverb or adjective can be selected for the set of values. Join paths for values in the set of values can be determined. A structured query language pattern can be determined based, at least in part, on at least one value in the set of values, the at least one adverb or adjective for the set of values, and the join paths for the set of values. The structured query language pattern can be stored to first training data configured, at least in part, for use in machine learning to train a text-to-SQL model, the text-to-SQL model comprising a first artificial neural network and configured to convert first natural language text to a first structured query language query.
Socher et al (US 20240020538 A1) - Embodiments described herein provide systems and methods for a customized generative AI platform that provides users with a tool to generate various formats of responses to user inputs that incorporate results from searches performed by the generative AI platform. The system may use a neural network to utilize input data and contextual information to identify potential search queries, gather relevant data, sort information, generate text-based responses to user inputs, and present response and search results via user-engageable elements.
KIM et al (US 20230169074 A1) - Provided are a device and method for converting a natural language query into a structured query language (SQL) query for a database search. The method includes an operation A of labeling natural language queries included in training data, an operation B of converting the natural language queries into second SQL queries by applying the natural language queries to an SQL conversion model, an operation C of verifying the second SQL queries, and an operation D of training the SQL conversion model by comparing the second SQL queries with third SQL queries corresponding to the natural language queries of the training data.
SHAHRIAR et al (US 20230161763 A1) - Systems and methods for determining a query for a data store are described. A natural language text may be analyzed using heuristic processing and one or more machine learning models. Query parameters may be determined from the heuristic processing and machine learning and combined to form a query in a query language. In the heuristic processing, parsing rules may be used to remove conditional terms to generate a base question. The base question may be input to the one or more machine learning model to generate a base query which may be combined with query parameters related to the conditional terms.
MUTHU et al (US 20250209372 A1) - Aspects of the present disclosure relate to generating optimized machine learning model prompts. Embodiments include providing an input prompt to a child machine learning model that directs the child machine learning model to generate an output. Embodiments further include generating a parent model prompt comprising instructions to generate a score for the input prompt based on one or more scoring criteria, the input prompt, and the output of the child machine learning model. Embodiments further include providing the parent model prompt to a parent machine learning model. Embodiments further include generating, by a generative machine learning model, an optimized prompt for the child machine learning model based on the generated score for the input prompt.
GREENE et al (US 20260119895 A1) - The present disclosure relates generally to systems and methods for updating an input prompt for a generative AI model (e.g., an LLM) based on feedback that is provided in connection with an output from the generative AI model that is unsatisfactory. For example, where a user indicated that an output from the generative AI model is incorrect, inaccurate, or is an otherwise unsatisfactory response to an input prompt, this disclosure describes models to facilitate generation of feedback hints and/or additional information that can be included within an updated prompt that, when provided as an input to the generative AI model, has an improved likelihood to return an output that is in-line with user expectations. Indeed, features of the systems and methods described herein provide a framework for improving outputs of generative AI models that are more accurate or otherwise responsive to the input prompts.
Rajmohan et al (US 12645723 B1) - Techniques for generating optimized input prompts for generative language models combines automated prompt refinement with intelligent context retrieval. A prompt optimization service automatically refines initial prompt instructions through controlled mutation and evaluation cycles using multiple generative language models. A context management service maintains a vector database of historical examples pre-encoded as embedding vectors, enabling efficient retrieval of semantically similar examples for new tasks. A dynamic prompt generator assembles optimized prompts by combining refined instructions with relevant historical examples and task context. The system supports deployment with both large language models and smaller fine-tuned models, enabling efficient operation across diverse applications. Configurable parameters allow balancing optimization quality against computational costs. The system maintains efficiency through single-call mutations, fast vector similarity search, and support for compact fine-tuned models that reduce resource requirements while preserving response quality.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JARED M BIBBEE whose telephone number is (571)270-1054. The examiner can normally be reached Monday-Thursday 8AM-6PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, APU MOFIZ can be reached at 5712724080. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JARED M BIBBEE/Primary Examiner, Art Unit 2161
Prosecution Insights