DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities: Paragraph 0052 Line 2 recites "SQL LLM 18". The associated figure lists SQL LLM as part 138.
Appropriate correction is required.
Examiner’s Notes
Claims of co-pending applications 18/799,168, 18/799,411, 18/799,235 and 18/799,506 are similar to the current claims, but are non-obvious at this time. However, if the claims become obvious due to amendments, a double patenting rejection will be reconsidered.
While claim elements do recite abstract ideas(receiving questions, generating prompts in natural language, identification of relevant tables), the claims when viewed as a whole are directed to an improvement to the functioning of a technology. Specifically, the claims are directed an improvement to the text-to SQL workflow. In light of Ex Parte Desjardins, a 35 USC § 101 rejection will not be made at this time. However, if future amendments alter the scope of the claims, a 35 USC § 101 rejection will be reconsidered.
The prior art rejections below may be overcome if Claim 1 is appended with limitations pertaining to paragraphs 0015 and 0016 from the specification. Separately they are obvious in view of prior art but together they may become non-obvious in view of the current state of the art.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1,7,8,11,17,20 are rejected under 35 U.S.C. 103 as being unpatentable over Rai(US Patent 12373425) in view of Wu(A Multi-Source Retrieval Question Answering Framework Based on RAG).
Regarding Claim 1, Rai discloses a computing system for processing a natural language question, the computing system comprising:
a memory, a communication interface, and a processor operatively coupled to the memory and the communication interface(computer system 1200 includes a processing unit 1204 that communicates with a number of peripheral subsystems via a bus subsystem 1202. These peripheral subsystems may include a processing acceleration unit 1206, an I/O subsystem 1208, a storage subsystem 1218 and a communications subsystem 1224. Storage subsystem 1218 includes tangible computer-readable storage media 1222 and a system memory 1210.[Col 33 Line 27-34]);
a retrieval system and a structured query language (SQL) large language model (LLM) stored in the memory and executable by the processor(The software may be stored on a non-transitory storage medium, The processing depicted in FIG. 7 may be implemented in software (e.g., code, instructions, program) executed by one or more processing units[Col 21 Line 54-57]);
the processor configured to:
receive the natural language question(a natural language query for retrieving features from a feature store is received from a user[Col 22 Line 4-5]);
generate a prompt that comprises the natural language question and a database schema corresponding to a database(At block 710, an input prompt is generated by appending text identifying the feature store to the natural language query[Col 22 Line 6-8], generator module comprises an algorithm or series of functional steps that are executed by the LLM[Col 22 Line 23-25], executing the algorithm or series of functional steps comprises: (i) executing a first function[Col 22 Line 29-31],In some instances, the first function also gathers information about the databases, including the databases' structure, the metadata, and schemas[Col 22 Line 47-50]);
process, using the retrieval system, the prompt to identify one or more tables in the database, the one or more tables relevant to the natural language question(At block 715, one or more tables or databases are determined, by LLM, from the feature store that are relevant to the natural language query based on the input prompt. At block 720, metadata for the one or more tables or databases is retrieved, by the LLM, from the feature store[Col 22 Line 14-19]);
a prompt that comprises the natural language question, the database schema, and one or more identities of the one or more tables(the programming language query is generated based on the input prompt, the metadata and the schemas for the databases, the metadata and schema for the tables, and the one or more feature groups[Col 22 Line 54-58], programming language queries (e.g., SQL) and executing the system language queries on a feature store to acquire search results for the natural language queries according to various embodiments[Col 11 Line 57-60]);
generate, using the SQL LLM, a set of SQL code based on the prompt(This is the prompt generated by the retrieval system)( the programming language query is generated based on the input prompt, the metadata and the schemas for the databases, the metadata and schema for the tables, and the one or more feature groups[Col 22 Line 54-58], programming language queries (e.g., SQL) and executing the system language queries on a feature store to acquire search results for the natural language queries according to various embodiments[Col 11 Line 57-60]);
initiate executing the set of SQL code on the database and receiving a result(programming language queries (e.g., SQL) and executing the system language queries on a feature store to acquire search results for the natural language queries according to various embodiments[Col 11 Line 57-60]); and
provide the result message responsive to the natural language question(At block 740, the list of features is output to the user[Col 22 Line 63]).
Rai does not teach a retrieval system that generates an augmented prompt nor does Rai teach obtaining, using the retrieval system and the prompt, one or more identities of one or more additional data sources and associated additional metadata; retrieve, using the retrieval system, an additional result from the one or more additional data sources; generate a result message using the result and the additional result.
However, Wu teaches a retrieval system that generates an augmented prompt(multi-source retrieval framework, named MSRAG[Abstract], Information-Web&Original Question[Figure 1], Information-GPT&Original Question[Figure 1]) and obtaining, using the retrieval system and the prompt(Original question [Figure 1]), one or more identities of one or more additional data sources(Initially, within the Web retrieval module, we utilize the Web search engine to perform searches for questions in order to obtain real-time information[II. System Model], GPT-3.5, leveraging its robust semantic understanding capabilities and vast knowledge repository of linguistic contexts to generate search information pertinent to the given queries[II. System Model]) and associated additional metadata(retrieval of metadata is taught by Rai, metadata for the one or more tables or databases is retrieved, by the LLM[Col 22 Line 14-19]); retrieve, using the retrieval system, an additional result from the one or more additional data sources; generate a result message using the result and the additional result(answers generated by all three components, a loss function calculation is performed to select the answer with the lowest loss value as the optimal answer[II. System Model]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the idea of a retrieval system utilizing multiple data sources with the system disclosed in Rai because it would attenuate information noise in retrieval and enhance the relevance of retrieved information(Wu).
Claim 11 is directed to a method claim with similar limitations to that of Claim 1 and is rejected under the same rationale.
Claim 20 is directed to computer readable memory claim with similar limitations to that of Claim 1 and is rejected under the same rationale.
Regarding Claim 7, Rai modified by Wu teaches the retrieval system comprises a retrieval LLM, and the retrieval LLM generates the augmented prompt(the utilization of GPT- 3.5 to replace conventional retrievers, generating the required retrieval information from its extensive corpus knowledge base, thus further improving the effectiveness of retrieval information[Wu, II. System Model], Wu Figure 1).
Regarding Claim 8, Rai modified by Wu teaches a preliminary LLM in the memory, and the preliminary LLM generates the prompt(At block 710, an input prompt is generated by appending text identifying the feature store to the natural language query[Col 22 Line 6-8], generator module comprises an algorithm or series of functional steps that are executed by the LLM[Col 22 Line 23-25], executing the algorithm or series of functional steps comprises: (i) executing a first function[Col 22 Line 29-31],In some instances, the first function also gathers information about the databases, including the databases' structure, the metadata, and schemas[Col 22 Line 47-50]) that comprises the natural language question, the database schema and metadata of the database(At block 730, a programming language query is generated, by the LLM, based on the input prompt, the metadata, and the one or more feature groups. In some instances, the programming language query is generated based on the input prompt, the metadata and the schemas for the databases, the metadata and schema for the tables, and the one or more feature groups. [Rai Col 22 Line 52-58]); wherein the retrieval system comprises a retrieval LLM(the utilization of GPT- 3.5 to replace conventional retrievers, generating the required retrieval information from its extensive corpus knowledge base, thus further improving the effectiveness of retrieval information[Wu, II. System Model], Wu Figure 1), and the retrieval LLM processes the prompt to identify the one or more tables in the database(In some instances, the generator module comprises an algorithm or series of functional steps that are executed by the LLM to determine the one or more tables or databases from the feature store[Rai Col 22 Line 23-26]) and a subset of the metadata that corresponds to the one or more tables; and wherein the retrieval LLM generates the augmented prompt(the utilization of GPT- 3.5 to replace conventional retrievers, generating the required retrieval information from its extensive corpus knowledge base, thus further improving the effectiveness of retrieval information[Wu, II. System Model], Wu Figure 1) that further comprises the metadata and the subset of the metadata(query is generated based on the input prompt, the metadata and the schemas for the databases, the metadata and schema for the tables, and the one or more feature groups[Rai Col 22 Line 54-58]).
Claim 17 is directed to a method claim with similar limitations to that of Claim 8 and is rejected under the same rationale.
Claim(s) 2,12 are rejected under 35 U.S.C. 103 as being unpatentable over Rai(US Patent 12373425) in view of Wu(A Multi-Source Retrieval Question Answering Framework Based on RAG) as applied to claim 1 above, and further in view of Wang(Searching for Best Practices in Retrieval-Augmented Generation).
Regarding Claim 2, neither Rai nor Wu teach generating the result message comprises the processor comparing the result and the additional result to identify one or more redundancies, and combining the result and the additional result to generate the result message without the one or more redundancies.
However, Wang teaches generating the result message comprises the processor comparing the result(Retrieval results may contain redundant or unnecessary information, potentially preventing LLMs from generating accurate responses[3.7 Summarization]) and the additional result to identify one or more redundancies(Selective Context enhances LLM efficiency by identifying and removing
redundant information[3.7 Summarization]), and combining the result and the additional result to generate the result message without the one or more redundancies.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the idea of removing redundant information from results with the system disclosed in Rai modified by Wu because it would enhance generator output(Wang 2.1 Query and Retrieval Information).
Claim 12 is directed to a method claim with similar limitations to that of Claim 2 and is rejected under the same rationale.
Claim(s) 3,13 are rejected under 35 U.S.C. 103 as being unpatentable over Rai(US Patent 12373425) in view of Wu(A Multi-Source Retrieval Question Answering Framework Based on RAG) as applied to claim 1 above, and further in view of Zhuang(Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking).
Regarding Claim 3, Rai modified by Wu teaches that the SQL LLM generates the result message(programming language queries (e.g., SQL) and executing the system language queries on a feature store to acquire search results for the natural language queries according to various embodiments[Rai Col 11 Line 57-60], At block 740, the list of features is output to the user[Rai Col 22 Line 63]). Rai modified by Wu also teaches the use of reranking in RAG for use in LLM generation(Subsequently, Michael et al. proposed Re2G[3], which integrates neural initial retrieval and re-ranking into RAG, thereby enhancing information relevance[Wu 1. Introduction])
Rai modified by Wu does not teach generating a first importance weighting assigned to the result and a second importance weighting assigned to the additional result, the first importance weighting different from the second importance weighting; and inputting the result, the first importance weighting, the additional result, and the second importance weighting into the LLM.
However, Zhuang teaches generating a first importance(Given a query q and a list of candidate documents d = (d1,...,dm), an LLM ranker based on relevance generation takes each query-document pair (q,di) as input and prompts the LLM to answer whether the document is relevant to the query[3. LLM Rankers]) weighting assigned to the result and a second importance weighting assigned to the additional result(each query-document pair (q,di) as input and prompts the LLM to answer whether the document is relevant to the query[3. LLM Rankers]), the first importance weighting different from the second importance weighting; and inputting the result, the first importance weighting, the additional result, and the second importance weighting into the LLM(intermediate relevance labels in the prompt serve as a “cue” to the LLM to distinguish partially relevant documents from fully relevant or fully irrelevant ones[1. Introduction]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the idea of importance weighting with the system disclosed in Rai modified by Wu because it would allow for the LLM to distinguish partially relevant documents from fully relevant or fully irrelevant ones(Zhuang).
Claim 13 is directed to a method claim with similar limitations to that of Claim 3 and is rejected under the same rationale.
Claim(s) 4,5,6,14,15,16 are rejected under 35 U.S.C. 103 as being unpatentable over Rai(US Patent 12373425) in view of Wu(A Multi-Source Retrieval Question Answering Framework Based on RAG) as applied to claim 1 above, and further in view of Singh(US PGPub 20250225169).
Regarding Claim 4, while Wu teaches calculating a similarity score, it does not teach the processor is configured to further: execute a comparison of the result and the additional result to determine a similarity score; and when the comparison indicates that the result and the additional result do not match, then generate the result message that indicates that the result and the additional result are different.
However, Singh teaches the processor is configured to further: execute a comparison of the result and the additional result to determine a similarity score(The processor further reads the set of instructions to generate, using a trained classification model, a similarity score between the first data entity and the second data entity that accounts for feature interactions between the first data entity and the second data entity[0017]); and when the comparison indicates that the result and the additional result do not match, then generate the result message that indicates that the result and the additional result are different(In some embodiments, if the similarity score is lower than the predetermined threshold, an indication is generated that the first data entity does not matches the second data entity.[0057]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the idea of using a similarity score to affect the output with the system disclosed in Rai modified by Wu because it would allow identification of similar entities that differ because of small variation in an attribute(Singh Abstract) .
Claim 14 is directed to a method claim with similar limitations to that of Claim 4 and is rejected under the same rationale.
Regarding Claim 5, Singh teaches the result message comprises information regarding one or more differences between the result and the additional result(In some embodiments, the similarity score is output, in part, by a logits function that provides a score indicating a likelihood that the two data entities are matched[0056]).
Claim 15 is directed to a method claim with similar limitations to that of Claim 5 and is rejected under the same rationale
Regarding Claim 6, Singh teaches the processor is configured to further: execute a comparison of the result and the additional result to determine a similarity score(The processor further reads the set of instructions to generate, using a trained classification model, a similarity score between the first data entity and the second data entity that accounts for feature interactions between the first data entity and the second data entity[0017]); and when the comparison indicates that the result and the additional result match, then generate the result message that indicates that the result matches the additional result(in accordance with a determination that the similarity score is greater than a predetermined threshold, generate an indication that the first data entity matches the second data entity[0017]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the idea of using a similarity score to affect the output with the system disclosed in Rai modified by Wu because it would allow identification of similar entities that differ because of small variation in an attribute(Singh Abstract) .
Claim 16 is directed to a method claim with similar limitations to that of Claim 6 and is rejected under the same rationale.
Claim(s) 9,18 are rejected under 35 U.S.C. 103 as being unpatentable over Rai(US Patent 12373425) in view of Wu(A Multi-Source Retrieval Question Answering Framework Based on RAG) as applied to claim 1 above, and further in view of Sanjeeb(Build a robust text-to-SQL solution generating complex queries, self-correcting, and querying diverse data sources).
Regarding Claim 9, Rai nor Wu teaches that when the result comprises an error message, the processor is configured to: generate a new set of SQL code, using the SQL LLM, based on the augmented prompt; initiate executing the new set of SQL code on the database; and receive a new result comprising retrieved data from the database that is responsive to the new set of SQL code
However, Sanjeeb teaches when the result comprises an error message(If Athena provides an error message that mentions the syntax is incorrect, the model uses the error text from Athena’s response [Solution overview Step 7]), the processor is configured to: generate a new set of SQL code, using the SQL LLM, based on the augmented prompt; initiate executing the new set of SQL code on the database(The model creates the corrected SQL and continues the process. This iteration can be performed multiple times[Solution overview Step 9]); and receive a new result comprising retrieved data from the database that is responsive to the new set of SQL code(Finally, we run the SQL using Athena and generate output. Here, the output is presented to the user[Solution overview Step 10]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the error handling of Sanjeeb with the system disclosed in Rai modified by Wu because it would allow for more accurate and effective corrections in the generated SQL(Sanjeeb).
Claim 18 is directed to a method claim with similar limitations to that of Claim 9 and is rejected under the same rationale.
Claim(s) 10,19 are rejected under 35 U.S.C. 103 as being unpatentable over Rai(US Patent 12373425) in view of Wu(A Multi-Source Retrieval Question Answering Framework Based on RAG) as applied to claim 1 above, and further in view of Chakraborty(US Patent 12/306,828 cited in IDS).
Regarding Claim 10, Rai nor Wu disclose the processor is further configured to: receive the natural language question via the chat user interface, generate the result message in a form of a natural language response that comprises the result, and provide the natural language response via the chat user interface.
However, Chakraborty does teach the processor is further configured to: receive the natural language question via the chat user interface(application UI 602 may receive a user question in the form of natural language[Col 16 Line 57]), generate the result message in a form of a natural language response that comprises the result(The response may be provided in natural language and/or conversational form, such as using a conversational AI and/or the LLM to converse back with the user in the natural language format.[Col 18 Line 55]), and provide the natural language response via the chat user interface(Thereafter, a query response 114 may be output in application[Col 7 Line 37]).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the natural language chat with the system disclosed in Rai modified by Wu because it would allow users to directly interact with their data and databases, questioning LLMs to provide answers based on knowledge of that data(Chakraborty).
Claim 19 is directed to a method claim with similar limitations to that of Claim 10 and as such is rejected under the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ARJUN R SWAMY whose telephone number is (571)272-9763. The examiner can normally be reached Mon-Fri 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at (571) 272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ARJUN SWAMY/Examiner, Art Unit 2654
/HAI PHAN/Supervisory Patent Examiner, Art Unit 2654
Prosecution Insights