DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment and Arguments.
Applicant’s arguments, see page 12, filed 1/14/2026, with respect to claims 1-2, 4-7, 9-10 rejection have been fully considered and are not persuasive. Applicant argues
PNG
media_image1.png
517
621
media_image1.png
Greyscale
The Examiner respectfully disagrees this does not solve problems within computer technology. Since language models are being used to only generate SQL queries and going through a database. After the answers are generated, it being evaluated. Where the human would check the result with an answer key he has. Having an evaluation is not an improvement of a computer. The claims are not patent eligible.
Applicant further argues
PNG
media_image2.png
674
599
media_image2.png
Greyscale
PNG
media_image3.png
144
595
media_image3.png
Greyscale
The Examiner respectfully disagrees a librarian receives a prompt if a book is available. Where the librarian would enter the book information a certain way to retrieve the information from the database. Where the librarian would enter the author, title, and when it was published in a certain order for the database to recognize it. Where the librarian can have an example of the format and syntax when she searches for something. Further she can search for the title of the book and see if it’s available. When she receives these results, she would then see if they both gave a similar answer. In regards for the language model being a specialized machine learning model the claims do not indicate how this is trained, Hence being a general purpose model (paragraph 6, 61, 85, 87-88). The claims are not patent eligible.
Applicant further argues that
PNG
media_image4.png
413
597
media_image4.png
Greyscale
The Examiner respectfully disagrees after the human organizes/formats it he would evaluate the result with a key answer. Having a evaluation that’s non-generic does not prevent the evaluation from being done by a human. The claims are not patent eligible.
Applicant further argues
PNG
media_image5.png
723
473
media_image5.png
Greyscale
The Examiner respectfully disagrees does not prevent a human from performing the steps mentally as described above. Additionally, the person formatting it and validating the output is a mental process. Thus, these claims are directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Further the applicant does not mention any additional elements that would take it away from being a abstract idea. Therefore, the claims are not patent eligible.
Applicant further argues
PNG
media_image6.png
729
467
media_image6.png
Greyscale
The Examiner respectfully disagrees with the improvements concerns the matter of how data is organized evaluated to determine if the result is acceptable. Such collection, organization, manipulation, and comparison as discussed above can be performed by a human/processing information. Therefore, the claims are not patent eligible.
Therefore, the 101 rejection of claims 1-2, 4-7, 9-10 are maintained.
Applicant’s arguments with respect to claim(s) 1-2, 4-7, 9-10 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. The newly modified claim limitation “obtaining a first reference query result by querying a database with the reference query syntax and organizing the first reference query result into a second reference query result presented in a preset format based on the preset format, wherein the second reference query result presented in the preset format comprises at least one reference data combination, each of the at least one reference data combination comprises a first reference query target and a first reference data corresponding to the first reference query target, and the preset format comprises a {key:value} format, wherein, for each reference data combination, the key and the value respectively correspond to the first reference query target and the first reference data; obtaining a first query syntax generated by a first language model in response to the prompt; obtaining a first query result by querying the database with the first query syntax and organizing the first query result into a second query result presented in the preset format based on the preset format, wherein the second query result presented in the preset format comprises at least one first data combination, each of the at least one first data combination comprises a first query target and a first data corresponding to the first query target, wherein, for each first data combination, the key and the value of the preset format respectively correspond to the first query target and the first data;” necessitates the new ground of rejection.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-2, 4-7, 9-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1 and 6, Further claim 1 recites A method for evaluating a language model, executed by an electronic device, comprising:
obtaining a prompt and a reference query syntax corresponding to the prompt;
obtaining a first reference query result by querying a database with the reference query syntax and organizing the first reference query result into a second reference query result presented in a preset format based on the preset format, wherein the second reference query result presented in the preset format comprises at least one reference data combination, each of the at least one reference data combination comprises a first reference query target and a first reference data corresponding to the first reference query target, and the preset format comprises a {key:value} format, wherein, for each reference data combination, the key and the value respectively correspond to the first reference query target and the first reference data;
obtaining a first query syntax generated by a first language model in response to the prompt;
obtaining a first query result by querying the database with the first query syntax and organizing the first query result into a second query result presented in the preset format based on the preset format, wherein the second query result presented in the preset format comprises at least one first data combination, each of the at least one first data combination comprises a first query target and a first data corresponding to the first query target, wherein, for each first data combination, the key and the value of the preset format respectively correspond to the first query target and the first data;
evaluating a first validity of the first query syntax provided by the first language model based on whether the second query result completely comprises the second reference query result.
Further claim 6 states a storage circuit storing a program code; and
a processor coupled to the storage circuit and accessing the program code to execute:
The limitation of “obtaining…”, “obtaining…”, “obtaining…”,“obtaining…”, and “evaluating …” , as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind. For example, a librarian receives a prompt if a book is available. Where the librarian would enter the book information a certain way to retrieve the information from the database. Where the librarian would enter the author, title, and when it was published in a certain order for the database to recognize it. Where the librarian can have an example of the format and syntax when she searches for something. Further she can search for the title of the book and see if it’s available. When she receives these results, she would then see if they both gave a similar answer.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements that are computer components “processor” (paragraph 48) and “memory” (paragraphs 47) recited at a high-level of generality such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claims are directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of using the computer components amounts to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claims are not patent eligible.
Claims 2 additionally recite The method of claim 1, wherein the step of evaluating the first validity of the first query syntax provided by the first language model based on whether the second query result completely comprises the second reference query result comprises: determining that the first query syntax provided by the first language model is valid in response to determining that the second query result completely comprises the second reference query result; and determining that the first query syntax provided by the first language model is invalid in response to determining that the second query result does not completely comprise the second reference query result. However, this limitation does not prevent a human from performing the steps mentally as described above. Further, the librarian would see if the results of searching the title and the syntax search (author, title, publication date) give a similar answer. Thus, these claims are directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Therefore, the claims are not patent eligible.
Claims 4 additionally recites the method of claim 1, wherein the reference query syntax is a correct database query syntax designed to query the database in response to the prompt. However, these limitations encompass the librarian inputting the book information in the system in a certain way (card catalog). Thus, these claims are directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Therefore, the claims are not patent eligible.
Claims 5 additionally recites The method of claim 1, further comprising: obtaining a second query syntax generated by a second language model in response to the prompt; obtaining a third query result by querying the database with the second query syntax and organizing the third query result into a fourth query result presented in the preset format based on the preset format; evaluating a second validity of the second query syntax provided by the second language model based on whether the fourth query result completely comprises the second reference query result; determining a comparison result of the first language model and the second language model by comparing the first validity and the second validity. However, these limitations encompass a librarian receiving a prompt if a book is available. Where the librarian would enter the book information a certain way to retrieve the information from the database. Where the librarian would enter the author, title, and when it was published in a certain order for the database to recognize it. Where the librarian can have an example of the formant and syntax when she searches for something. Further she can search the title of the book and see if it’s available. When she receives these results, she would than see if they both gave a similar answer. Thus, these claims are directed towards a mental process. Similar to above, no additional limitations are provided that provide a practical application, or amount to significantly more than the abstract idea. Therefore, the claims are not patent eligible.
Claim 7 contains limitations similar to those found in claims 2 and therefore are not patent eligible for the same reasons.
Claim 9 contains limitations similar to those found in claims 4 and therefore are not patent eligible for the same reasons.
Claim 10 contains limitations similar to those found in claims 5 and therefore are not patent eligible for the same reasons.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1,2,4,6,7,9 are rejected under 35 U.S.C. 103 as obvious over Gao, Dawei, et al. "Text-to-sql empowered by large language models: A benchmark evaluation." arXiv preprint arXiv:2308.15363 (2023). in view of US Patent US 20230129994 A1, (Mukherjee; Maharaj.), in view of US Patent US 20210109931 A1, (Hebert.)
Claim 1
Regarding Claim 1, Gao teach
A method for evaluating a language model, executed by an electronic device, comprising:
obtaining a prompt and a reference query syntax corresponding to the prompt;
(page 4 Section 3.2 in-context learning text-to-SQL " In Text-to-SQL, given a set of triples Q = {(𝑞𝑖,𝑠𝑖, D𝑖)}, where 𝑞𝑖 and 𝑠𝑖 are natural language question and its corresponding SQL query on database D𝑖, the target of in-context learning for Text to-SQL is to maximize the possibility of LLM M generating the correct SQL 𝑠∗ on the target question 𝑞 and database D as follows:")
obtaining a first reference query result by querying a database with the reference query syntax and organizing the first reference query result into a second reference query result presented in a preset format based on the preset format;
(page 6 section 4.1 setting: metric: " To make a fair comparison, we follow prior study [61] to use exact-set-match accuracy (EM) and execution accuracy (EX). The exact-set-match accuracy measures the matched SQL keywords between the predicted SQL query and its corresponding ground truth. The execution accuracy, on the other hand, compares the execution output of the predicted SQL query with that of the ground truth SQL query on some database instances. "
Format is being interpreted as a default output )
obtaining a first query syntax generated by a first language model in response to the prompt;
(page 3 section 3.1 question representation: basic prompt: "Basic Prompt (BS𝑃). Basic Prompt [37] is a simple representation showninListing1. It is consisted of table schemas, natural language question prefixed by “Q: ” and a response prefix “A: SELECT” to prompt LLM to generate SQL. In this paper we named it as Basic Prompt due to its absence of instructions."
Page 4 section 3.2 in context learning for text-to-SQL "In Text-to-SQL, given a set of triples Q = {(𝑞𝑖,𝑠𝑖, D𝑖)}, where 𝑞𝑖 and 𝑠𝑖 are natural language question and its corresponding SQL query on database D𝑖, the target of in-context learning for Text to-SQL is to maximize the possibility of LLM M generating the correct SQL 𝑠∗ on the target question 𝑞 and database D as follows:"
page 1 section 1 introduction " Different from prior studies, the core problem in LLM-based Text-to-SQL solution is how to prompt LLM to generate correct SQL queries, namely prompt engineering. Such prompt engineering involves question representations [7,13,33,37], examples selection [14, 28, 29], and example organization [14].")
obtaining a first query result by querying the database with the first query syntax and organizing the first query result into a second query result presented in the preset format based on the preset format;
(Page 6 section 4.1 setting: metric: " The execution accuracy, on the other hand, compares the execution output of the predicted SQL query with that of the ground truth SQL query on some database instances. "
Format is being interpreted as a default output )
evaluating a first validity of the first query syntax provided by the first language model based on whether the second query result completely comprises the second reference query result. (does not teach the bold)
(page 6 section 4.1 setting: metric: " To make a fair comparison, we follow prior study [61] to use exact-set-match accuracy (EM) and execution accuracy (EX). The exact-set-match accuracy measures the matched SQL keywords between the predicted SQL query and its corresponding ground truth. The execution accuracy, on the other hand, compares the execution output of the predicted SQL query with that of the ground truth SQL query on some database instances. This metric provides a more precise estimate of the model’s performance …" )
Gao do not explicitly teach all of wherein the second reference query result presented in the preset format comprises at least one reference data combination, each of the at least one reference data combination comprises a first reference query target and a first reference data corresponding to the first reference query target, and the preset format comprises a {key:value} format, wherein, for each reference data combination, the key and the value respectively correspond to the first reference query target and the first reference data;
wherein the second query result presented in the preset format comprises at least one first data combination, each of the at least one first data combination comprises a first query target and a first data corresponding to the first query target, wherein, for each first data combination, the key and the value of the preset format respectively correspond to the first query target and the first data;
evaluating a first validity of the first query syntax provided by the first language model based on whether the second query result completely comprises the second reference query result. (the bolded)
However, Mukherjee teaches
evaluating a first validity of the first query syntax provided by the first language model based on whether the second query result completely comprises the second reference query result.
(Paragraph 26 "In accordance with one or more embodiments of the disclosure, a computing platform comprising at least one processor, a communication interface, and memory storing computer-readable instructions may receive a source query, which may be formatted in a first format for execution on a source database. The computing platform may execute the source query on the source database to produce a first data result. The computing platform may input the first data result into a reversal logic engine to produce a target query formatted in a second format corresponding to a target database. The computing platform may execute the target query on the target database to produce a second data result. The computing platform may compare the second data result to the first data result to identify whether or not the second data result matches the first data result. Based on identifying that the second data result matches the first data result, the computing platform may validate the target query. Based on identifying that the second data result does not match the first data result, the computing platform may adjust the reversal logic engine based on a discrepancy between the second data result and the first data result.")
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gao to incorporate the teachings of Mukherjee to provide a “evaluating a first validity of the first query syntax provided by the first language model based on whether the second query result completely comprises the second reference query result.” Doing so would Ensure correctness of the generated result, as recognized by Mukherjee. (Paragraph 26).
Gao in view of Mukherjee do not explicitly teach all of wherein the second reference query result presented in the preset format comprises at least one reference data combination, each of the at least one reference data combination comprises a first reference query target and a first reference data corresponding to the first reference query target, and the preset format comprises a {key:value} format, wherein, for each reference data combination, the key and the value respectively correspond to the first reference query target and the first reference data;
wherein the second query result presented in the preset format comprises at least one first data combination, each of the at least one first data combination comprises a first query target and a first data corresponding to the first query target, wherein, for each first data combination, the key and the value of the preset format respectively correspond to the first query target and the first data;
However, HEBERT teaches wherein the second reference query result presented in the preset format comprises at least one reference data combination, each of the at least one reference data combination comprises a first reference query target and a first reference data corresponding to the first reference query target, and the preset format comprises a {key:value} format, wherein, for each reference data combination, the key and the value respectively correspond to the first reference query target and the first reference data;
(“[0025] Constructor 216 uses path data 214 to create refined queries. Consider an example with a user “Don” logged in to application 204 through browser 206. Don requests to view his profile, resulting in a “GET/myprofile” command generated by application 204 which triggers the SQL statement query “SELECT * FROM USERS.” The SQL statement can be generated by application 204 or through an API and is captured by tester 212. This query results in all information about all users being retrieved. An example of returned results, which are captured by tester 212, in JSON format for a database having four users is:…” (see the table for the remaining citation)
“[0028] Constructor 216 modifies the query to reduce or eliminate the security risk by returning less data. In this example, constructor 216 can narrow the “SELECT” statement by providing arguments to align the returned results with the displayed results, such as “SELECT name FROM USERS WHERE ‘user'='Don’”. Tester 212 can replace the initial query with the refined query. In some examples, the refined query is stored in test data 218 and is executed against database 208 to verify that the expected result of “{“name”:”Don”}” is returned. The results of executing the refined query can also be stored in test data 218. In some examples, the refined query automatically replaces the initial query. In other examples, an alert is generated to request manual approval by a developer or other technical staff. The initial query can be stored in case usage of the refined query identifies a problem so that application 204 can revert to the initial query.”)
wherein the second query result presented in the preset format comprises at least one first data combination, each of the at least one first data combination comprises a first query target and a first data corresponding to the first query target, wherein, for each first data combination, the key and the value of the preset format respectively correspond to the first query target and the first data;
(“[0025] Constructor 216 uses path data 214 to create refined queries. Consider an example with a user “Don” logged in to application 204 through browser 206. Don requests to view his profile, resulting in a “GET/myprofile” command generated by application 204 which triggers the SQL statement query “SELECT * FROM USERS.” The SQL statement can be generated by application 204 or through an API and is captured by tester 212. This query results in all information about all users being retrieved. An example of returned results, which are captured by tester 212, in JSON format for a database having four users is:…” (see the table for the remaining citation)
“[0028] Constructor 216 modifies the query to reduce or eliminate the security risk by returning less data. In this example, constructor 216 can narrow the “SELECT” statement by providing arguments to align the returned results with the displayed results, such as “SELECT name FROM USERS WHERE ‘user'='Don’”. Tester 212 can replace the initial query with the refined query. In some examples, the refined query is stored in test data 218 and is executed against database 208 to verify that the expected result of “{“name”:”Don”}” is returned. The results of executing the refined query can also be stored in test data 218. In some examples, the refined query automatically replaces the initial query. In other examples, an alert is generated to request manual approval by a developer or other technical staff. The initial query can be stored in case usage of the refined query identifies a problem so that application 204 can revert to the initial query.”)
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gao in view of Mukherjee to incorporate the teachings of HEBERT to provide a “wherein the second reference query result presented in the preset format comprises at least one reference data combination, each of the at least one reference data combination comprises a first reference query target and a first reference data corresponding to the first reference query target, and the preset format comprises a {key:value} format, wherein, for each reference data combination, the key and the value respectively correspond to the first reference query target and the first reference data; wherein the second query result presented in the preset format comprises at least one first data combination, each of the at least one first data combination comprises a first query target and a first data corresponding to the first query target, wherein, for each first data combination, the key and the value of the preset format respectively correspond to the first query target and the first data;” Doing so would reduce security risk, as recognized by HEBERT. (Paragraph 19).
Claim 6
Regarding Claim 6, Gao in view of Mukherjee in further view of HEBERT, further Gao teach
a storage circuit storing a program code; and
(Page 1 section 1 introduction " Different from prior studies, the core problem in LLM-based Text-to-SQL solution is how to prompt LLM to generate correct SQL queries, namely prompt engineering. Such prompt engineering involves question representations [7,13,33,37], examples selection [14, 28, 29], and example organization [14]."
page 4 Section 3.2 in-context learning text-to-SQL " In Text-to-SQL, given a set of triples Q = {(𝑞𝑖,𝑠𝑖, D𝑖)}, where 𝑞𝑖 and 𝑠𝑖 are natural language question and its corresponding SQL query on database D𝑖, the target of in-context learning for Text to-SQL is to maximize the possibility of LLM M generating the correct SQL 𝑠∗ on the target question 𝑞 and database D as follows:"
It would be inherent to have a program code, processor, and memory)
a processor coupled to the storage circuit and accessing the program code to execute:
(Page 1 section 1 introduction " Different from prior studies, the core problem in LLM-based Text-to-SQL solution is how to prompt LLM to generate correct SQL queries, namely prompt engineering. Such prompt engineering involves question representations [7,13,33,37], examples selection [14, 28, 29], and example organization [14]."
page 4 Section 3.2 in-context learning text-to-SQL " In Text-to-SQL, given a set of triples Q = {(𝑞𝑖,𝑠𝑖, D𝑖)}, where 𝑞𝑖 and 𝑠𝑖 are natural language question and its corresponding SQL query on database D𝑖, the target of in-context learning for Text to-SQL is to maximize the possibility of LLM M generating the correct SQL 𝑠∗ on the target question 𝑞 and database D as follows:"
It would be inherent to have a program code, processor, and memory)
Claim 6 contains limitations similar to those found in claims 1 and therefore are not patent eligible for the same reasons.
Claim 2 and 7
Regarding Claim 2 and 7, Gao in view of Mukherjee in further view of HEBERT, further Gao teaches 2. The method of claim 1, wherein the step of evaluating the first validity of the first query syntax provided by the first language model based on whether the second query result completely comprises the second reference query result comprises:
determining that the first query syntax provided by the first language model is valid in response to determining that the second query result completely comprises the second reference query result; and (except the bolded)
(page 6 section 4.1 setting: metric: " To make a fair comparison, we follow prior study [61] to use exact-set-match accuracy (EM) and execution accuracy (EX). The exact-set-match accuracy measures the matched SQL keywords between the predicted SQL query and its corresponding ground truth. The execution accuracy, on the other hand, compares the execution output of the predicted SQL query with that of the ground truth SQL query on some database instances. This metric provides a more precise estimate of the model’s performance …" )
determining that the first query syntax provided by the first language model is invalid in response to determining that the second query result does not completely comprise the second reference query result. (except the bolded)
(page 6 section 4.1 setting: metric: " To make a fair comparison, we follow prior study [61] to use exact-set-match accuracy (EM) and execution accuracy (EX). The exact-set-match accuracy measures the matched SQL keywords between the predicted SQL query and its corresponding ground truth. The execution accuracy, on the other hand, compares the execution output of the predicted SQL query with that of the ground truth SQL query on some database instances. This metric provides a more precise estimate of the model’s performance …" )
Gao in view of Mukherjee in further view of HEBERT, further Mukherjee teaches determining that the first query syntax provided by the first language model is valid in response to determining that the second query result completely comprises the second reference query result; and(Paragraph 26 "In accordance with one or more embodiments of the disclosure, a computing platform comprising at least one processor, a communication interface, and memory storing computer-readable instructions may receive a source query, which may be formatted in a first format for execution on a source database. The computing platform may execute the source query on the source database to produce a first data result. The computing platform may input the first data result into a reversal logic engine to produce a target query formatted in a second format corresponding to a target database. The computing platform may execute the target query on the target database to produce a second data result. The computing platform may compare the second data result to the first data result to identify whether or not the second data result matches the first data result. Based on identifying that the second data result matches the first data result, the computing platform may validate the target query. Based on identifying that the second data result does not match the first data result, the computing platform may adjust the reversal logic engine based on a discrepancy between the second data result and the first data result.")
determining that the first query syntax provided by the first language model is invalid in response to determining that the second query result does not completely comprise the second reference query result.(Paragraph 26 "In accordance with one or more embodiments of the disclosure, a computing platform comprising at least one processor, a communication interface, and memory storing computer-readable instructions may receive a source query, which may be formatted in a first format for execution on a source database. The computing platform may execute the source query on the source database to produce a first data result. The computing platform may input the first data result into a reversal logic engine to produce a target query formatted in a second format corresponding to a target database. The computing platform may execute the target query on the target database to produce a second data result. The computing platform may compare the second data result to the first data result to identify whether or not the second data result matches the first data result. Based on identifying that the second data result matches the first data result, the computing platform may validate the target query. Based on identifying that the second data result does not match the first data result, the computing platform may adjust the reversal logic engine based on a discrepancy between the second data result and the first data result.")
See claim one for rationale.
Claim 4 and 9
Regarding Claim 4 and 9, Gao in view of Mukherjee in further view of HEBERT, further Gao teaches
The method of claim 1, wherein the reference query syntax is a correct database query syntax designed to query the database in response to the prompt.
(Section 4 experiment: datasets " Each instance is consisted of a natural language question on a specific database and its corresponding SQL query. ")
Claims 5 and 10 are rejected under 35 U.S.C. 103 as obvious over Gao, Dawei, et al. "Text-to-sql empowered by large language models: A benchmark evaluation." arXiv preprint arXiv:2308.15363 (2023). in view of US Patent US 20230129994 A1, (Mukherjee; Maharaj.), in view of US Patent US 20210109931 A1, (Hebert.) in further view of US Patent US 12554709 B2, (Sun; Ruoxi.)
Claim 5 and 10
Regarding Claim 5 and 10, Gao teach
5. The method of claim 1, further comprising:
obtaining a second query syntax generated by a second language model in response to the prompt; (except the bolded )
(page 4 Section 3.2 in-context learning text-to-SQL " In Text-to-SQL, given a set of triples Q = {(𝑞𝑖,𝑠𝑖, D𝑖)}, where 𝑞𝑖 and 𝑠𝑖 are natural language question and its corresponding SQL query on database D𝑖, the target of in-context learning for Text to-SQL is to maximize the possibility of LLM M generating the correct SQL 𝑠∗ on the target question 𝑞 and database D as follows:")
obtaining a third query result by querying the database with the second query syntax and organizing the third query result into a fourth query result presented in the preset format based on the preset format;
(page 6 section 4.1 setting: metric: " To make a fair comparison, we follow prior study [61] to use exact-set-match accuracy (EM) and execution accuracy (EX). The exact-set-match accuracy measures the matched SQL keywords between the predicted SQL query and its corresponding ground truth. The execution accuracy, on the other hand, compares the execution output of the predicted SQL query with that of the ground truth SQL query on some database instances. "
Format is being interpreted as a default output )
evaluating a second validity of the second query syntax provided by the second language model based on whether the fourth query result completely comprises the second reference query result; (except the bolded )
(page 6 section 4.1 setting: metric: " To make a fair comparison, we follow prior study [61] to use exact-set-match accuracy (EM) and execution accuracy (EX). The exact-set-match accuracy measures the matched SQL keywords between the predicted SQL query and its corresponding ground truth. The execution accuracy, on the other hand, compares the execution output of the predicted SQL query with that of the ground truth SQL query on some database instances. This metric provides a more precise estimate of the model’s performance …" )
Gao do not explicitly teach all of obtaining a second query syntax generated by a second language model in response to the prompt; (the bolded )
evaluating a second validity of the second query syntax provided by the second language model based on whether the fourth query result completely comprises the second reference query result; (bolded )
a second validity of the second query syntax provided by the second language model based on whether the fourth query result
completely comprises the second reference query result.
determining a comparison result of the first language model and the second language model by comparing the first validity and the second validity.
However, Mukherjee teaches
completely comprises the second reference query result.(Paragraph 26 "In accordance with one or more embodiments of the disclosure, a computing platform comprising at least one processor, a communication interface, and memory storing computer-readable instructions may receive a source query, which may be formatted in a first format for execution on a source database. The computing platform may execute the source query on the source database to produce a first data result. The computing platform may input the first data result into a reversal logic engine to produce a target query formatted in a second format corresponding to a target database. The computing platform may execute the target query on the target database to produce a second data result. The computing platform may compare the second data result to the first data result to identify whether or not the second data result matches the first data result. Based on identifying that the second data result matches the first data result, the computing platform may validate the target query. Based on identifying that the second data result does not match the first data result, the computing platform may adjust the reversal logic engine based on a discrepancy between the second data result and the first data result.")
determining a comparison result of the first language model and the second language model by comparing the first validity and the second validity.
(Paragraph 26 "In accordance with one or more embodiments of the disclosure, a computing platform comprising at least one processor, a communication interface, and memory storing computer-readable instructions may receive a source query, which may be formatted in a first format for execution on a source database. The computing platform may execute the source query on the source database to produce a first data result. The computing platform may input the first data result into a reversal logic engine to produce a target query formatted in a second format corresponding to a target database. The computing platform may execute the target query on the target database to produce a second data result. The computing platform may compare the second data result to the first data result to identify whether or not the second data result matches the first data result. Based on identifying that the second data result matches the first data result, the computing platform may validate the target query. Based on identifying that the second data result does not match the first data result, the computing platform may adjust the reversal logic engine based on a discrepancy between the second data result and the first data result.")
See claim one for rationale.
Gao in view of Mukherjee in further view of HEBERT do not explicitly teach all of obtaining a second query syntax generated by a second language model in response to the prompt; a second validity of the second query syntax provided by the second language model based on whether the fourth query result
However, Sun teach obtaining a second query syntax generated by a second language model in response to the prompt;(col2 lines 55-64 "In an example, converting the natural language query into a database language query includes: generating various database description prompts; sampling one or more large language models (LLMs) multiple times with the various database description prompts to generate a plurality of potential database language queries; executing the plurality of potential database language queries to generate a plurality of potential results; and selecting the database language query that provides a result consistent with a threshold amount of the plurality of potential results.")
a second validity of the second query syntax provided by the second language model based on whether the fourth query result
(col2 lines 55-64 "In an example, converting the natural language query into a database language query includes: generating various database description prompts; sampling one or more large language models (LLMs) multiple times with the various database description prompts to generate a plurality of potential database language queries; executing the plurality of potential database language queries to generate a plurality of potential results; and selecting the database language query that provides a result consistent with a threshold amount of the plurality of potential results."
Col 6 lines 18-23 "The execution engine 112 can be configured to execute the potential database query languages to generate corresponding potential results. The execution engine 112 can be further configured to remove errors from the corresponding results. The execution engine 112 can also be configured to concatenate the corresponding results.")
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Gao in view of Mukherjee in further view of HEBERT to incorporate the teachings of Sun to provide a “obtaining a second query syntax generated by a second language model in response to the prompt; a second validity of the second query syntax provided by the second language model based on whether the fourth query result ” Doing so would add robustness against individual model failures, as recognized by Sun . (col2 lines 55-64 & col 6 lines 24-34).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALI M HASSAN whose telephone number is (571)272-5331. The examiner can normally be reached Monday - Friday 8:00am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Paras Shah can be reached at (571)270-1650. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALI M HASSAN/Examiner, Art Unit 2653
/Paras D Shah/Supervisory Patent Examiner, Art Unit 2653
09/05/2026
Prosecution Insights