DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the application and claims filed 10/30/2023. Claims 1-20 are pending and have been examined. Claims 1-20 are rejected.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. The present application claims priority to U.S. Provisional Application No. 63/510,074, filed on 06/23/2023.
Information Disclosure Statement
Acknowledgment is made of the information disclosure statement filed 9/13/2024 which complies with 37 CFR 1.97. As such, the information disclosure statement has been placed in the application file and the information referred to therein has been considered by the examiner.
Claim Objections
Claims 1-20 are objected to because of the following informalities:
Independent claims 1, 9 and 17 each recite “the one or more source document;” (see, line 7 of claim 1, line 11 of claim 9 and line 9 of claim 17. These recitations include a typographical error and should read “the one or more source documents;” (see, line 4 of claim 1, line 8 of claim 9 and line 6 of claim 17, which introduced “one or more source documents”). Appropriate correction is required.
Also, claims 2-8, 10-16 and 18-20, which each depend directly or indirectly from claims 1, 9 and 17, respectively, are objected to based on their respective dependencies from claims 1, 9 and 17.
Specification
The disclosure is objected to because of the following informalities:
The specification is objected to as failing to provide proper antecedent basis for the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). Correction of the following is required:
Claims 17 - 20 do not appear to have support in the originally filed specification. There does not appear to be any discussion of any “processor-readable medium.” The specification fails to mention, let alone describe or discuss any “processor-readable medium” as recited in claim 17, any such “medium” as recited in claims 18 – 20 or any “processor-readable” media. Appropriate correction is required.
The title of the invention is objected to as being unclear. A new title is required that is clearly indicative of the invention to which the claims are directed. In particular, the title of the invention is “SYSTEMS AND METHODS FOR RETRIEVAL BASED QUESTION ANSWERING USING
NEURA NETWORK MODELS”; however, the title includes a typographical error. In particular, it appears that “NEURA NETWORK MODELS” should read “NEURAL NETWORK MODELS”. The examiner suggests that one way to address this objection would be to amend the title to read “SYSTEMS AND METHODS FOR RETRIEVAL BASED QUESTION ANSWERING USING NEURAL NETWORK MODELS”.
A correctly spelled title indicative of the invention will help in proper indexing, classifying, searching, etc. See, MPEP § 606.01. However, the title of the invention should be limited to 500 characters. The examiner suggests including the aspect(s) of the claims which Applicant believes to be novel or nonobvious over the prior art.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 3-4, 6, 11-12, 14 and 18-19 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claims 3, 11 and 18 each recite “wherein the respective answer is generated by the first language model further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information to answer the question.” (see, lines 1 - 4 of claims 3, 11 and 18). Claims 4, 12 and 19 each recite “wherein the respective answer is generated by the first language model further based on a prompt that contains a demonstration differentiating irrelevant source documents from relevant source documents that contain sufficient information to answer the question.” (see, lines 1-4 of claims 4, 12 and 19).
The terms “irrelevant”, “relevant”, “insufficient” and “sufficient” are relative terms which render the claims indefinite. Specifically, the terms “irrelevant” and “insufficient” recited in claims 3, 11 and 18, and the terms “relevant” and “sufficient” recited in claims 4, 12 and 19 are not defined by the claims, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention.
For examination purposes, 1) “irrelevant source documents” has been interpreted as any source documents, data objects or files that are not deemed to be pertinent to or relevant for the claimed purpose 2) “relevant source documents” has been interpreted as any source documents, data objects or files that are deemed to be pertinent to or relevant for the claimed purpose 3) “insufficient information” has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose and 4) “sufficient information” has been interpreted as any information or data that is sufficient, pertinent to, or useful for the claimed purpose . Appropriate correction is required.
Claims 6 and 14 both recite “generating, by the first language model, the respective answer when the initial answer indicates the concatenation of the one or more source documents contain insufficient information to answer the question.” (see, lines 5-7 of claims 6 and 14).
For examination purposes “insufficient information” has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The analysis below of the claims’ subject matter eligibility follows the 2019 Revised Patent Subject Matter Eligibility Guidance, 84 Fed. Reg. 50-57 (January 7, 2019) (“2019 PEG”) and the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, 89 Fed. Reg. 58128-58138 (July 17, 2024) (“2024 AI SME Update”).
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter (Step 1). If the claim does fall within one of the statutory categories, the second step in the analysis is to determine whether the claim is directed to a judicial exception (Step 2A, Prong 1), it is determined whether or not the claims recite a judicial exception (e.g., mathematical concepts, mental processes, certain methods of organizing human activity). If it is determined in Step 2A, Prong 1 that the claims recite a judicial exception, the analysis proceeds to the second prong (Step 2A, Prong 2), where it is determined whether or not the claims integrate the judicial exception into a practical application, the analysis proceeds to determining whether the claim is a patent-eligible application of the exception (Step 2B). If an abstract idea is present in the claim, any element or combination of elements in the claim must be sufficient to ensure that the claim integrates the judicial exception into a practical application, or else amounts to significantly more than the abstract idea itself.
Regarding independent claim 1, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 1 is directed to a method, corresponding to a process, one of the statutory categories.
Step 2A Prong 1: The claim recites
“selecting … one or more source documents based on the question;
generating … a respective answer from an input combining the question and a respective source document from the one or more source document;
generating a respective indicator associated with the respective answer indicating a quality of the respective answer; and
generating … a response to the user input by selecting an answer from generated answers based on respective indicators.”
The “selecting … one or more source documents” limitation, as drafted, under its broadest reasonable interpretation (BRI ), covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to select “one or more source documents” based on a received/observed question (corresponding to mental processes which can be done mentally or by pen and paper).
The “generating … a respective answer” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine an answer based on an observed/received input (corresponding to mental processes which can be done mentally or by pen and paper).
The “generating a respective indicator” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “final answer based on respective indicators.” (corresponding to mental processes which can be done mentally or by pen and paper).
The “generating … a response to the user input” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to generate/determine “a response … from … answers based on respective indicators.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites
“A method for generating an answer to an input question using one or more neural network models, comprising:
receiving, via a user interface, a user input indicating a question; …
by a retrieval model at a server …
by a first language model …and …
via the user interface”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites “A method for generating an answer … using one or more neural network models”, “a retrieval model”, “a first language model” and “the user interface” limitations which amount to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – generically-recited models - neural network, retrieval and language, and a generic user interface performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
The “one or more neural network models”, “retrieval model” and “first language model” are each recited at a high level of generality and therefore are being interpreted as performing a mental process on a generic computer. See MPEP 2106.04(a)(2) § III.C which states that “a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept” still recite a mental process.
Regarding the “one or more neural network models”, “retrieval model” and “first language model” limitations, no details of models or their training are recited and the models give the indication that they can be constructed and adjusted or modified by hand with pen and paper based on observed and received/given data (i.e., the “user input indicating a question” and “the one or more source document”) by using evaluation/judgement to generate an answer to the question. Given a sufficiently small set of data, such determining can be done mentally or with pen and paper.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., “a retrieval model”, “a first language model” and “the user interface” ), then they fall within the “Mental Processes” grouping of abstract ideas.
The “receiving … a user input indicating a question” limitation describes data gathering with a user input as the data. This receiving a user input limitation can be characterized as insignificant extra-solution activity (i.e., data gathering). See MPEP 2106.05(g).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “method for generating an answer”, “a retrieval model”, “a first language model” and “the user interface” limitations do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites “generating an answer to an input question”, retrieving a model, by a model, and via an interface for applying the abstract ideas). Mere instructions to generate, retrieve or use the generic computer component do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer – generic models cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
The “receiving … a user input indicating a question” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Receiving, communicating, and storing data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, the recitation of receiving a user input indicating a question is a well-understood, routine, conventional activity of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
This claim is not patent eligible.
Regarding claim 2, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 2 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective indicator is generated … based on an input combining the respective answer and the question.”
The “the respective indicator is generated” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to generate/identify an indicator based on observed/received input combining an answer and a question (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites
“by a second language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “second language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “second language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites “by a second language model” for applying the abstract ideas). Mere instructions to use a second language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic model cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 3, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 3 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective answer is generated … further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information to answer the question.”1
The “respective answer is generated” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine a negative answer when the language model finds a source document containing insufficient information to answer the question. (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites
“by a second language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “second language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “second language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by a second language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 4, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 4 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective answer is generated … further based on a prompt that contains a demonstration differentiating irrelevant source documents from relevant source documents that contain sufficient information to answer the question.”2
The “differentiating irrelevant source documents” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “irrelevant or relevant source documents and sufficient information.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 5, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 5 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective answer is generated … further based on a prompt that contains a demonstration generating a summary of the respective source document, based on which an answer is generated.”
The “generating a summary of the respective source document” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “a summary of the respective source document.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 6, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 6 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites:
“generating … an initial answer from an input combining the question and a concatenation of the one or more source documents prior to generating respective answers based on respective source documents independently; and generating … the respective answer when the initial answer indicates the concatenation of the one or more source documents contain insufficient information to answer the question.”3
The “answers based on respective source documents” and “source documents contain insufficient information” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “an answer from an input combining the question and a respective source document from the one or more source document” and “a negative answer when the first language model determines that the respective source document contains insufficient information to answer the question.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model, (first)
by the first language model” (second)
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” x 2, limitations which amount to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” x 2, limitations do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to user language models do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 7, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 7 is directed to a method as depending from claim 1, thus the analysis for patent eligibility of claim 1 is incorporated herein.
Step 2A Prong 1: The claim recites:
“removing at least one source documents from the one or more source documents based on respective indicators.”
The “removing at least one source documents” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine if “the respective answer is ‘unknown.’” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 8, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 8 is directed to a method as depending from claim 7, thus the analysis for patent eligibility of claim 7 is incorporated herein.
Step 2A Prong 1: The claim recites:
“generating … a final answer using an input combining unremoved source documents and corresponding answers.”
The “a final answer using an input” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “a final answer using an input.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding independent claim 9, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 9 is directed to a system, corresponding to a machine, one of the statutory categories.
Step 2A Prong 1: The claim recites:
“selecting … one or more source documents based on the question;
generating … a respective answer from an input combining the question and a respective source document from the one or more source document;
generating a respective indicator associated with the respective answer indicating a quality of the respective answer; and
generating … a response to the user input by selecting an answer from generated answers based on respective indicators.”
The “selecting … one or more source documents” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “one or more source documents.” (corresponding to mental processes which can be done mentally or by pen and paper).
The “generating … a respective answer” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “an answer from an input.” (corresponding to mental processes which can be done mentally or by pen and paper).
The “generating a respective indicator” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “final answer based on respective indicators.” (corresponding to mental processes which can be done mentally or by pen and paper).
The “generating … a response to the user input” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to generate/determine “a response … from … answers based on respective indicators.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites
“A system for generating an answer to an input question using one or more neural network models, the system comprising:
a communication interface configured to receive, via a user interface, a user input indicating a question;
a memory storing a plurality of processor-executable instructions;
and one or more processors executing the instructions to perform operations comprising: … by a retrieval model at a server …
by a first language model …
“via the user interface”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “system for generating an answer”, “processor executable instructions”, “a retrieval model”, “a first language model” and “the user interface” limitations which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – memory, generically-recited models (neural network, retrieval, language), and user interface performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
The “system”, “memory”, “retrieval model”, “first language model”, “server”, “neural network models” are each recited at a high level of generality and therefore are being interpreted as performing a mental process on a generic computer. See MPEP 2106.04(a)(2) § III.C which states that “a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept” still recite a mental process.
Regarding the “system for generating an answer”, “processor executable instructions”, “a retrieval model”, “a first language model” and “the user interface” limitations, no details of models or their training are recited and the models give the indication that they can be constructed and adjusted or modified by hand with pen and paper based on observed and received/given data (i.e., the “user input indicating a question” and “the one or more source document”) by using evaluation/judgement to generate an answer to the question. Given a sufficiently small set of data, such determining can be done mentally or with pen and paper.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., “system”, “memory”, “retrieval model”, “first language model”, “server”, “neural network models”), then they fall within the “Mental Processes” grouping of abstract ideas.
The “user input indicating a question” limitation describes data gathering with a user input as the data. This receiving a user input limitation can be characterized as insignificant extra-solution activity (i.e., data gathering). See MPEP 2106.05(g).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “system for generating an answer”, “processor executable instructions”, “a retrieval model”, “a first language model” and “the user interface” limitations do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites “generating an answer to an input question”, “a memory”, retrieving a model, by a model, and via an interface for applying the abstract ideas for applying the abstract ideas). Mere instructions to generate, retrieve or use the generic computer component do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer – generic models cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
The “user input indicating a question” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Receiving, communicating, and storing data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, the recitation of receiving a user input indicating a question is a well-understood, routine, conventional activity of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
This claim is not patent eligible.
Regarding claim 10, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 10 is directed to a system as depending from claim 9, thus the analysis for patent eligibility of claim 9 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective indicator is generated … based on an input combining the respective answer and the question.”
The “the respective indicator is generated” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to generate/identify an indicator based on observed/received input combining an answer and a question(corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites
“by a second language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “second language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “second language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites “by a second language model” for applying the abstract ideas). Mere instructions to use a second language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic model cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 11, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 11 is directed to a system as depending from claim 9, thus the analysis for patent eligibility of claim 9 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective answer is generated … further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information to answer the question.”4
The “respective answer is generated” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine a negative answer when the language model finds a source document containing insufficient information to answer the question. (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites
“by a second language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “second language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “second language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by a second language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 12, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 12 is directed to a system as depending from claim 9, thus the analysis for patent eligibility of claim 9 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective answer is generated … further based on a prompt that contains a demonstration differentiating irrelevant source documents from relevant source documents that contain sufficient information to answer the question.”5
The “differentiating irrelevant source documents” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “irrelevant or relevant source documents and sufficient information.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 13, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 13 is directed to a system as depending from claim 9, thus the analysis for patent eligibility of claim 9 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective answer is generated … further based on a prompt that contains a demonstration generating a summary of the respective source document, based on which an answer is generated.”
The “generating a summary of the respective source document” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “a summary of the respective source document.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 14, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 14 is directed to a system as depending from claim 9, thus the analysis for patent eligibility of claim 9 is incorporated herein.
Step 2A Prong 1: The claim recites:
“generating … an initial answer from an input combining the question and a concatenation of the one or more source documents prior to generating respective answers based on respective source documents independently; and
generating … the respective answer when the initial answer indicates the concatenation of the one or more source documents contain insufficient information to answer the question.”6
The “answers based on respective source documents” and “source documents contain insufficient information” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “an answer from an input combining the question and a respective source document from the one or more source document” and “a negative answer when the first language model determines that the respective source document contains insufficient information to answer the question.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model, (first)
by the first language model” (second)
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” x 2, limitations which amount to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” x 2, limitations do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to user language models do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 15, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 15 is directed to a system as depending from claim 9, thus the analysis for patent eligibility of claim 9 is incorporated herein.
Step 2A Prong 1: The claim recites:
“removing at least one source documents from the one or more source documents based on respective indicators.”
The “removing at least one source documents” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine if “the respective answer is ‘unknown.’” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The judicial exceptions are not integrated into a practical application.
The claim does not recite any additional elements that integrate the abstract idea into a practical application or provide significantly more than the abstract idea, and thus the claim is subject-matter ineligible.
Step 2B: The claim does not recite additional elements that are sufficient to amount to significantly more than the judicial exception.
This claim is not patent eligible.
Regarding claim 16, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 16 is directed to a system as depending from claim 15, thus the analysis for patent eligibility of claim 15 is incorporated herein.
Step 2A Prong 1: The claim recites:
“generating … a final answer using an input combining unremoved source documents and corresponding answers.”
The “a final answer using an input” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “a final answer using an input.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding independent claim 17, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 17 is directed to a non-transitory processor-readable medium, corresponding to a manufacture, one of the statutory categories.
Step 2A Prong 1: The claim recites
“selecting … one or more source documents based on the question;
generating … a respective answer from an input combining the question and a respective source document from the one or more source document;
generating a respective indicator associated with the respective answer indicating a quality of the respective answer; and
generating … a response to the user input by selecting an answer from generated answers based on respective indicators.”
The “selecting … one or more source documents” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to select “one or more source documents.” (corresponding to mental processes which can be done mentally or by pen and paper).
The “generating … a respective answer” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “an answer from an input.” (corresponding to mental processes which can be done mentally or by pen and paper).
The “generating a respective indicator” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “final answer based on respective indicators.” (corresponding to mental processes which can be done mentally or by pen and paper).
The “generating … a response to the user input” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to generate/determine “a response … from … answers based on respective indicators.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites
“A non-transitory processor-readable medium storing a plurality of processor-executable instructions for generating an answer to an input question using one or more neural network models, the instructions being executed by one or more processors to perform operations comprising:
receiving, via a user interface, a user input indicating a question; …
by a retrieval model at a server …
by a first language model …
“via the user interface”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “processor executable instructions for generating an answer”, “a retrieval model”, “a first language model” and “the user interface” limitations which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – generically-recited models (neural network, retrieval, language), and user interface performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
The “non-transitory processor-readable medium” , “neural network models”, “processors”, “user interface”, “retrieval model”, “first language model”, “server” are each recited at a high level of generality and therefore are being interpreted as performing a mental process on a generic computer. See MPEP 2106.04(a)(2) § III.C which states that “a concept that is performed in the human mind and applicant is merely claiming that concept performed 1) on a generic computer, or 2) in a computer environment, or 3) is merely using a computer as a tool to perform the concept” still recite a mental process.
Regarding the “processor executable instructions for generating an answer”, “a retrieval model”, “a first language model” and “the user interface” limitations, no details of models or their training are recited and the models give the indication that they can be constructed and adjusted or modified by hand with pen and paper based on observed and received/given data (i.e., the “user input indicating a question” and “the one or more source document”) by using evaluation/judgement to generate an answer to the question. Given a sufficiently small set of data, such determining can be done mentally or with pen and paper.
If the claim limitations, under their broadest reasonable interpretations, cover performance of the limitations in the mind but for the recitation of generic computer components (i.e., “non-transitory processor-readable medium” , “neural network models”, “processors”, “user interface”, “retrieval model”, “first language model”, “server”), then they fall within the “Mental Processes” grouping of abstract ideas.
The “user input indicating a question” limitation describes data gathering with a user input as the data. This receiving a user input limitation can be characterized as insignificant extra-solution activity (i.e., data gathering). See MPEP 2106.05(g).
Accordingly, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “processor-readable medium for generating an answer”, “a retrieval model”, “a first language model” and “the user interface” limitations do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites “generating an answer to an input question”, retrieving a model, by a model, and via an interface for applying the abstract ideas). Mere instructions to generate, retrieve or use the generic computer component do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer – generic models cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
The “user input indicating a question” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Receiving, communicating, and storing data are insignificant extra-solution activities that are well-understood, routine, and conventional. See MPEP2106.05(d)(II) (“The courts have recognized the following computer functions as well‐understood, routine, and conventional functions… i. Receiving or transmitting data over a network…iv. Storing and retrieving information in memory”) (citing OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015)). Therefore, the recitation of receiving a user input indicating a question is a well-understood, routine, conventional activity of receiving or transmitting data over a network, as discussed in MPEP § 2106.05(d).
This claim is not patent eligible.
Regarding claim 18, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 18 is directed to a non-transitory processor-readable medium as depending from claim 17, thus the analysis for patent eligibility of claim 17 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective answer is generated … further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information to answer the question.”7
The “respective answer is generated” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to generate/identify an indicator based on observed/received input combining an answer and a question (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites
“by a second language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “second language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “second language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by a second language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 19, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 19 is directed to a non-transitory processor-readable medium, as depending from claim 17, thus the analysis for patent eligibility of claim 17 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective answer is generated … further based on a prompt that contains a demonstration differentiating irrelevant source documents from relevant source documents that contain sufficient information to answer the question.”8
The “differentiating irrelevant source documents” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “irrelevant or relevant source documents and sufficient information.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Regarding claim 20, this claim is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 20 is directed to a non-transitory processor-readable medium as depending from claim 17, thus the analysis for patent eligibility of claim 17 is incorporated herein.
Step 2A Prong 1: The claim recites:
“wherein the respective answer is generated … further based on a prompt that contains a demonstration generating a summary of the respective source document, based on which an answer is generated.”
The “generating a summary of the respective source document” limitation, as drafted, under its BRI, covers concepts performed in the human mind (including an observation, evaluation, judgement, or opinion) to determine “a summary of the respective source document.” (corresponding to mental processes which can be done mentally or by pen and paper).
Step 2A Prong 2: The claim recites:
“by the first language model”
The judicial exceptions are not integrated into a practical application. In particular, the claim recites the “first language model” limitation which amounts to the recitation of the words “apply it” (or an equivalent) or amounts to no more than mere instructions to implement an abstract idea or other exception on a computer or merely uses a computer as a tool to perform an abstract idea (i.e., generic computer components – a generically-recited “language model” performing generic computer functions) which does not integrate a judicial exception into a practical application. See MPEP 2106.05(f).
Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea.
Step 2B: The “first language model” limitation does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element represents mere instructions to apply an exception (i.e., the additional element recites by the first language model for applying the abstract ideas). Mere instructions to use a language model do not amount to significantly more than the judicial exception. As noted above, merely asserting that a judicial exception is to be carried out on a generic computer cannot provide significantly more than the judicial exception. Mere instructions to apply an exception cannot provide an inventive concept.
This claim is not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-2, 5, 9-10, 13, 17, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mandry. (U.S. Publication No. 20220351722, hereinafter “Mandry”).
Regarding independent claim 1, Mandry discloses the invention as claimed including A method for generating an answer to an input question using one or more neural network models, comprising (see, e.g., paragraph 77, “a particular set of one or more ML models of the second ML models 422 that is configured to extract answers to questions (e.g., the questions 404) from documents having the domain 402.” [i.e., one or more neural network models/ ML models generating an answer/extract answer to an input question/to questions]):
receiving, via a user interface, a user input indicating a question (see, e.g., paragraph 39, “may be configured to cause display of a graphical user interface (GUI) at the entity device 130 to facilitate the above-described operations, such as by displaying questions that the virtual assistant application 110 is configured to answer for user selection” [i.e., via a user interface/graphical user interface, receiving user input of a question/ displaying questions to answer for user selection]);
selecting, by a retrieval model at a server, one or more source documents based on the question (see, e.g., paragraphs 73 and 88, “or a device that is coupled to or accessible to the server 102 via the one or more networks 160) a second set of one or more ML models (referred to herein as “the second ML models 422) that are configured to extract one or more responses from input documents based on domain-specific questions” and “Thus, the UI 500 enables a user to select an input document for use in generating responses to be used by a virtual assistant” [i.e., by a retrieval model at a server/or a device (e.g., second ML model – which evaluates responses akin to a retrieval model) that is coupled to or accessible to the server, selecting one or more source documents based on the question/extracting (by selecting the input document) one or more responses from input documents based on questions]);
generating, by a first language model, a respective answer from an input combining the question and a respective source document from the one or more source document (see, e.g., paragraphs 75 and 77, “one or more natural language processing (NLP) operations on the text to generate feature data, one or more automatic speech recognition (ASR) operations if the document 441 is in an audio-based format, other processing operations, or the like, to convert the document 441 into a format that is useable by the response generator 420 or the second ML models 422” and “The response generator 420 may extract one or more candidate responses 406 from the document 441 based on the questions 404” [i.e., generating an answer by a language model/ML model that performs natural language processing operations to generate feature data by combining the question and one or more source documents/by converting the document into a format usable by response generator which extracts responses from the document based on the questions]);
generating a respective indicator associated with the respective answer indicating a quality of the respective answer (see, e.g., paragraphs 78 and 79, “in order to configure the second ML models 422 to extract answers to the questions 404 even though the particular responses may be in different locations or have different formats in different types of leases. In this manner, the second ML models 422 may generate the candidate responses 406 instead of requiring the entity to manually enter information from each of the various leases” and “In some implementations, the second ML models 422 may include multiple different sets of ML models configured to extract responses from input documents based on domain-specific questions, and each set of ML models of the second ML models 422 may be configured to generate respective accuracy scores … Each of the responses may be associated with a respective accuracy score generated by the set of ML models ... The response generator 420 may select one or more of the candidate responses 406 based on the accuracy scores 408, and the selected responses may be output to the databases 132 as responses 444. In some implementations, the response generator 420 may select candidate responses that are associated with accuracy scores that satisfy (e.g., are greater than, or greater than or equal to) a threshold … the response generator 420 may select a subset of the candidate responses 406 having a highest accuracy score(s)” [i.e., (respective indicator/accuracy score) indicating quality of the respective answer/the greater than the accuracy score threshold the higher quality the answer]);
generating, via the user interface, a response to the user input by selecting an answer from generated answers based on respective indicators. (see, e.g., paragraphs 78 and 79, “in order to configure the second ML models 422 to extract answers to the questions 404 even though the particular responses may be in different locations or have different formats in different types of leases. In this manner, the second ML models 422 may generate the candidate responses 406 instead of requiring the entity to manually enter information from each of the various leases” and “In some implementations, the response generator 420 may select candidate responses that are associated with accuracy scores that satisfy (e.g., are greater than, or greater than or equal to) a threshold. For example, if the first accuracy score and the third accuracy score of the accuracy scores 408 satisfy a threshold, the responses 444 may include the first responses and the third responses of the candidate responses 406. In some other implementations, the response generator 420 may select a subset of the candidate responses 406 having a highest accuracy score(s)” [i.e., responses to user input are generated/ candidate responses are generated, by selecting an answer based on respective indicators/responses that are associated with accuracy scores that satisfy a threshold - response generator selects response with highest accuracy score]).
Regarding claim 2, as discussed above, Mandry discloses the method of claim 1.
Mandry further discloses wherein the respective indicator is generated by a second language model based on an input combining the respective answer and the question (see, e.g., paragraphs 78 and 79, “The second ML models 422 may be trained using training data based on leases associated with different terms, states, lessee types, etc., in order to configure the second ML models 422 to extract answers to the questions 404 even though the particular responses may be in different locations or have different formats in different types of leases” and “In some implementations, the second ML models 422 may include multiple different sets of ML models configured to extract responses from input documents based on domain-specific questions, and each set of ML models of the second ML models 422 may be configured to generate respective accuracy scores. For example, the candidate responses 406 may include first responses generated by a first set of ML models of the second ML models 422, second responses generated by a second set of ML models of the second ML models 422, and third responses generated by a third set of ML models of the second ML models 422. Each of the responses may be associated with a respective accuracy score generated by the set of ML models” [e.g., (Second language model/second machine learning/ML model) uses answers/training data from first model to extract “new” answers to questions combined with training data/”old” answers]).
Regarding claim 5, as discussed above, Mandry discloses the method of claim 1.
Mandry further discloses wherein the respective answer is generated by the first language model further based on a prompt that contains a demonstration generating a summary of the respective source document, based on which an answer is generated (see, e.g., paragraph 102, “In some implementations, the one or more ML models are further configured to generate summaries of the responses based on the input documents” [e.g., (language model generates an answer based on a summary of a source document/ML model generates summaries on the input documents based on the (responses/answers)]).
Regarding independent claim 9, Mandry discloses the invention as claimed including A system for generating an answer to an input question using one or more neural network models, the system comprising: (see, e.g., paragraphs 6 and 8, “The system may also leverage artificial intelligence and machine learning to extract responses to domain-specific questions from an input document with little to no user input” and “The one or more ML models may also be configured to generate training phrase(s) associated with the responses, the training phrases for use in indexing or categorizing the responses in the response database further training the one or ML models, or both” [i.e., (A system using one or more neural network models/the system may leverage one or more machine learning models) to generate answers to an input question/to extract responses to questions]);
a communication interface configured to receive, via a user interface, a user input indicating a question; (see, e.g., paragraph 39, “To illustrate management of the responses, the response manager 122 may be configured to receive input from the entity device 130 that indicates a question to be answered by the virtual assistant application … In some implementations, the response manager 122 may be configured to cause display of a graphical user interface (GUI) at the entity device 130 to facilitate the above-described operations, such as by displaying questions that the virtual assistant application 110 is configured to answer for user selection, displaying the retrieved responses corresponding to a selected question” [i.e., (communication interface/graphical user interface (GUI)) (receiving a user input indicating a question/receive input that indicates a question to be answered)]);
a memory storing a plurality of processor-executable instructions; (see, e.g., paragraph 11, “The system includes at least one memory storing instructions and one or more processors coupled to the at least one memory” [i.e., (memory storing a plurality of instructions/memory storing instructions) both also mention processor executable instructions]);
and one or more processors executing the instructions to perform operations comprising: (see, e.g., paragraph 11, “The one or more processors are configured to execute the instructions to cause the one or more processors to receive an indication of one or more MDUs for which the virtual assistant is configured to provide responses to questions” [i.e., (one or more processors executing instructions to perform operations/one or more processors execute the instructions to cause…)]);
selecting, by a retrieval model at a server, one or more source documents based on the question (see, e.g., paragraphs 73 and 88, “or a device that is coupled to or accessible to the server 102 via the one or more networks 160) a second set of one or more ML models (referred to herein as “the second ML models 422) that are configured to extract one or more responses from input documents based on domain-specific questions” and “Thus, the UI 500 enables a user to select an input document for use in generating responses to be used by a virtual assistant” [i.e., by a retrieval model at a server/or a device (e.g., second ML model – which evaluates responses akin to a retrieval model) that is coupled to or accessible to the server, selecting one or more source documents based on the question/extracting (by selecting the input document) one or more responses from input documents based on questions]);
generating, by a first language model, a respective answer from an input combining the question and a respective source document from the one or more source document; (see, e.g., paragraphs 75 and 77, “one or more natural language processing (NLP) operations on the text to generate feature data, one or more automatic speech recognition (ASR) operations if the document 441 is in an audio-based format, other processing operations, or the like, to convert the document 441 into a format that is useable by the response generator 420 or the second ML models 422” and “The response generator 420 may extract one or more candidate responses 406 from the document 441 based on the questions 404” [i.e., (generating an answer by a language model/ML model that performs natural language processing operations to generate feature data) by combining the question and one or more source documents/by converting the document into a format usable by response generator which extracts responses from the document based on the questions]),
generating a respective indicator associated with the respective answer indicating a quality of the respective answer (see, e.g., paragraphs 78 and 79, “to configure the second ML models 422 to extract answers to the questions 404 even though the particular responses may be in different locations or have different formats in different types of leases. In this manner, the second ML models 422 may generate the candidate responses 406 instead of requiring the entity to manually enter information from each of the various leases” and “In some implementations, the second ML models 422 may include multiple different sets of ML models configured to extract responses from input documents based on domain-specific questions, and each set of ML models of the second ML models 422 may be configured to generate respective accuracy scores … Each of the responses may be associated with a respective accuracy score generated by the set of ML models ... The response generator 420 may select one or more of the candidate responses 406 based on the accuracy scores 408, and the selected responses may be output to the databases 132 as responses 444. In some implementations, the response generator 420 may select candidate responses that are associated with accuracy scores that satisfy (e.g., are greater than, or greater than or equal to) a threshold … the response generator 420 may select a subset of the candidate responses 406 having a highest accuracy score(s)” [i.e., respective indicator/accuracy score indicating quality of the respective answer/the greater than the accuracy score threshold the higher quality the answer]);
generating, via the user interface, a response to the user input by selecting an answer from generated answers based on respective indicators (see, e.g., paragraphs 78 and 79, “in order to configure the second ML models 422 to extract answers to the questions 404 even though the particular responses may be in different locations or have different formats in different types of leases. In this manner, the second ML models 422 may generate the candidate responses 406 instead of requiring the entity to manually enter information from each of the various leases” and “In some implementations, the response generator 420 may select candidate responses that are associated with accuracy scores that satisfy (e.g., are greater than, or greater than or equal to) a threshold. For example, if the first accuracy score and the third accuracy score of the accuracy scores 408 satisfy a threshold, the responses 444 may include the first responses and the third responses of the candidate responses 406. In some other implementations, the response generator 420 may select a subset of the candidate responses 406 having a highest accuracy score(s)” [i.e., responses to user input are generated/ candidate responses are generated, by selecting an answer based on respective indicators/responses that are associated with accuracy scores that satisfy a threshold - response generator selects response with highest accuracy score]).
Regarding claim 10, as discussed above, Mandry discloses the system of claim 9.
Mandry further discloses wherein the respective indicator is generated by a second language model based on an input combining the respective answer and the question (see, e.g., paragraphs 78 and 79, “The second ML models 422 may be trained using training data based on leases associated with different terms, states, lessee types, etc., in order to configure the second ML models 422 to extract answers to the questions 404 even though the particular responses may be in different locations or have different formats in different types of leases” and “In some implementations, the second ML models 422 may include multiple different sets of ML models configured to extract responses from input documents based on domain-specific questions, and each set of ML models of the second ML models 422 may be configured to generate respective accuracy scores. For example, the candidate responses 406 may include first responses generated by a first set of ML models of the second ML models 422, second responses generated by a second set of ML models of the second ML models 422, and third responses generated by a third set of ML models of the second ML models 422. Each of the responses may be associated with a respective accuracy score generated by the set of ML models” [e.g., (Second language model/ second machine learning model) uses (answers/training data from first model) to extract “new” answers to questions combined with (training data/”old” answers)]).
Regarding claim 13, as discussed above, Mandry discloses the system of claim 1.
Mandry further discloses wherein the respective answer is generated by the first language model further based on a prompt that contains a demonstration generating a summary of the respective source document, based on which an answer is generated (see, e.g., paragraph 102, “In some implementations, the one or more ML models are further configured to generate summaries of the responses based on the input documents” [e.g., (language model generates an answer based on a summary of a source document/ML model generates summaries on the input documents based on the (responses/answers)]).
Regarding independent claim 17, Mandry discloses the invention as claimed including A non-transitory processor-readable medium storing a plurality of processor-executable instructions for generating an answer to an input question using one or more neural network models, the instructions being executed by one or more processors to perform operations comprising: (see, e.g., paragraphs 12 and 13, “a computer program product is described that includes a computer-readable storage device, such as a non-transitory computer-readable storage medium, that includes instructions that, when executed by one or more processors, cause the one or more processors to perform operations” and “The method also includes providing, by the one or more processors, the document and one or more domain-specific questions associated with a domain corresponding to the document to one or more machine learning (ML) models to generate one or more responses and one or more training phrases associated with the one or more domain-specific questions, the one or more responses, or a combination thereof” [i.e., (A non-transitory processor-readable medium storing instructions/non-transitory computer-readable storage medium that includes instructions) (used for generating answer to input question of a neural network model/generating one or more responses to one or more questions via a machine learning model) (instructions being executed by processor to perform operations/instructions that when executed by one or more processors cause processors to perform operations)]);
receiving, via a user interface, a user input indicating a question; (see, e.g., paragraph 39, “may be configured to cause display of a graphical user interface (GUI) at the entity device 130 to facilitate the above-described operations, such as by displaying questions that the virtual assistant application 110 is configured to answer for user selection” [i.e., via a user interface/graphical user interface, receiving user input of a question/ displaying questions to answer for user selection]);
selecting, by a retrieval model at a server, one or more source documents based on the question (see, e.g., paragraphs 73 and 88, “or a device that is coupled to or accessible to the server 102 via the one or more networks 160) a second set of one or more ML models (referred to herein as “the second ML models 422) that are configured to extract one or more responses from input documents based on domain-specific questions” and “Thus, the UI 500 enables a user to select an input document for use in generating responses to be used by a virtual assistant” [i.e., by a retrieval model at a server/or a device (e.g., second ML model – which evaluates responses akin to a retrieval model) that is coupled to or accessible to the server, selecting one or more source documents based on the question/extracting (by selecting the input document) one or more responses from input documents based on questions]);
generating, by a first language model, a respective answer from an input combining the question and a respective source document from the one or more source document (see, e.g., paragraphs 75 and 77, “one or more natural language processing (NLP) operations on the text to generate feature data, one or more automatic speech recognition (ASR) operations if the document 441 is in an audio-based format, other processing operations, or the like, to convert the document 441 into a format that is useable by the response generator 420 or the second ML models 422” and “The response generator 420 may extract one or more candidate responses 406 from the document 441 based on the questions 404” [i.e., generating an answer by a language model/ML model that performs natural language processing operations to generate feature data by combining the question and one or more source documents/by converting the document into a format usable by response generator which extracts responses from the document based on the questions]);
generating a respective indicator associated with the respective answer indicating a quality of the respective answer (see, e.g., paragraphs 78 and 79, “in order to configure the second ML models 422 to extract answers to the questions 404 even though the particular responses may be in different locations or have different formats in different types of leases. In this manner, the second ML models 422 may generate the candidate responses 406 instead of requiring the entity to manually enter information from each of the various leases” and “In some implementations, the second ML models 422 may include multiple different sets of ML models configured to extract responses from input documents based on domain-specific questions, and each set of ML models of the second ML models 422 may be configured to generate respective accuracy scores … Each of the responses may be associated with a respective accuracy score generated by the set of ML models ... The response generator 420 may select one or more of the candidate responses 406 based on the accuracy scores 408, and the selected responses may be output to the databases 132 as responses 444. In some implementations, the response generator 420 may select candidate responses that are associated with accuracy scores that satisfy (e.g., are greater than, or greater than or equal to) a threshold … the response generator 420 may select a subset of the candidate responses 406 having a highest accuracy score(s)” [i.e., respective indicator/accuracy score indicating quality of the respective answer/the greater than the accuracy score threshold the higher quality the answer]);
generating, via the user interface, a response to the user input by selecting an answer from generated answers based on respective indicators (see, e.g., paragraphs 78 and 79, “in order to configure the second ML models 422 to extract answers to the questions 404 even though the particular responses may be in different locations or have different formats in different types of leases. In this manner, the second ML models 422 may generate the candidate responses 406 instead of requiring the entity to manually enter information from each of the various leases” and “In some implementations, the response generator 420 may select candidate responses that are associated with accuracy scores that satisfy (e.g., are greater than, or greater than or equal to) a threshold. For example, if the first accuracy score and the third accuracy score of the accuracy scores 408 satisfy a threshold, the responses 444 may include the first responses and the third responses of the candidate responses 406. In some other implementations, the response generator 420 may select a subset of the candidate responses 406 having a highest accuracy score(s)” [i.e., responses to user input are generated/ candidate responses are generated, by selecting an answer based on respective indicators/responses that are associated with accuracy scores that satisfy a threshold - response generator selects response with highest accuracy score]).
Regarding claim 20, as discussed above, Mandry discloses the medium of claim 17.
Mandry further discloses wherein the respective answer is generated by the first language model further based on a prompt that contains a demonstration generating a summary of the respective source document, based on which an answer is generated (see, e.g., paragraph 102, “In some implementations, the one or more ML models are further configured to generate summaries of the responses based on the input documents” [e.g., (language model generates an answer based on a summary of a source document/ML model generates summaries on the input documents based on the (responses/answers)]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3-4, 11-12 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Mandry as applied to claims 1, 9 and 17above in view of Siva et al. (U.S. Publication No. 20190371303, hereinafter “Siva”).
Regarding claim 3, as discussed above, Mandry discloses the method of claim 1. Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose wherein the respective answer is generated by the first language model further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information to answer the question.
In the same field, analogous art Siva teaches wherein the respective answer is generated by the first language model further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information9 to answer the question (see, paragraph 63, “a negative label to the other answer that is determined to be semantically irrelevant with respect to the question (506). The server 108 generates a machine learning model 312 based at least in part on the question, the positive label assigned to the answer that is semantically relevant, and the negative label assigned to the other answer that is semantically irrelevant (508)” [i.e., (negative answer contains insufficient information/negative labelled answer is irrelevant) is generated by language model/ML/machine learning model based on prompt/question]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Siva so that “a negative label to the other answer that is determined to be semantically irrelevant with respect to the question” (see Siva, e.g., paragraph 63). Doing so would have allowed Mandry to use Siva‘s positive and negative labelled answer to determine whether semantically relevant answer can be provided with respect toa subsequent question as suggested by Siva (see Siva, Abstract).
Regarding claim 4, as discussed above, Mandry discloses the method of claim 1.
Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose wherein the respective answer is generated by the first language model further based on a prompt that contains a demonstration differentiating irrelevant source documents from relevant source documents that contain sufficient information to answer the question.10
In the same field, analogous art Siva teaches wherein the respective answer is generated by the first language model further based on a prompt that contains a demonstration differentiating irrelevant source documents from relevant source documents that contain sufficient information to answer the question (see, paragraph 63, “The server 108 assigns a positive label to the answer that is determined to be semantically relevant with respect to the question, and a negative label to the other answer that is determined to be semantically irrelevant with respect to the question (506). The server 108 generates a machine learning model 312 based at least in part on the question, the positive label assigned to the answer that is semantically relevant, and the negative label assigned to the other answer that is semantically irrelevant (508)” [i.e., irrelevant source and relevant source are differentiated as negative and positive labeled answers, while (language model is based on a prompt/machine learning model is based on the question)]).11
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Siva so that “a positive label to the answer that is determined to be semantically relevant with respect to the question, and a negative label to the other answer that is determined to be semantically irrelevant with respect to the question” (see Siva, e.g., paragraph 63). Doing so would have allowed Mandry to use Siva‘s positive and negative labelled answer to determine whether semantically relevant answer can be provided with respect to a subsequent question as suggested by Siva (see Siva, Abstract).
Regarding claim 11, as discussed above, Mandry discloses the system of claim 9. Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose wherein the respective answer is generated by the first language model further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information to answer the question.
In the same field, analogous art Siva teaches wherein the respective answer is generated by the first language model further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information12 to answer the question (see, paragraph 63, “a negative label to the other answer that is determined to be semantically irrelevant with respect to the question (506). The server 108 generates a machine learning model 312 based at least in part on the question, the positive label assigned to the answer that is semantically relevant, and the negative label assigned to the other answer that is semantically irrelevant (508)” [i.e., (negative answer contains insufficient information/negative labelled answer is irrelevant) is generated by (language model/machine learning model) based on prompt/question]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Siva so that “a negative label to the other answer that is determined to be semantically irrelevant with respect to the question” (see Siva, e.g., paragraph 63). Doing so would have allowed Mandry to use Siva‘s positive and negative labelled answer to determine whether semantically relevant answer can be provided with respect toa subsequent question as suggested by Siva (see Siva, Abstract).
Regarding claim 12, as discussed above, Mandry discloses the system of claim 9.
Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose wherein the respective answer is generated by the first language model further based on a prompt that contains a demonstration differentiating irrelevant source documents from relevant source documents that contain sufficient information to answer the question.
In the same field, analogous art Siva teaches wherein the respective answer is generated by the first language model further based on a prompt that contains a demonstration differentiating irrelevant source documents from relevant source documents that contain sufficient information to answer the question (see, paragraph 63, “The server 108 assigns a positive label to the answer that is determined to be semantically relevant with respect to the question, and a negative label to the other answer that is determined to be semantically irrelevant with respect to the question (506). The server 108 generates a machine learning model 312 based at least in part on the question, the positive label assigned to the answer that is semantically relevant, and the negative label assigned to the other answer that is semantically irrelevant (508)” [i.e., irrelevant source and relevant source are differentiated as negative and positive labeled answers, while (language model is based on a prompt/machine learning model is based on the question)]).13
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Siva so that “a positive label to the answer that is determined to be semantically relevant with respect to the question, and a negative label to the other answer that is determined to be semantically irrelevant with respect to the question” (see Siva, e.g., paragraph 63). Doing so would have allowed Mandry to use Siva‘s positive and negative labelled answer to determine whether semantically relevant answer can be provided with respect toa subsequent question as suggested by Siva (see Siva, Abstract).
Regarding claim 18, as discussed above, Mandry discloses the medium of claim 17. Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose wherein the respective answer is generated by the first language model further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information to answer the question.
In the same field, analogous art Siva teaches wherein the respective answer is generated by the first language model further based on a prompt that guides the first language model to generate a negative answer when the first language model determines that the respective source document contains insufficient information14 to answer the question (see, paragraph 63, “a negative label to the other answer that is determined to be semantically irrelevant with respect to the question (506). The server 108 generates a machine learning model 312 based at least in part on the question, the positive label assigned to the answer that is semantically relevant, and the negative label assigned to the other answer that is semantically irrelevant (508)” [i.e., (negative answer contains insufficient information/negative labelled answer is irrelevant) is generated by (language model/machine learning model) based on prompt/question]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Siva so that “a negative label to the other answer that is determined to be semantically irrelevant with respect to the question” (see Siva, e.g., paragraph 63). Doing so would have allowed Mandry to use Siva‘s positive and negative labelled answer to determine whether semantically relevant answer can be provided with respect toa subsequent question as suggested by Siva (see Siva, Abstract).
Claims 6 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Mandry as applied to claims 1 and 9 above in view of Lindgren et al. (U.S. Publication No. 20230153700, hereinafter “Lindgren”).
Regarding claim 6, as discussed above, Mandry discloses the method of claim 1.
Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose generating, by the first language model, an initial answer from an input combining the question and a concatenation of the one or more source documents prior to generating respective answers based on respective source documents independently;
and generating, by the first language model, the respective answer when the initial answer indicates the concatenation of the one or more source documents contain insufficient information to answer the question.
In the same field, analogous art Lindgren teaches generating, by the first language model, an initial answer from an input combining the question and a concatenation of the one or more source documents prior to generating respective answers based on respective source documents independently (see, paragraph 32, “More particularly, embedding models (e.g., factorized models, such as two tower neural network models) are widely used for scoring (query, document) pairs in information retrieval tasks. These models are typically trained by optimizing the model parameters to score relevant “positive” pairs higher than the irrelevant “negative” ones. While a large set of negatives typically improves the model performance, limited computation and memory budgets place constraints on the number of negatives used during training” [i.e., language model generating an answer based on the question and source document/neural network models generates a score based on query and document pairs of information]),
and generating, by the first language model, the respective answer when the initial answer indicates the concatenation of the one or more source documents contain insufficient information15 to answer the question (see, paragraph 32, “embedding models (e.g., factorized models, such as two tower neural network models) are widely used for scoring (query, document) pairs in information retrieval tasks. These models are typically trained by optimizing the model parameters to score relevant “positive” pairs higher than the irrelevant “negative” ones. While a large set of negatives typically improves the model performance, limited computation and memory budgets place constraints on the number of negatives used during training” [i.e., answer is generated by language model based on document with insufficient/irrelevant information/score is generated by a neural network model based on document with a large set of negatives (a large set of irrelevant documents)]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Lindgren to provide neural network models useable for scoring (query, document) pairs in information retrieval tasks and because using a large set of (negatives/irrelevant documents) typically improves the model performance in environments with limited computation and memory budgets that place constraints on the number of (negatives/irrelevant documents) used during training (see Lindgren, e.g., paragraph 32). Doing so would have allowed Mandry to use Lindgren‘s negative sampling technique to accelerate training by using cache, where by the cache improves speed and computational efficiency for performance of the model, as suggested by Lindgren (see Lindgren, paragraphs 31 and 33).
Regarding claim 14, as discussed above, Lindgren discloses the system of claim 9.
Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose generating, by the first language model, an initial answer from an input combining the question and a concatenation of the one or more source documents prior to generating respective answers based on respective source documents independently;
and generating, by the first language model, the respective answer when the initial answer indicates the concatenation of the one or more source documents contain insufficient information to answer the question.
In the same field, analogous art Lindgren teaches generating, by the first language model, an initial answer from an input combining the question and a concatenation of the one or more source documents prior to generating respective answers based on respective source documents independently (see, paragraph 32, “embedding models (e.g., factorized models, such as two tower neural network models) are widely used for scoring (query, document) pairs in information retrieval tasks. These models are typically trained by optimizing the model parameters to score relevant “positive” pairs higher than the irrelevant “negative” ones. While a large set of negatives typically improves the model performance, limited computation and memory budgets place constraints on the number of negatives used during training” [i.e., language model generating an answer based on the question and source document/neural network models generates a score based on query and document pairs of information]),
and generating, by the first language model, the respective answer when the initial answer indicates the concatenation of the one or more source documents contain insufficient information16 to answer the question (see, paragraph 32, “embedding models (e.g., factorized models, such as two tower neural network models) are widely used for scoring (query, document) pairs in information retrieval tasks. These models are typically trained by optimizing the model parameters to score relevant “positive” pairs higher than the irrelevant “negative” ones. While a large set of negatives typically improves the model performance, limited computation and memory budgets place constraints on the number of negatives used during training” [i.e., answer is generated by language model based on document with insufficient (irrelevant) information/score is generated by a neural network model based on document with a large set of negatives (a large set of irrelevant documents)]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Lindgren to provide neural network models useable for scoring (query, document) pairs in information retrieval tasks and because using a large set of (negatives/irrelevant documents) typically improves the model performance in environments with limited computation and memory budgets that place constraints on the number of (negatives/irrelevant documents) used during training (see Lindgren, e.g., paragraph 32). Doing so would have allowed Mandry to use Lindgren‘s negative sampling technique to accelerate training by using cache, where by the cache improves speed and computational efficiency for performance of the model, as suggested by Lindgren (see Lindgren, paragraphs 31 and 33).
Claims 7-8 and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Mandry as applied to claim 1 and 9 above and further in view of Chu - Carroll et al. (U.S. Publication No. 20120078895, hereinafter “Chu”).
Regarding claim 7, as discussed above, Mandry discloses the method of claim 1.
Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose removing at least one source documents from the one or more source documents based on respective indicators.
In the same field, analogous art Lindgren teaches removing at least one source documents from the one or more source documents based on respective indicators (see, paragraph 38, “Alternatively, text nuggets can be arranged in a different order, e.g. the order in which they appear in the retrieved documents. A filter eliminates lexical redundancy by removing nuggets if a given percentage of their tokens (e.g. 95%) are subsumed by higher scoring nuggets or the seed. In addition, nuggets are dropped if their relevance scores are below an absolute threshold, or if the total character length of all nuggets exceeds a threshold that is relative to the length of the seed” [i.e., removing at least one source document based on indicator detecting irrelevancy/removing text nuggets from a document based on irrelevancy according to a score]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Chu so that nuggets are dropped if their relevance scores are below an absolute threshold, or if the total character length of all nuggets exceeds a threshold that is relative to the length of the seed (see Chu, e.g., paragraph 38). Doing so would have allowed Mandry to use Chu‘s remining nuggets for a new "pseudo-document", which can be processed (e.g. indexed and searched with an information retrieval system) along with the original seed, as suggested by Chu (see Chu, paragraph 38).
Regarding claim 8, as discussed above, Mandry in view of Lindgren teaches the method of claim 7.
Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose generating, by the first language model, a final answer using an input combining unremoved source documents and corresponding answers.
In the same field, analogous art Lindgren teaches generating, by the first language model, a final answer using an input combining unremoved source documents and corresponding answers (see, paragraphs 25 and 38, “to answer questions. The corpus is created by starting with existing data, automatically identifying other documents that may have relevant data and automatically retrieving "nuggets" of content from those other documents … These nuggets are then included in the corpus if they are determined to be relevant based on a classifier that takes into account a list of features, to be described in greater detail below. In one aspect, a statistical classifier is trained using a machine learning algorithm…” and “A filter eliminates lexical redundancy by removing nuggets if a given percentage of their tokens (e.g. 95%) are subsumed by higher scoring nuggets or the seed … Reasonable thresholds can be determined by inspecting a sample of ranked nuggets. The remaining nuggets are compiled into a new "pseudo-document", which can be processed (e.g. indexed and searched with an information retrieval system) along with the original seed” [i.e., (language model/machine learning algorithm) generating an answer combining (remaining nuggets in documents/unremoved source documents) with the answer/original seed]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Chu so that the remaining nuggets are compiled into a new "pseudo-document", which can be processed (e.g. indexed and searched with an information retrieval system) along with the original seed (see Chu, e.g., paragraph 38). Doing so would have allowed Mandry to use Chu‘s machine learning algorithm to feed in the pseudo-document so as to retrieve the type of information that is of interest, as suggested by Chu (see Chu, paragraph 39).
Regarding claim 15, as discussed above, Mandry discloses the system of claim 9.
Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose removing at least one source documents from the one or more source documents based on respective indicators.
In the same field, analogous art Lindgren teaches removing at least one source documents from the one or more source documents based on respective indicators (see, paragraph 38, “Alternatively, text nuggets can be arranged in a different order, e.g. the order in which they appear in the retrieved documents. A filter eliminates lexical redundancy by removing nuggets if a given percentage of their tokens (e.g. 95%) are subsumed by higher scoring nuggets or the seed. In addition, nuggets are dropped if their relevance scores are below an absolute threshold, or if the total character length of all nuggets exceeds a threshold that is relative to the length of the seed” [i.e., removing at least one source document based on indicator detecting irrelevancy/removing text nuggets from a document based on irrelevancy according to a score]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Chu so that nuggets are dropped if their relevance scores are below an absolute threshold, or if the total character length of all nuggets exceeds a threshold that is relative to the length of the seed (see Chu, e.g., paragraph 38). Doing so would have allowed Mandry to use Chu‘s remining nuggets for a new "pseudo-document", which can be processed (e.g. indexed and searched with an information retrieval system) along with the original seed, as suggested by Chu (see Chu, paragraph 38).
Regarding claim 16, as discussed above, Mandry in view of Lindgren teaches the system of claim 15.
Although Mandry substantially discloses the claimed invention, Mandry does not explicitly disclose generating, by the first language model, a final answer using an input combining unremoved source documents and corresponding answers.
In the same field, analogous art Lindgren teaches generating, by the first language model, a final answer using an input combining unremoved source documents and corresponding answers (see, paragraphs 25 and 38, “to answer questions. The corpus is created by starting with existing data, automatically identifying other documents that may have relevant data and automatically retrieving "nuggets" of content from those other documents … These nuggets are then included in the corpus if they are determined to be relevant based on a classifier that takes into account a list of features, to be described in greater detail below. In one aspect, a statistical classifier is trained using a machine learning algorithm…” and “A filter eliminates lexical redundancy by removing nuggets if a given percentage of their tokens (e.g. 95%) are subsumed by higher scoring nuggets or the seed … Reasonable thresholds can be determined by inspecting a sample of ranked nuggets. The remaining nuggets are compiled into a new "pseudo-document", which can be processed (e.g. indexed and searched with an information retrieval system) along with the original seed” [i.e., (language model/machine learning algorithm) generating an answer combining (remaining nuggets in documents/unremoved source documents) with the answer/original seed]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Mandry to incorporate the teachings of Chu so that the remaining nuggets are compiled into a new "pseudo-document", which can be processed (e.g. indexed and searched with an information retrieval system) along with the original seed (see Chu, e.g., paragraph 38). Doing so would have allowed Mandry to use Chu‘s machine learning algorithm to feed in the pseudo-document so as to retrieve the type of information that is of interest, as suggested by Chu (see Chu, paragraph 39).
Conclusion
When responding to this office action, Applicant is advised to clearly point out the patentable novelty which he or she thinks the claims present, in view of the state of the art disclosed by the reference cited or the objections made. He or she must also show how the amendments avoid such references or objections See 37 CFR 1.111 (c).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to STEVEN PENG whose telephone number is (571)270-0897. The examiner can normally be reached Monday - Friday 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamran Afshar can be reached at (571) 272-7796. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/STEVEN PENG/Examiner, Art Unit 2125
/KAMRAN AFSHAR/Supervisory Patent Examiner, Art Unit 2125
1 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
2 As indicated in the section 112(b) rejection of this claim above, a) irrelevant, b) relevant, and c) sufficient have been interpreted as a) any source documents, data objects or files that are not deemed to pertinent to or relevant for the claimed purpose, b) any source documents, data objects or files that are deemed to be pertinent to or relevant for the claimed purpose, c) any information or data that is sufficient, pertinent to, or useful for the claimed purpose
3 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
4 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
5 As indicated in the section 112(b) rejection of this claim above, a) irrelevant, b) relevant, and c) sufficient have been interpreted as a) any source documents, data objects or files that are not deemed to pertinent to or relevant for the claimed purpose, b) any source documents, data objects or files that are deemed to be pertinent to or relevant for the claimed purpose, c) any information or data that is sufficient, pertinent to, or useful for the claimed purpose
6 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
7 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
8 As indicated in the section 112(b) rejection of this claim above, a) irrelevant, b) relevant, and c) sufficient have been interpreted as a) any source documents, data objects or files that are not deemed to pertinent to or relevant for the claimed purpose, b) any source documents, data objects or files that are deemed to be pertinent to or relevant for the claimed purpose, c) any information or data that is sufficient, pertinent to, or useful for the claimed purpose
9 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
10 As indicated in the section 112(b) rejection of this claim above, sufficient information has been interpreted as any information or data that is sufficient, pertinent to, or useful for the claimed purpose
11 As indicated in the section 112(b) rejection of this claim above, a) irrelevant, b) sufficient have been interpreted as a) any source documents, data objects or files that are not deemed to be pertinent to or relevant for the claimed purpose, b) any information or data that is sufficient, pertinent to, or useful for the claimed purpose
12 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
13 As indicated in the section 112(b) rejection of this claim above, a) irrelevant, b) sufficient have been interpreted as a) any source documents, data objects or files that are not deemed to be pertinent to or relevant for the claimed purpose, b) any information or data that is sufficient, pertinent to, or useful for the claimed purpose
14 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
15 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
16 As indicated in the section 112(b) rejection of this claim above, insufficient information has been interpreted as any information or data that is not sufficient, pertinent to, or useful for the claimed purpose
Prosecution Insights