Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
New corrected drawings in compliance with 37 CFR 1.121(d) are required in this application because the text in multiple Figures (See at least Figures 4-9) is too small and/or in grey and therefore is not possible to read or accurately reproduce the text in the identified Figures. Applicant is advised to employ the services of a competent patent draftsperson outside the Office, as the U.S. Patent and Trademark Office no longer prepares new drawings. The corrected drawings are required in reply to the Office action to avoid abandonment of the application. The requirement for corrected drawings will not be held in abeyance.
Claim Objections
Claims 1-13 are objected to because of the following informalities: Claim 1 recites “extensibility of the SmartRFP software taool”. Please amend “taool” to “tool”.
Claim 3 recites “wherein the GenAI”. Please amend “GenAI” to read “GenAI engine”.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
Referring to claim 1,
SmartRFP software tool, GenAI engine, API component, persistent component, open AI component, and a downstream review component.
Referring to claim 2,
SmartRFP software tool
Referring to claim 3,
GenAI
Referring to claim 6,
Content queue mechanism
Referring to claim 7,
SmartRFP software tool
Referring to claim 9,
GenAI engine
Referring to claim 10,
Feedback mechanism
Referring to claim 11,
Hallucination prevention module
Referring to claim 13,
Blind spot detection module
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof. See paragraphs 70, 78, 87 104-105, 111, 120-121, 250, 271, 291, 305, 342, 354, and 372.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-19 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1 and 14 contains the trademark/trade name “Open AI”. Where a trademark or trade name is used in a claim as a limitation to identify or describe a particular material or product, the claim does not comply with the requirements of 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph. See Ex parte Simpson, 218 USPQ 1020 (Bd. App. 1982). The claim scope is uncertain since the trademark or trade name cannot be used properly to identify any particular material or product. A trademark or trade name is used to identify a source of goods, and not the goods themselves. Thus, a trademark or trade name does not identify or describe the goods associated with the trademark or trade name. In the present case, the trademark/trade name is used to identify/describe a component and services utilized and, accordingly, the identification/description is indefinite.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-13 recite a system (machine), Claims 14-19 recite a method (process) and Claim 20 recites a method (process) therefore fall into a statutory category. The Examiner is interpreting that the method of claim 14 performs the functions of the system as claimed in the claim 1 for Examination purposes.
Step 2A – Prong 1 (Is a Judicial Exception Recited?):
Referring to claims 1-20 the claims recite a manner of organizing the RFP process based on the analysis of collected information, which under its broadest reasonable interpretation, covers concepts under the Certain Methods of Organizing Human Activities and Mental Processes grouping of abstract ideas.
The abstract idea portion of the claims is as follows:
(Claim 1)
[a communication interface configured for] receiving an RFP document as input;
[a SmartRFP software tool configured for] processing the RFP document to generate an RFP response document, [wherein the SmartRFP software tool comprises: a user interface communicatively coupled to the communication interface to receive the RFP document and manage an RFP process;
[a generative artificial intelligence (GenAI) engine configured to] extract content from the RFP document, identify questions in the RFP document, and generate contextually accurate answers based on the client persona and identified questions using an authoritative repository;
[wherein the GenAI] generates the answers by decomposing the identified questions into multiple sub-questions, rephrasing each sub-question, and classifying each rephrased sub-question into one or more topics;
[ and a content management system for] managing and building the content and for storing and retrieving the content [from the authoritative repository in collaboration with the GenAI engine];
[and an application programming interface (API) component configured to facilitate data exchange and integration with other applications and to support interoperability and extensibility of the SmartRFP software tAool];
[a persistent component communicatively configured to] store the data [in one or more persistent databases];
[an open AI component configured to integrate AI capabilities with the SmartRFP software tool to enhance processing and analysis of the RFP document];
and [a downstream review component configured to] review, finalize, and approve the RFP response document downstream in the RFP process;
[a dashboard] calculating and reporting performance and data insights.
(Claim 20)
receiving an RFP document as input [at a generative artificial intelligence (GenAI) module];
identifying questions in the RFP document;
decomposing the identified questions into sub-questions;
rephrasing the sub-questions;
classifying each rephrased sub-questions into one or more predefined topics;
expanding each rephrased sub-question into multiple queries to cover different aspects of the sub-questions;
embedding each query to create vector representation for performing similarity searches to calculate a similarity score;
retrieving relevant documents for each query based on the similarity score;
re-ranking, [using a generative pre-trained transformer], the relevant documents to identify most relevant documents based on the similarity score;
processing the most relevant documents to generate [AI] answers for each sub-question;
and compiling and merging the generated [AI] answers to generate a comprehensive response for each sub-question.
Where the portions not bracketed recite the abstract idea
Here the claims recite concepts covering Certain Methods of Organizing Activity, in particular commercial or legal interactions (business relations or contract management) or managing personal behavior or interactions between people (including following rules or instructions) and Mental Processing (including an opinion, observation, evaluation, or judgement) but for the recitation of generic computer components. In the present application concepts reciting a manner of organizing the RFP process based on the analysis of collected information. (See paragraphs 3-7).
If a claim limitation, under its broadest reasonable interpretation, covers concepts capable of being performed in commercial or legal interactions (business relations or contract management) managing personal behavior or interactions between people (including following rules or instructions) it falls under the Certain Methods of Organizing Human Activity grouping of abstract ideas. See MPEP 2106.04. Further if a claim limitation, under its broadest reasonable interpretation, covers concepts capable of being performed via the human mind or pen and paper (including an opinion, observation, evaluation, or judgement) it falls under the Mental Processes grouping of abstract ideas. See Id.
Accordingly, the claims recite an abstract idea.
Step 2A-Prong 2 (Is the Exception Integrated into a Practical Application?):
The examiner views the following as the additional elements:
A communication interface. (See paragraphs 72)
A SmartRFP software tool. (See paragraph 70, 121, and 372)
A user interface. (See paragraph 72)
A GenAI engine. (See paragraphs 87, 121, 250, and 372)
A content management system. (See paragraph 116)
Authoritative repository. (See paragraph 118)
A persistent component. (See paragraphs 104-105, 121, and 372)
An API component. (See paragraph 78, 121, and 372)
One or more persistent databases. (See paragraph 105)
An open AI component. (See paragraphs 111, 121, and 372)
A downstream review component. (See paragraphs 120-121 and 372)
A dashboard. (See paragraph 82)
A GenAI module. (See paragraphs 87, 121, 250, and 372)
A generative pre-trained transformer. (See paragraphs 97 and 162)
These additional elements are recited at a high-level of generality such that they act to merely “apply” the abstract idea using generic computing components and do not integrate the abstract idea into a practical application. (See MPEP 2106.05 (f))
Referring to “facilitate data exchange and integration with other applications and to support interoperability and extensibility of the SmartRFP software tAool” and “integrate AI capabilities with the SmartRFP software tool to enhance processing and analysis of the RFP document” the examiner views these steps as results-oriented solution lacking details and therefore equivalent to mere instructions to apply the abstract idea using generic computing components. (See MPEP 2106.05 (f) and paragraphs 78 and 114).
The combination of these additional elements and/or results oriented steps are no more than mere instructions to apply the exception using generic computing components. (See Id.) Accordingly, even in combination these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Step 2B (Does the claim recite additional elements that amount to Significantly More than the Judicial Exception?):
As noted above, the claims as a whole merely describes a method and system that generally “apply” the concepts discussed in prong 1 above. (See MPEP 2106.05 f (II)) In particular applicant has recited the computing components at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computer components. As the court stated in TLI Communications v. LLC v. AV Automotive LLC, 823 F.3d 607, 613 (Fed. Cir. 2016) merely invoking generic computing components or machinery that perform their functions in their ordinary capacity to facilitate the abstract idea are mere instructions to implement the abstract idea within a computing environment and does not add significantly more to the abstract idea. Accordingly, these additional computer components do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Therefore, even when viewed as a whole, nothing in the claim adds significantly more (i.e. an inventive concept) to the abstract idea and as a result the claim is not patent eligible.
Dependent claim 2 recites the additional element of the generic Smart RFP tool (See paragraph 70, 121, and 372) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Additionally, the claim recites the results-oriented solution steps of “utilize embeddings techniques and machine learning models for processing the RFP document.” (See paragraph 95 and 153) and are therefore viewed as equivalent as mere instructions for implementing the abstract idea using generic computing components which does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 2 is considered to be patent ineligible.
Dependent claim 3 recites the additional element of the generic GenAI (See paragraphs 87, 121, 250 and 372) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Additionally, the claim recites the results-oriented solution steps of “implement parallel processing of PDF files to read all pages of a PDF document simultaneously using parallel processing algorithms for content extraction.” (See paragraphs 146-147) and are therefore viewed as equivalent as mere instructions for implementing the abstract idea using generic computing components which does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 3 is considered to be patent ineligible.
Dependent claim 4 further defines the abstract idea as identified. Additionally, the claim recites the additional elements of the generic content management system (See paragraph 116) and authoritative repository (See paragraph 118) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 4 is considered to be patent ineligible.
Dependent claims 5 and 18 further define the abstract idea as identified. Additionally, the claim recites the additional elements of the generic pipeline (See paragraph 52), one or more processors (See paragraph 373), and Smart RFP software tool (See paragraphs 70, 121, and 372) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 5 is considered to be patent ineligible.
Dependent claim 6 further define the abstract idea as identified. Additionally, the claim recites the additional elements of the generic content queue mechanism (See paragraphs 121, 342, 354 and 372) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 6 is considered to be patent ineligible.
Dependent claim 7 further define the abstract idea as identified. Additionally, the claim recites the additional elements of the generic smartRFP software tool (See paragraphs 70, 121, and 372) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 7 is considered to be patent ineligible.
Dependent claim 8 further defines the abstract idea as identified. Additionally, the claim recites the additional elements of the generic the one or more persistent databases (See paragraph 105), a vector database (See paragraph 107), a relational database (See paragraph 109), and a cloud storage service (See paragraph 110) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 8 is considered to be patent ineligible.
Dependent claim 9 further defines the abstract idea as identified. Additionally, the claim recites the additional elements of the generic the one or more GenAI engine (See paragraphs 87, 121, 250, and 372) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 9 is considered to be patent ineligible
Dependent claim 10 further defines the abstract idea as identified. Additionally, the claim recites the additional elements of the generic the one or more feedback mechanism (See paragraphs 121, 291, and 372) and workflow management system (364 and 388) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 10 is considered to be patent ineligible
Dependent claim 11 further defines the abstract idea as identified. Additionally, the claim recites the additional elements of the generic hallucination prevention module (See paragraph 121, 271, and 372) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 11 is considered to be patent ineligible.
Dependent claim 12 recites the additional element of the generic collaboration platform (See paragraph 387) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Additionally, the claim recites the results-oriented solution steps of “enabling simultaneous stakeholder input and tailored responses and for supporting simultaneous editing and commenting on the RFP documents by multiple users.” (See paragraph 387) and are therefore viewed as equivalent as mere instructions for implementing the abstract idea using generic computing components which does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 12 is considered to be patent ineligible.
Dependent claim 13 further defines the abstract idea as identified. Additionally, the claim recites the additional elements of the generic blind spot detection module (See paragraphs 121, 305, and 372) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 13 is considered to be patent ineligible.
Dependent claim 15 recites the results-oriented solution steps of “utilizing embeddings techniques and machine learning models for processing the RFP document” (See paragraphs 95 and 153) and are therefore viewed as equivalent as mere instructions for implementing the abstract idea using generic computing components which does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 15 is considered to be patent ineligible.
Dependent claim 16 recites the results-oriented solution steps of “implementing parallel processing of PDF files to read all pages of a PDF document simultaneously using parallel processing algorithms for content extraction.” (See paragraphs 146-147) and are therefore viewed as equivalent as mere instructions for implementing the abstract idea using generic computing components which does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 16 is considered to be patent ineligible.
Dependent claim 17 further defines the abstract idea as identified. Additionally, the claim recites the additional elements of the generic authoritative repository (See paragraph 118) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 17 is considered to be patent ineligible.
Dependent claim 19 further define the abstract idea as identified. Additionally, the claim recites the additional elements of the generic automated RFP processing system (See paragraph 69) and SmartRFP software tool (See paragraph 70) at a high-level of generality such that it amounts to no more than mere instructions to apply the exception using generic computing components and therefore does not integrate the abstract idea into a practical application or adds significantly more. Therefore claim 19 is considered to be patent ineligible.
In conclusion the claims do not provide an inventive concept, because the claims do not recite additional elements or a combination of elements that amount to significantly more than the judicial exception of the claims. There is no indication that the combination of elements improves the functioning of a computer or improves any other technology, and the collective functions merely provide conventional computer implementation. Therefore, whether taken individually or as an order combination, the claims are nonetheless rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 5-11, 14-16, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Patent No. 12,493,634) in view of Liu et al. (US Patent No. 11,379,670).
Referring to claims 1 and 14,
Chen which is directed to automated request for proposal and questionnaire processing using artificial intelligence, teaches
(Claim 14) A method for automating a request for proposal (RFP) process, comprising:
(Claim 1) A system for automating a request for proposal (RFP) process, comprising: a communication interface configured for receiving an RFP document as input; (Chen column 1 lines 15-23 teaching the present invention relates generally to the field of artificial intelligence and, more particularly, to an apparatus and method for automating the process of responding to Requests for Proposals (RFPs), Requests for Information (RFIs), Security Questionnaires (SQs), and other B2B questionnaires (collectively referred to as RFPs in this filing) using advanced data management and AI-based answer generation techniques. Chen column 7 lines 26-31 teaching FIG. 3 illustrates processing associated with the answer module 144. A user at customer machine 102 receives from the answer module 144 a user interface with prompts to specify a new project 300. The user uploads an RFP document and provides relevant metadata (e.g., customer name, due date, assignment details) 302.)
a SmartRFP software tool configured for processing the RFP document to generate an RFP response document, wherein the SmartRFP software tool comprises: (Chen column 7 lines 26-38 teaching FIG. 3 illustrates processing associated with the answer module 144. A user at customer machine 102 receives from the answer module 144 a user interface with prompts to specify a new project 300. The user uploads an RFP document and provides relevant metadata (e.g., customer name, due date, assignment details) 302. The answer module 144 analyzes the uploaded document to determine its structure and extracts questions 304. The answer module 144 evaluates whether it has sufficient confidence to extract questions automatically 306. If not (306—No), it prompts the user to manually extract or confirm questions and sections 308. The user is then prompted to confirm extracted questions and sections 310. Chen column 7 lines 54-65 teaching the answers are generated by sending prompts to a large language model 319. The answer module 144 evaluates the confidence of generated answers using multiple factors (e.g., number of sources used, recency of sources, token probability) 320. Data sources are then identified 322. That is, the vector database of the data management module 142 is queried to trace to original data sources to establish data lineage. Answers are displayed to the user along with confidence scores and source tracing information 324. Users can review, edit, and collaborate on the generated responses within the platform 326. Completed answers are then exported 328.)
a user interface communicatively coupled to the communication interface to receive the RFP document and manage an RFP process; (Chen column 2 lines 21-37 teaching the data management module 142 and answer module 144 automate the RFP response process using advanced data management techniques and artificial intelligence. The data management module 142 is capable of ingesting various types of customer data, including product documentation, sales materials, security policies, and competitor information. This module analyzes the imported data, deploys different strategies for “chunking” the data, and implements context string embedding techniques to optimize data storage and retrieval.
(8) The answer module 144 utilizes the processed data to create appropriate responses to RFP questions. This module employs sophisticated algorithms to analyze user inputs, determine document structure, extract relevant questions, and generate accurate answers using a combination of semantic matching, prompt engineering, and language model integration.
Chen column 7 lines 26-37 teaching FIG. 3 illustrates processing associated with the answer module 144. A user at customer machine 102 receives from the answer module 144 a user interface with prompts to specify a new project 300. The user uploads an RFP document and provides relevant metadata (e.g., customer name, due date, assignment details) 302. The answer module 144 analyzes the uploaded document to determine its structure and extracts questions 304. The answer module 144 evaluates whether it has sufficient confidence to extract questions automatically 306. If not (306—No), it prompts the user to manually extract or confirm questions and sections 308. The user is then prompted to confirm extracted questions and sections 310.)
a generative artificial intelligence (GenAI) engine configured to extract content from the RFP document, identify questions in the RFP document, and generate contextually accurate answers based on the client persona and identified questions using an authoritative repository; (Chen column 2 lines 21-37 teaching the data management module 142 and answer module 144 automate the RFP response process using advanced data management techniques and artificial intelligence. The data management module 142 is capable of ingesting various types of customer data, including product documentation, sales materials, security policies, and competitor information. This module analyzes the imported data, deploys different strategies for “chunking” the data, and implements context string embedding techniques to optimize data storage and retrieval.
The answer module 144 utilizes the processed data to create appropriate responses to RFP questions. This module employs sophisticated algorithms to analyze user inputs, determine document structure, extract relevant questions, and generate accurate answers using a combination of semantic matching, prompt engineering, and language model integration. Chen column 9 lines 1-55 teaching the steps that the system goes through to generate an answer for any one question is as follows: When we receive a question to autogenerate an answer for, we first go through our context finding step. The context finding step performs the following actions: Semantic search across our indexed embeddings (as described in 204 through 208). The system first generates an embedding on the question using the same embeddings model that was used to index the content in the indexing step described above. The system then performs a cosine similarity look up search to fetch the 10 closest embeddings. We then use the text associated with those embeddings as part of our context in subsequent steps. Another technique we use is called Hypothetical Document Embeddings (HyDE). Because embeddings can sometimes be in paragraph format, it can lead to a bad fit for questions, which tend to be shorter and pithier. To combat this, we leverage LLMs to generate hypothetical paragraph texts that could answer the question at hand. We then perform regular semantic matching with those hypothetical paragraphs. We also perform regular keyword and ranking searches, which leverage keywords in the question to match embedded documents. If we have matched against a Question-and-Answer from our library, and our semantic cosine similarity is extremely high (>0.95), we assume that the Question from our Question-and-Answer library is roughly the same question. In these cases, we return the Answer from our Question-and-Answer library verbatim. Otherwise, we then send the question, along with the semantically matched question through two metadata generating steps: The first is to leverage an LLM call to hypothesize the intent of the question based on the context available. For example, a question that asks: “What is the maximum focal length of the anterior x-ray sensor?” is likely looking for a direct, factual answer to that question. But a question that asks: “Please describe your company's ESG policy?” is likely looking for a longer, more prose heavy descriptive answer to the question. This intent is then leveraged in a latter step of our autogeneration process. The second is to understand the verbosity of the answer required. Again, we leverage an LLM prompt. In the previous example, the first question is looking for a short, concise answer to the question since it is a factual response, but the second is looking for something longer and more verbose. After we have computed the metadata, we then perform the actual autogeneration composition step to generate the actual answer.)
wherein the GenAI generates the answers by decomposing the identified questions into multiple sub-questions, rephrasing each sub-question, and classifying each rephrased sub-question into one or more topics; ((Chen column 6 line 40 to column 7 line 5 teaching the parsing system is a combination of a heuristic-based or OCR-based reader that extracts the raw text from the file. The raw text is then analyzed, organized, and chunked up (separated into smaller, organized pieces of text) based on the contents of the file. In this case, since the file is a technical specification document, the raw text will be broken up into a combination of chunks that contain narrative text, chunks that contain the parsed specification table text, and chunks that contain any other kind of text. All chunks are then organized by detected title elements so that each chunk contains information that is semantically similar and is labeled with additional semantic context. Finally, the chunks are augmented with metadata based on the contents of the chunk. For example, the chunks that contain narrative text are augmented with summaries of the text in the chunk, while table-based chunks are augmented with aggregate information based on the parsed table—this could be high level aggregations of the data provided in table columns, or row level totals appended to the end of rows, etc. All chunks are augmented with “discovery questions”, meaning questions that are generated based on the text in the chunk which aid in semantic similarity matching further in the process, when the chunk is discovered for use as part of answer generation. All chunks are also scanned for customer names that appear within the text, which are then removed to provide better generic performance results, and to minimize the likelihood that undesired references to other customers end up in autogenerated text. The chunks are then embedded directly within the system's vector database. Based on the type of information contained within the chunks, we choose the right embedding model—i.e. tabular data versus narrative text versus other informational text. Chen column 7 lines 39-47 teaching the answer module 144 then analyzes the extracted questions to determine if they require RFP/RFI compatibility analysis (e.g., checking compliance with requested features) 314. Based on the analysis, the system establishes categories 316. The vector database of the data management module 142 (step 212 of FIG. 2 ) is used to form categories. Categories may be based upon semantic similarity, Q&A pairs that are semantically similar and other designated categories. Chen column 12 line 46 to column 13 line 40 teaching the user then uses the UI to run autogeneration on the four questions in the RFP. Below are the details for how one such run is executed for the first question. 1. Question 1—Provide the name of your CEO a. Step 1: Run semantic matching on the question. i. Strip the question of any newlines and replace them with spaces. ii. Send the question to a pre-trained, off the shelf embedding model (e.g., OpenAI's text-embedding-ada-002®) iii. The response of the embedding model will be a nth dimensional floating-point vector. iv. Make an augmented SQL request to the vector database to match against this question embedding. 1. Rule 1: Use the cosine similarity strategy to determine closeness between vectors (this is industry standard). 2. Rule 2: As part of the SQL request, filter the indexed embeddings (in the knowledge base) that are only part of the same customer organization. a. This is to ensure data is not leaked between different customer organizations. 3. Rule 3: After we receive the raw matched embeddings from the SQL query, run tag matching to filter further. a. Users can append tags to questions and resources in the knowledge base which will determine what can or cannot be considered a match, even if the embedding similarity is high. b. Tags are considered exclusionary and are part of a tag group. For example, “product” is a tag group that can contain “Product A” and “Product B”. If a question is tagged with “Product A”, it will not match against embeddings that are tagged with “Product B”. b. Step 2: Perform transformation on the retrieved semantic matches according to the following rules: i. Rule 1: If the semantic match is the embedding of a Question-and-Answer pair, make a SQL query to reconstruct the entire Q&A and use that as part of the context in the following Steps. 1. We do this because questions and answers are emedded separately from each other to maximize match probabilities. ii. Rule 2: If the semantic match is the embedding of a Discovery Question (see 204), then make a SQL query to fetch the associated chunk and use that instead as part of the context in the following Steps. iii. Rule 3: If the semantic match is the embedding of a part of an unstructured document, make a SQL query to fetch topics/subsections/document name to include alongside this chunk as part of the context in the following Steps. c. Step 2: Run the question through a query intent prompt to an LLM system to determine the underlying intent of the question.
and a content management system for managing and building the content and for storing and retrieving the content from the authoritative repository in collaboration with the GenAI engine; (Chen column 2 lines 21-30 teaching the data management module 142 and answer module 144 automate the RFP response process using advanced data management techniques and artificial intelligence. The data management module 142 is capable of ingesting various types of customer data, including product documentation, sales materials, security policies, and competitor information. This module analyzes the imported data, deploys different strategies for “chunking” the data, and implements context string embedding techniques to optimize data storage and retrieval. Chen column 2 lines 38-64 teaching FIG. 2 illustrates operations associated with the data management module 142. The data management module 142 converts structured and unstructured documents and question-and-answer pairs into searchable embeddings. Customer data 200 is imported into the data management module 142 from customer machine 102. The customer data may include: Previously completed RFPs Product documentation Sales materials (e.g., pitch decks, battle cards) Security policies and audit documentation Competitor documentation Other relevant information Existing knowledge bases (e.g., Q&A library)
(10) After data import through import interface 202, the data is analyzed 204 and placed into categories, such as pitch deck, product information, answered Q&A, marketing website materials and other designated categories. By way of example, the system analyzes the data and deploys different strategies for processing, including: Analyzing long-form documents (e.g., product update documents) Processing structured data (e.g., Excel files of completed RFP Q&A) Handling web-based content (e.g., marketing website materials). Chen column 8 lines 57-67 teaching after the questions, answer locations, and other relevant metadata like dropdown locations have been extracted from the document, the user is prompted to run autogeneration on the RFP, which will initiate an AI-based first-draft populating of answers (“AI autogeneration”) to the questions. Before initiating AI autogeneration, the user can modify a variety of factors in control of the output, such as word length, structure, verbosity, direct custom instructions, and more. The user can also define what subset of content is eligible to be used to generate answers per question, via the system's tagging feature. Chen column 10 lines 52-62 teaching consider Question 1, which asks “How many years has your company been in service?” When this question gets queued for AI autogeneration within the system, it is classified as a question that is looking for a direct fact. As a result, the system looks for resources and artifacts that have been indexed that are closely related to this question and references a direct fact about the company's age. Depending on the number of sources that are discovered, and the relevance and the recency of those sources, the system filters the sources that are the most desirable to answer the question.
The system then passes the sources and a set of instructions to the LLM to generate a first draft response. If the user provided custom instructions at time of AI autogeneration, this is also considered.)
an open AI component configured to integrate AI capabilities with the SmartRFP software tool to enhance processing and analysis of the RFP document; (Chen column 2 lines 21-37 teaching the data management module 142 and answer module 144 automate the RFP response process using advanced data management techniques and artificial intelligence. The data management module 142 is capable of ingesting various types of customer data, including product documentation, sales materials, security policies, and competitor information. This module analyzes the imported data, deploys different strategies for “chunking” the data, and implements context string embedding techniques to optimize data storage and retrieval.
The answer module 144 utilizes the processed data to create appropriate responses to RFP questions. This module employs sophisticated algorithms to analyze user inputs, determine document structure, extract relevant questions, and generate accurate answers using a combination of semantic matching, prompt engineering, and language model integration.
Chen column 7 lines 26-38 teaching FIG. 3 illustrates processing associated with the answer module 144. A user at customer machine 102 receives from the answer module 144 a user interface with prompts to specify a new project 300. The user uploads an RFP document and provides relevant metadata (e.g., customer name, due date, assignment details) 302. The answer module 144 analyzes the uploaded document to determine its structure and extracts questions 304. The answer module 144 evaluates whether it has sufficient confidence to extract questions automatically 306. If not (306—No), it prompts the user to manually extract or confirm questions and sections 308. The user is then prompted to confirm extracted questions and sections 310. Chen column 7 line 54 to column 8 line 3 teaching different strategies are then developed for answer generation 318. For example, a strategy may be based upon semantic similarity matching to existing Q&A pairs. There might be a partial matching and categorization of questions. There might be various prompt engineering techniques with language models. The answers are generated by sending prompts to a large language model 319. The answer module 144 evaluates the confidence of generated answers using multiple factors (e.g., number of sources used, recency of sources, token probability) 320. Data sources are then identified 322. That is, the vector database of the data management module 142 is queried to trace to original data sources to establish data lineage. Answers are displayed to the user along with confidence scores and source tracing information 324. Users can review, edit, and collaborate on the generated responses within the platform 326. Completed answers are then exported 328. Throughout this process, the system leverages a combination of Large Language Models (LLMs) provided by third party providers and fine-tuned open source LLMs (which can be self-hosted, on-premise, or cloud-based) to ensure optimal performance and flexibility.)
and a downstream review component configured to review, finalize, and approve the RFP response document downstream in the RFP process; (Chen column 7 lines 54-65 teaching the answers are generated by sending prompts to a large language model 319. The answer module 144 evaluates the confidence of generated answers using multiple factors (e.g., number of sources used, recency of sources, token probability) 320. Data sources are then identified 322. That is, the vector database of the data management module 142 is queried to trace to original data sources to establish data lineage. Answers are displayed to the user along with confidence scores and source tracing information 324. Users can review, edit, and collaborate on the generated responses within the platform 326. Completed answers are then exported 328.)
a dashboard calculating and reporting performance and data insights. (Chen column 7 lines 54-65 teaching the answers are generated by sending prompts to a large language model 319. The answer module 144 evaluates the confidence of generated answers using multiple factors (e.g., number of sources used, recency of sources, token probability) 320. Data sources are then identified 322. That is, the vector database of the data management module 142 is queried to trace to original data sources to establish data lineage. Answers are displayed to the user along with confidence scores and source tracing information 324. Users can review, edit, and collaborate on the generated responses within the platform 326. Completed answers are then exported 328.)
Chen does not teach or suggest and an application programming interface (API) component configured to facilitate data exchange and integration with other applications and to support interoperability and extensibility of the SmartRFP software tAool; a persistent component communicatively configured to store the data in one or more persistent databases;
However, Liu, which is directed to automatically populating responses using AI, teaches
and an application programming interface (API) component configured to facilitate data exchange and integration with other applications and to support interoperability and extensibility of the SmartRFP software tAool; (Liu column 16 lines 14-66 teaching this type of cloud-based data intake and query system may have several- benefits, including, but not limited to, lossless data ingestion, more robust disaster recovery, and faster or more efficient processing, searching, and indexing. A cloud-based data intake and query system as described in this section may provide separately scalable storage resources and compute resources, or separately scalable search and index resources. Additionally, the cloud-based data intake and query system may allow for applications to be developed on top of the data intake and query system, to extend or enhance functionality, through a gateway layer or one or more Application Programming Interfaces (APIs), which may provide customizable access control or targeted exposure to the workings of data intake and query system 108. In some embodiments, a cloud-based data intake and query system (e.g., the data intake and query system 108 configured for use with cloud-computing services) may include an intake system. Such an intake system can include, but is not limited to an intake buffer, such as Apache KAFKA® or Amazon KINESIS®, or an extensible compute layer, such as Apache SPARK™ or Apache FLINK®. In some embodiments, the search function and the index function may be separated or containerized, so that search functions and index functions may run or scale independently. In some embodiments, data that is indexed may be stored in buckets, which may be stored in a persistent storage once certain bucket requirements have been met, and retrieved as needed for searching. In some embodiments, the search functions and index functions run in stateless containers, which may be coordinated by an orchestration platform. These containerized search and index functions may retrieve data needed to carry out searching and indexing from the buckets or various other services that may also run in containers, or within other components of the orchestration platform. In this manner, loss of a single container, or even multiple containers, does not result in data loss, because the data can be quickly recovered from the various services or components or the buckets in which the data is persisted. In some embodiments, the cloud-based data intake and query system may implement tenant-based and user-based access control. In some embodiments, the cloud-based data intake and query system may implement an abstraction layer, through a gateway portal, an API, or some combination thereof, to control or limit access to the functionality of the cloud-based data intake and query system. 3.0 Automated Request Text Population Engine An automated request text population engine is discussed below that can receive request texts, such as in the form of a questionnaire, for example, and can perform one or more analyses on each request text, such as comparison with one or more request texts stored within a knowledge base. A request text stored within the knowledge base corresponds to a response (e.g., answer to a question) that may be suitable for the received request text. Liu column 17 lines 12-21 teaching for instance, a first prompt may read, “Company Address:” followed by a first blank text box and a second prompt may read, “Company Phone Number:” followed by a second blank text box. The automated request text population engine may identify each prompt as a “request text,” and each request text is then compared to the requests text stored in a knowledge base. Prior to analysis of the received file, the knowledge base is generated based on documents and data supplied by or corresponding to Company X. Liu column 24 lines 56-65 teaching additionally, or in the alternative, the communication interface 804 may be implemented with one or more radio units for supporting wireless communications with other electronic devices. The communication interface logic 808 may include logic for performing operations of receiving and transmitting one or more objects via the communication interface 804 to enable communication between the automated request text population engine 810 and network devices via a network (e.g., the internet) and/or cloud computing services.)
a persistent component communicatively configured to store the data in one or more persistent databases; (Liu column 16 lines 29-60 teaching in some embodiments, a cloud-based data intake and query system (e.g., the data intake and query system 108 configured for use with cloud-computing services) may include an intake system. Such an intake system can include, but is not limited to an intake buffer, such as Apache KAFKA® or Amazon KINESIS®, or an extensible compute layer, such as Apache SPARK™ or Apache FLINK®. In some embodiments, the search function and the index function may be separated or containerized, so that search functions and index functions may run or scale independently. In some embodiments, data that is indexed may be stored in buckets, which may be stored in a persistent storage once certain bucket requirements have been met, and retrieved as needed for searching. In some embodiments, the search functions and index functions run in stateless containers, which may be coordinated by an orchestration platform. These containerized search and index functions may retrieve data needed to carry out searching and indexing from the buckets or various other services that may also run in containers, or within other components of the orchestration platform. In this manner, loss of a single container, or even multiple containers, does not result in data loss, because the data can be quickly recovered from the various services or components or the buckets in which the data is persisted. In some embodiments, the cloud-based data intake and query system may implement tenant-based and user-based access control. In some embodiments, the cloud-based data intake and query system may implement an abstraction layer, through a gateway portal, an API, or some combination thereof, to control or limit access to the functionality of the cloud-based data intake and query system. Liu column 24 line 42 to column 25 line 8 teaching now referring to FIG. 8, an example embodiment of a logical representation of an automated request text population engine 810 is shown in accordance with some embodiments. The automated request text population engine 810, in an embodiment may be stored on a non-transitory computer-readable storage medium (“persistent storage”) 806 of a computing device 802 that includes a housing. The communication interface 804, in combination with a communication logic 808, enables communications with external network devices and/or other network appliances to receive updates for the automated request text population engine 810. According to one embodiment of the disclosure, the communication interface 804 may be implemented as a physical interface including one or more ports for wired connectors. Additionally, or in the alternative, the communication interface 804 may be implemented with one or more radio units for supporting wireless communications with other electronic devices. The communication interface logic 808 may include logic for performing operations of receiving and transmitting one or more objects via the communication interface 804 to enable communication between the automated request text population engine 810 and network devices via a network (e.g., the internet) and/or cloud computing services. The processor(s) 802 is further coupled to the persistent storage 806. According to one embodiment of the disclosure, the automated request text population engine 810, stored on the persistent storage 806, includes: (i) a knowledge base generation and update logic module 812, (ii) a knowledge base repository 814, (iii) a pre-processing logic module 816, (iv) one or more text similarity processing logic modules 818 1-818 M (wherein M≥1), (v) a word embedding logic module 820, (vi) a similarity determination logic module 822, and (vii) an answer retrieval logic module 824.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen to incorporate and an application programming interface (API) component configured to facilitate data exchange and integration with other applications and to support interoperability and extensibility of the SmartRFP software tAool; a persistent component communicatively configured to store the data in one or more persistent databases as taught in Liu with the motivation of leveraging conventional technology to facilitate the interaction of components with each other and to prevent the loss of data by providing a means to persist the data. (Liu column 16 lines 14-66)
Referring to claims 2 and 15,
Chen further teaches wherein the SmartRFP software tool is configured to utilize embeddings techniques and machine learning models for processing the RFP document. (Chen column 7 lines 26-53 teaching FIG. 3 illustrates processing associated with the answer module 144. A user at customer machine 102 receives from the answer module 144 a user interface with prompts to specify a new project 300. The user uploads an RFP document and provides relevant metadata (e.g., customer name, due date, assignment details) 302. The answer module 144 analyzes the uploaded document to determine its structure and extracts questions 304. The answer module 144 evaluates whether it has sufficient confidence to extract questions automatically 306. If not (306—No), it prompts the user to manually extract or confirm questions and sections 308. The user is then prompted to confirm extracted questions and sections 310. The answer module 144 then analyzes the extracted questions to determine if they require RFP/RFI compatibility analysis (e.g., checking compliance with requested features) 314. Based on the analysis, the system establishes categories 316. The vector database of the data management module 142 (step 212 of FIG. 2 ) is used to form categories. Categories may be based upon semantic similarity, Q&A pairs that are semantically similar and other designated categories. Different strategies are then developed for answer generation 318. For example, a strategy may be based upon semantic similarity matching to existing Q&A pairs. There might be a partial matching and categorization of questions. There might be various prompt engineering techniques with language models. The answers are generated by sending prompts to a large language model 319. The answer module 144 evaluates the confidence of generated answers using multiple factors (e.g., number of sources used, recency of sources, token probability) 320. Data sources are then identified 322. That is, the vector database of the data management module 142 is queried to trace to original data sources to establish data lineage. Answers are displayed to the user along with confidence scores and source tracing information 324. Users can review, edit, and collaborate on the generated responses within the platform 326. Completed answers are then exported 328. Throughout this process, the system leverages a combination of Large Language Models (LLMs) provided by third party providers and fine-tuned open source LLMs (which can be self-hosted, on-premise, or cloud-based) to ensure optimal performance and flexibility.) Chen column 9 lines 3-37 teaching when we receive a question to autogenerate an answer for, we first go through our context finding step. The context finding step performs the following actions: Semantic search across our indexed embeddings (as described in 204 through 208). The system first generates an embedding on the question using the same embeddings model that was used to index the content in the indexing step described above. The system then performs a cosine similarity look up search to fetch the 10 closest embeddings. We then use the text associated with those embeddings as part of our context in subsequent steps. Another technique we use is called Hypothetical Document Embeddings (HyDE). Because embeddings can sometimes be in paragraph format, it can lead to a bad fit for questions, which tend to be shorter and pithier. To combat this, we leverage LLMs to generate hypothetical paragraph texts that could answer the question at hand. We then perform regular semantic matching with those hypothetical paragraphs. We also perform regular keyword and ranking searches, which leverage keywords in the question to match embedded documents. If we have matched against a Question-and-Answer from our library, and our semantic cosine similarity is extremely high (>0.95), we assume that the Question from our Question-and-Answer library is roughly the same question. In these cases, we return the Answer from our Question-and-Answer library verbatim. Otherwise, we then send the question, along with the semantically matched question through two metadata generating steps: The first is to leverage an LLM call to hypothesize the intent of the question based on the context available.)
Referring to claims 3 and 16,
Liu further teaches wherein the GenAI is configured to implement parallel processing of PDF files to read all pages of a PDF document simultaneously using parallel processing algorithms for content extraction. (Liu column 19 lines 7-46 teaching in one embodiment, the questionnaire may be received as an attachment to an email communication. Receiving the questionnaire, set forth in block 604, may include performing of an optical character recognition (OCR) procedure on the attachment in order to convert the attachment into editable text. The editable text may then be extracted for analysis as described below. In response to receipt of the plurality of request texts, the automated request text population engine processes a first request text to determine a “similar” request text stored in the knowledge base (referred to herein as the “selected similar request text”) (block 606). In one embodiment, the selected similar request text may be the “most similar” request text stored in the knowledge base may be determined. The processing of the first request text may include a plurality of processes, one or more of which may be performed in a serial manner, concurrently (at least partially overlapping in time) or in parallel. As an initial step, the processing may include performance of pre-processing operations that result in a tokenization of the words within the first request text. The pre-processing operations may include removal of “stop words” (examples of which are provided below) and/or punctuation, which, in some cases, may provide no assistance in understanding the first request text. Further, removing the stop words and punctuation separates each remaining word into a “token.” Additional pre-processing operations may also be performed including converting all letters to lowercase, stemming words by removing a portion of a word such as one or more letters from the end of a word (e.g., converting “swinging” to “swing”), lemmatizing words (e.g., converting “swung” to “swing”) by converting a word to its root form, and/or normalizing words (e.g., converting “btw” to “by the way”) by converting abbreviations to the corresponding words canonical form. In order to perform operations corresponding to stemming words, lemmatizing words and/or normalizing words, an additional database can be pre-configured (and likely updated often) in order for the automated request text population engine to perform such operations. Liu column 23 lines 20-51 teaching additionally, or in the alternative, the method 700 may include performance of word embedding operations using the tokenized words and content of the knowledge base (block 710). As discussed above, the word embedding operations may include a trained neural network receiving a text corpus as input (e.g., the tokenized request texts) and outputs a numeric vector representing the tokenized text of each request text. Liu column 23 lines 28-51 teaching in the presently described embodiment, subsequently and utilizing the numeric vectors and content of the knowledge base, the method 700 continues with the performance of one or more text similarity processing logic modules (blocks 712 and 714). In some embodiments, when a plurality of text similarity processing logic modules are executed, such execution may occur in parallel. In some embodiments, the execution may occur serially or concurrently. In some embodiments, as discussed above, execution of the second and third text similarity processing logic modules may include execution of the WMD algorithm and/or the SCM algorithm. Following the performance of at least one of the first, second or third text similarity processing logic modules, the automated request text population engine determines a similar request text in the knowledge base for each request text in the questionnaire (wherein the determined similar request text is referred to as the “selected similar request text”) (block 716). Determination of the selected similar request text in the knowledge base for each request text in the questionnaire may include an analysis of the results produced by the first, second and/or third text similarity processing logic modules to determine which request text resulted in the “highest” similarity score.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen to incorporate wherein the GenAI is configured to implement parallel processing of PDF files to read all pages of a PDF document simultaneously using parallel processing algorithms for content extraction as taught in Liu with the motivation of facilitating the efficient analysis of acquired files. (Liu column 19 lines 7-46).
Referring to claims 5 and 18,
Liu further teaches further comprising a pipeline executable by one or more processors to implement a refinement process; (Liu column 14 lines 35-58 teaching Various embodiments of the present disclosure can be implemented using, or in conjunction with, a pipelined command language. A pipelined command language is a language in which a set of inputs or data is operated on by a first command in a sequence of commands, and then subsequent commands in the order they are arranged in the sequence. Such commands can include any type of functionality for operating on data, such as retrieving, searching, filtering, aggregating, processing, transmitting, and the like. As described herein, a query can thus be formulated in a pipelined command language and include any number of ordered or unordered commands for operating on data. Splunk Processing Language (SPL) is an example of a pipelined command language in which a set of inputs or data is operated on by any number of commands in a particular sequence. A sequence of commands, or command sequence, can be formulated such that the order in which the commands are arranged defines the order in which the commands are applied to a set of data or the results of an earlier executed command. For example, a first command in a command sequence can operate to search or filter for specific data in particular set of data. The results of the first command can then be passed to another command listed later in the command sequence for further processing.)
wherein the pipeline includes the SmartRFP software tool; (Liu column 14 lines 35-58 teaching Various embodiments of the present disclosure can be implemented using, or in conjunction with, a pipelined command language. A pipelined command language is a language in which a set of inputs or data is operated on by a first command in a sequence of commands, and then subsequent commands in the order they are arranged in the sequence. Such commands can include any type of functionality for operating on data, such as retrieving, searching, filtering, aggregating, processing, transmitting, and the like. As described herein, a query can thus be formulated in a pipelined command language and include any number of ordered or unordered commands for operating on data. Splunk Processing Language (SPL) is an example of a pipelined command language in which a set of inputs or data is operated on by any number of commands in a particular sequence. A sequence of commands, or command sequence, can be formulated such that the order in which the commands are arranged defines the order in which the commands are applied to a set of data or the results of an earlier executed command. For example, a first command in a command sequence can operate to search or filter for specific data in particular set of data. The results of the first command can then be passed to another command listed later in the command sequence for further processing. Liu column 15 lines 38-67 teaching due to its flexible nature, use of a pipelined command language in various embodiments is advantageous because it can perform “filtering” as well as “processing” functions. In other words, a single query can include a search command and search term expressions, as well as data-analysis expressions. For example, a command at the beginning of a query can perform a “filtering” step by retrieving a set of data based on a condition (e.g., records associated with server response times of less than 1 microsecond). The results of the filtering step can then be passed to a subsequent command in the pipeline that performs a “processing” step (e.g. calculation of an aggregate value related to the filtered events such as the average response time of servers with response times of less than 1 microsecond). Furthermore, the search command can allow events to be filtered by keyword as well as field value criteria. For example, a search command can filter out all events containing the word “warning” or filter out all events where a field value associated with a field “clientip” is “10.0.1.2.” The results obtained or generated in response to a command in a query can be considered a set of results data. The set of results data can be passed from one command to another in any data format. In one embodiment, the set of result data can be in the form of a dynamically created table. Each command in a particular query can redefine the shape of the table. In some implementations, an event retrieved from an index in response to a query can be considered a row with a column for each field value. Columns contain basic information about the data and also may contain data that has been dynamically extracted at search time. Liu column 17 lines 33-37 teaching therefore, the operations performed by the automated request text population engine do not merely provide an increase in efficiency in populating responses to received request texts, but provide an increase in the accuracy at which responses are provided. Liu column 19 lines 7-24 teaching in one embodiment, the questionnaire may be received as an attachment to an email communication. Receiving the questionnaire, set forth in block 604, may include performing of an optical character recognition (OCR) procedure on the attachment in order to convert the attachment into editable text. The editable text may then be extracted for analysis as described below. In response to receipt of the plurality of request texts, the automated request text population engine processes a first request text to determine a “similar” request text stored in the knowledge base (referred to herein as the “selected similar request text”) (block 606). In one embodiment, the selected similar request text may be the “most similar” request text stored in the knowledge base may be determined. The processing of the first request text may include a plurality of processes, one or more of which may be performed in a serial manner, concurrently (at least partially overlapping in time) or in parallel.)
wherein the refinement process is implemented continuously to refine various stages of the RFP process; (Liu column 14 lines 35-58 teaching Various embodiments of the present disclosure can be implemented using, or in conjunction with, a pipelined command language. A pipelined command language is a language in which a set of inputs or data is operated on by a first command in a sequence of commands, and then subsequent commands in the order they are arranged in the sequence. Such commands can include any type of functionality for operating on data, such as retrieving, searching, filtering, aggregating, processing, transmitting, and the like. As described herein, a query can thus be formulated in a pipelined command language and include any number of ordered or unordered commands for operating on data. Splunk Processing Language (SPL) is an example of a pipelined command language in which a set of inputs or data is operated on by any number of commands in a particular sequence. A sequence of commands, or command sequence, can be formulated such that the order in which the commands are arranged defines the order in which the commands are applied to a set of data or the results of an earlier executed command. For example, a first command in a command sequence can operate to search or filter for specific data in particular set of data. The results of the first command can then be passed to another command listed later in the command sequence for further processing.)
and wherein the refinement process includes analyzing historical RFPs data and tracking previous RFP performance based on various types of metrics to provide insight to refine the various stages of the RFP process. (Liu column 14 lines 35-58 teaching Various embodiments of the present disclosure can be implemented using, or in conjunction with, a pipelined command language. A pipelined command language is a language in which a set of inputs or data is operated on by a first command in a sequence of commands, and then subsequent commands in the order they are arranged in the sequence. Such commands can include any type of functionality for operating on data, such as retrieving, searching, filtering, aggregating, processing, transmitting, and the like. As described herein, a query can thus be formulated in a pipelined command language and include any number of ordered or unordered commands for operating on data. Splunk Processing Language (SPL) is an example of a pipelined command language in which a set of inputs or data is operated on by any number of commands in a particular sequence. A sequence of commands, or command sequence, can be formulated such that the order in which the commands are arranged defines the order in which the commands are applied to a set of data or the results of an earlier executed command. For example, a first command in a command sequence can operate to search or filter for specific data in particular set of data. The results of the first command can then be passed to another command listed later in the command sequence for further processing. Liu column 15 lines 3-67 teaching as such, a query formulated using SPL comprises a series of consecutive commands that are delimited by pipe “|” characters. The pipe character indicates to the system that the output or result of one command (to the left of the pipe) should be used as the input for one of the subsequent commands (to the right of the pipe). This enables formulation of queries defined by a pipeline of sequenced commands that refines or enhances the data at each step along the pipeline until the desired results are attained. Accordingly, various embodiments described herein can be implemented with Splunk Processing Language (SPL) used in conjunction with the SPLUNK® ENTERPRISE system. While a query can be formulated in many ways, a query can start with a search command and one or more corresponding search terms at the beginning of the pipeline. Such search terms can include any combination of keywords, phrases, times, dates, Boolean expressions, fieldname-field value pairs, etc. that specify which results should be obtained from an index. The results can then be passed as inputs into subsequent commands in a sequence of commands by using, for example, a pipe character. The subsequent commands in a sequence can include directives for additional processing of the results once it has been obtained from one or more indexes. For example, commands may be used to filter unwanted information out of the results, extract more information, evaluate field values, calculate statistics, reorder the results, create an alert, create summary of the results, or perform some type of aggregation function. In some embodiments, the summary can include a graph, chart, metric, or other visualization of the data. An aggregation function can include analysis or calculations to return an aggregate value, such as an average value, a sum, a maximum value, a root mean square, statistical values, and the like. Due to its flexible nature, use of a pipelined command language in various embodiments is advantageous because it can perform “filtering” as well as “processing” functions. In other words, a single query can include a search command and search term expressions, as well as data-analysis expressions. For example, a command at the beginning of a query can perform a “filtering” step by retrieving a set of data based on a condition (e.g., records associated with server response times of less than 1 microsecond). The results of the filtering step can then be passed to a subsequent command in the pipeline that performs a “processing” step (e.g. calculation of an aggregate value related to the filtered events such as the average response time of servers with response times of less than 1 microsecond). Furthermore, the search command can allow events to be filtered by keyword as well as field value criteria. For example, a search command can filter out all events containing the word “warning” or filter out all events where a field value associated with a field “clientip” is “10.0.1.2.” The results obtained or generated in response to a command in a query can be considered a set of results data. The set of results data can be passed from one command to another in any data format. In one embodiment, the set of result data can be in the form of a dynamically created table. Each command in a particular query can redefine the shape of the table. In some implementations, an event retrieved from an index in response to a query can be considered a row with a column for each field value. Columns contain basic information about the data and also may contain data that has been dynamically extracted at search time. Therefore, the operations performed by the automated request text population engine do not merely provide an increase in efficiency in populating responses to received request texts, but provide an increase in the accuracy at which responses are provided. As discussed herein, there may be plenty of human error involved in the population of request text responses; however, the automated request text population engine automatically analyzes pre-stored request texts within a knowledge base to determine a match (or most closely matched) request text and provides the corresponding response. Therefore, even as a request text of a first questionnaire may vary slightly from a request text of a second questionnaire with the intent being the same, the automated request text population engine provides the same response to each request text, thereby removing the human-error element. In addition, the automated request text population engine's use of a single knowledge base prevents the use of varying versions of information (e.g., old documents containing outdated information providing responses to some request texts). As the automated population process queries a single knowledge base (one that may be updated at regular intervals such as daily or weekly), the most up-to-date information will be provided as responses. Liu column 19 lines 7-24 teaching in one embodiment, the questionnaire may be received as an attachment to an email communication. Receiving the questionnaire, set forth in block 604, may include performing of an optical character recognition (OCR) procedure on the attachment in order to convert the attachment into editable text. The editable text may then be extracted for analysis as described below. In response to receipt of the plurality of request texts, the automated request text population engine processes a first request text to determine a “similar” request text stored in the knowledge base (referred to herein as the “selected similar request text”) (block 606). In one embodiment, the selected similar request text may be the “most similar” request text stored in the knowledge base may be determined. The processing of the first request text may include a plurality of processes, one or more of which may be performed in a serial manner, concurrently (at least partially overlapping in time) or in parallel.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen in view of Liu to incorporate further comprising a pipeline executable by one or more processors to implement a refinement process; wherein the pipeline includes the SmartRFP software tool; wherein the refinement process is implemented continuously to refine various stages of the RFP process; and wherein the refinement process includes analyzing historical RFPs data and tracking previous RFP performance based on various types of metrics to provide insight to refine the various stages of the RFP process as taught in Liu with the motivation of controlling the flow of data and assessing what further refinement/processing is needed to optimize the data for analysis. (Liu column 14 lines 35-58 and Liu column 15 lines 3-67)
Referring to claim 6,
Chen further teaches further comprising a content queue mechanism configured to flag the answers for reuse or update and subject flagged answers to a streamlined workflow process for evaluation and approval. (Chen column 7 lines 54-65 teaching the answers are generated by sending prompts to a large language model 319. The answer module 144 evaluates the confidence of generated answers using multiple factors (e.g., number of sources used, recency of sources, token probability) 320. Data sources are then identified 322. That is, the vector database of the data management module 142 is queried to trace to original data sources to establish data lineage. Answers are displayed to the user along with confidence scores and source tracing information 324. Users can review, edit, and collaborate on the generated responses within the platform 326. Completed answers are then exported 328. Chen column 10 lines 10-46 teaching after the answer generation step, we then perform what is known as the confidence calculation step. This is a mixture of heuristic-based analysis and LLM prompting. The heuristic-based analysis looks at the metadata of the context that was provided and assigns a numerical score to the “correctness” of the context provided as a whole. The higher the semantic similarity of context provided and the more recent the context was added and verified, the higher the score, and vice versa. For example, a piece of context that has extremely high semantic similarity (>0.85) but has not been updated for 3 years ago is awarded a low score. A piece of context that has medium semantic similarity but was very recently updated is awarded a higher score. Etc. We also leverage prompting to understand how complete the answer generated answers the question provided. If it fully and completely answers the question, we provided a HIGH score. If it partially answers the question, we apply a MEDIUM score, and if it does not answer the question at all, then we apply a LOW score. Finally, the numerical score and the HIGH/MEDIUM/LOW score are combined into a single HIGH/MEDIUM/LOW score by determining how far away negatively or positively the numerical score is from the median of 0.5 and adjusting the categorical score. This final confidence score is then made available to the user in the UI. We then perform a step to translate the answer to a different language if the user has a language other than US English specified. Finally, we save the answer, including the context documents that were used to generate the answer back into the system.)
Referring to claim 7,
Liu further teaches wherein the SmartRFP software tool is configured to generate the RFP response document in various formats while maintaining an original layout of the RFP document. (Liu column 18 line 46 to column 19 line 6 teaching subsequent to the generation of the knowledge base in the presently described embodiment, the automated request text population engine receives one or more request texts for which input is to be provided (block 604). In some embodiments, the one or more request texts refer to a questionnaire. For example, as discussed above, a questionnaire such as a Request for Proposal may be received by a corporation from a potential client that is to be answered, wherein the answers provided by the corporation will be analyzed by the potential client to determine whether to engage in business with the corporation. For purposes of clarity, this disclosure will refer to one or more request texts as a “questionnaire”; however, the disclosure is not intended to be so limited and pertains to any collection of one or more request texts. In some embodiments, the questionnaire may be received in a communication transmitted via a network connection. Additionally, a request text may be accompanied by multiple options for responding (e.g., a multiple answer options). The automated request text population engine may also translate any accompanying options for responding and compare the provided options for responding with a retrieved answer from the knowledge base. For example, when a questionnaire includes a multiple-choice question, e.g., a question and four possible answers, the automated request text population engine obtains the text corresponding to each possible answer and stores each as a possible answer for comparison with a retrieved answer from the knowledge base discussed below.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen to incorporate wherein the SmartRFP software tool is configured to generate the RFP response document in various formats while maintaining an original layout of the RFP document as taught in Liu with the motivation of providing responses for various types of questions and maintaining the original copy of review and comparison. (Liu column 18 line 46 to column 19 line 6)
Referring to claim 8,
Liu further teaches wherein the one or more persistent databases include at least one of a vector database, a relational database, and a cloud storage service to manage the storage and retrieval of the data. (Liu column 20 lines 50-60 teaching in Natural Language Processing (NLP), the cosine similarity is particularly used in positive space (e.g. after one hot encoding, which refers to a representation of categorical variables as binary vectors), where the outcome is neatly bounded in [0, 1]. In utilizing the cosine measure, or cosine similarity, to determine the similarity of the request text and an answer stored in the knowledge base, the automated request text population engine determines the cosine angle between a vector representing the requested text and a vector representing an answer in the knowledge base.
Liu column 24 line 66 to column 25 line teaching the processor(s) 802 is further coupled to the persistent storage 806. According to one embodiment of the disclosure, the automated request text population engine 810, stored on the persistent storage 806, includes: (i) a knowledge base generation and update logic module 812, (ii) a knowledge base repository 814, (iii) a pre-processing logic module 816, (iv) one or more text similarity processing logic modules 818 1-818 M (wherein M≥1), (v) a word embedding logic module 820, (vi) a similarity determination logic module 822, and (vii) an answer retrieval logic module 824. Liu column 16 lines 14-66 teaching this type of cloud-based data intake and query system may have several- benefits, including, but not limited to, lossless data ingestion, more robust disaster recovery, and faster or more efficient processing, searching, and indexing. A cloud-based data intake and query system as described in this section may provide separately scalable storage resources and compute resources, or separately scalable search and index resources. Additionally, the cloud-based data intake and query system may allow for applications to be developed on top of the data intake and query system, to extend or enhance functionality, through a gateway layer or one or more Application Programming Interfaces (APIs), which may provide customizable access control or targeted exposure to the workings of data intake and query system 108. In some embodiments, a cloud-based data intake and query system (e.g., the data intake and query system 108 configured for use with cloud-computing services) may include an intake system. Such an intake system can include, but is not limited to an intake buffer, such as Apache KAFKA® or Amazon KINESIS®, or an extensible compute layer, such as Apache SPARK™ or Apache FLINK®. In some embodiments, the search function and the index function may be separated or containerized, so that search functions and index functions may run or scale independently. In some embodiments, data that is indexed may be stored in buckets, which may be stored in a persistent storage once certain bucket requirements have been met, and retrieved as needed for searching. In some embodiments, the search functions and index functions run in stateless containers, which may be coordinated by an orchestration platform. These containerized search and index functions may retrieve data needed to carry out searching and indexing from the buckets or various other services that may also run in containers, or within other components of the orchestration platform. In this manner, loss of a single container, or even multiple containers, does not result in data loss, because the data can be quickly recovered from the various services or components or the buckets in which the data is persisted. In some embodiments, the cloud-based data intake and query system may implement tenant-based and user-based access control. In some embodiments, the cloud-based data intake and query system may implement an abstraction layer, through a gateway portal, an API, or some combination thereof, to control or limit access to the functionality of the cloud-based data intake and query system. 3.0 Automated Request Text Population Engine An automated request text population engine is discussed below that can receive request texts, such as in the form of a questionnaire, for example, and can perform one or more analyses on each request text, such as comparison with one or more request texts stored within a knowledge base. A request text stored within the knowledge base corresponds to a response (e.g., answer to a question) that may be suitable for the received request text.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen to incorporate wherein the one or more persistent databases include at least one of a vector database, a relational database, and a cloud storage service to manage the storage and retrieval of the data as taught in Liu with the motivation of persisting data that is available for analysis to prevent data loss. (Liu column 16 lines 14-66)
Referring to claim 9,
Chen further teaches wherein the GenAI engine is configured to generate personalized answers based on the context of the identified questions. (Chen column 2 lines 21-37 teaching the data management module 142 and answer module 144 automate the RFP response process using advanced data management techniques and artificial intelligence. The data management module 142 is capable of ingesting various types of customer data, including product documentation, sales materials, security policies, and competitor information. This module analyzes the imported data, deploys different strategies for “chunking” the data, and implements context string embedding techniques to optimize data storage and retrieval.
Chen column 7 lines 39-65 teaching the answer module 144 then analyzes the extracted questions to determine if they require RFP/RFI compatibility analysis (e.g., checking compliance with requested features) 314. Based on the analysis, the system establishes categories 316. The vector database of the data management module 142 (step 212 of FIG. 2 ) is used to form categories. Categories may be based upon semantic similarity, Q&A pairs that are semantically similar and other designated categories. Different strategies are then developed for answer generation 318. For example, a strategy may be based upon semantic similarity matching to existing Q&A pairs. There might be a partial matching and categorization of questions. There might be various prompt engineering techniques with language models. The answers are generated by sending prompts to a large language model 319. The answer module 144 evaluates the confidence of generated answers using multiple factors (e.g., number of sources used, recency of sources, token probability) 320. Data sources are then identified 322. That is, the vector database of the data management module 142 is queried to trace to original data sources to establish data lineage. Answers are displayed to the user along with confidence scores and source tracing information 324. Users can review, edit, and collaborate on the generated responses within the platform 326. Completed answers are then exported 328.)
Referring to claim 10,
Liu further teaches further comprising a feedback mechanism coupled to a workflow management system for continuous content enhancement and quality improvement. (Liu column 17 lines 39 to 67 teaching the automated request text population engine automatically analyzes pre-stored request texts within a knowledge base to determine a match (or most closely matched) request text and provides the corresponding response. Therefore, even as a request text of a first questionnaire may vary slightly from a request text of a second questionnaire with the intent being the same, the automated request text population engine provides the same response to each request text, thereby removing the human-error element. In addition, the automated request text population engine's use of a single knowledge base prevents the use of varying versions of information (e.g., old documents containing outdated information providing responses to some request texts). As the automated population process queries a single knowledge base (one that may be updated at regular intervals such as daily or weekly), the most up-to-date information will be provided as responses. Liu column 19 lines 7-24 teaching in one embodiment, the questionnaire may be received as an attachment to an email communication. Receiving the questionnaire, set forth in block 604, may include performing of an optical character recognition (OCR) procedure on the attachment in order to convert the attachment into editable text. The editable text may then be extracted for analysis as described below. In response to receipt of the plurality of request texts, the automated request text population engine processes a first request text to determine a “similar” request text stored in the knowledge base (referred to herein as the “selected similar request text”) (block 606). In one embodiment, the selected similar request text may be the “most similar” request text stored in the knowledge base may be determined. Liu column 24 lines 20-41 teaching in the presently described embodiment, subsequently, a determination is made as to whether request text represents the last request text in the questionnaire (block 726). When the request text is not the last request text in the questionnaire (“no” at block 726), ‘i’ is incremented and the method 700 returns to block 718. When the request text is the last request text in the questionnaire (“yes” at block 726), the questionnaire including the answers populated by the automated request text population engine may be optionally provided to an administrator for expert review (or at least the flagged request texts) (block 730). Following the optional expert review, the questionnaire including the answers populated by the automated request text population engine (and optionally those populated by an administrator) is provided to the requestor who submitted the questionnaire (or user/network device that provided the questionnaire or is associated therewith) (block 732). Following the expert review and returning to FIG. 7A, the method 700 may include updating the knowledge base with entries, comments and/or selected or corrected answers or request texts as determined during expert review (block 734).)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen to incorporate further comprising a feedback mechanism coupled to a workflow management system for continuous content enhancement and quality improvement as taught in Liu with the motivation of allowing reviewing users to provided feedback or additional knowledge pertaining to relevant text or corrected answers for future use. (Liu column 17 lines 39 to 67 and Liu column 24 lines 20-41)
Referring to claim 11,
Chen further teaches further comprising a hallucination prevention module for preventing hallucinations and ensuring fairness in the RFP response document by providing references to authoritative sources used. (Chen column 7 lines 54-65 teaching the answers are generated by sending prompts to a large language model 319. The answer module 144 evaluates the confidence of generated answers using multiple factors (e.g., number of sources used, recency of sources, token probability) 320. Data sources are then identified 322. That is, the vector database of the data management module 142 is queried to trace to original data sources to establish data lineage. Answers are displayed to the user along with confidence scores and source tracing information 324. Users can review, edit, and collaborate on the generated responses within the platform 326. Completed answers are then exported 328.)
Claims 4 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Patent No. 12,493,634) in view of Liu et al. (US Patent No. 11,379,670) and Kusch (US 20250124232).
Referring to claims 4 and 17,
Chen in view of Liu does not teach or suggest wherein the content management system is configured to: self-learn or self-tag to identify winning content; calculate a winnability score to quantify a likelihood of winning a particular RFP; and for continuously updating the authoritative repository with content having a high winnability score for reuse for processing future RFPs.
However, Kusch, which is directed to applying LLMs to proposal generation for contracts, teaches
wherein the content management system is configured to: self-learn or self-tag to identify winning content; (Kusch paragraph 17 teaching in some embodiments, for example as shown in FIG. 1, the system 200 may include a neural network-based contract analyzer 208 that processes RFP documents and predicts bid/no-bid recommendations. The system 200 may further include a contract generator 210, as shown in FIG. 1. The contract generator 210 may include a distributed multi-agent system with specialized AI agents for different aspects of proposal development. The system may include a memory 242 that maintains context and enables coordination between agents. In some embodiments, the system 200 may include a flexible human-in-the-loop architecture allowing variable automation levels. In some embodiments, system 200 may include integrated quality control mechanisms at one or more process points. In some embodiments, system 200 may include a nested, hierarchical decision-making structure that may at least partially mimic traditional proposal generation processes. The system 200 may reduce manual effort while improving proposal quality and win rates through consistent, data-driven processes. The system 200, or one or more components of the system, may be dynamically retrained to incorporate new data and/or outcomes and/or based on performance monitoring. Kusch paragraph 23 teaching the systems and methods described herein may also use LLMs to develop and/or generate a competitive proposal based on the identified contract, prior company contracts, and/or other third party contracts. For example, the LLM may draft textual and/or graphical content for a proposal to ensure that the proposal meets (or exceeds) all the requirements of the identified contract. In some embodiments, two LLMs may be trained separately using model selection, fine tuning, and/or in-context learning to generate different capabilities per LLM. One LLM is trained and used to draft proposal content, the other LLM is trained and used to analyze the content of the proposal to determine whether it meets (or exceed) predefined requirements. In some embodiments, the LLM may obtain and use prior technical approaches performed by the company and/or prior performance of the company in other related contracts (e.g., based on fine tuning and/or in-context learning as described elsewhere herein). In some embodiments, the LLM may highlight particular team qualifications based on the prior technical approaches and/or prior performance of the company or the team member. The LLM may access a dataset that includes prior technical approaches and/or prior performance for the company or team member in order to highlight particular qualifications for the instant proposal. In some embodiments, the LLM may generate multiple bids for inclusion in a contract based on analysis of prior company-based contracts and/or contracts/proposals belonging to other third parties that may be unrelated to the company. For example, the LLM may learn an effectivity rate (e.g., by adjusting the weights of the fine tuning) for past contracts and/or for other third party contracts and use such learning as a basis for generating new competitive proposals. Kusch paragraph 65 teaching through this supervised learning process, the model develops the ability to distinguish between high-scoring and low-scoring response patterns. When the model encounters a technical requirement asking for “detailed staffing approach,” it learns that responses labeled as “outstanding” typically include specific labor categories, qualification requirements, recruitment strategies, and retention plans, while responses labeled as “unacceptable” often lack these elements or provide only generic statements. This labeled training enables the model to learn specific patterns that correlate with proposal success or failure. For instance, the model learns that proposal sections labeled as “significant strength” often include quantifiable past performance metrics, specific technical approaches validated through previous contracts, and clear mapping between requirements and proposed solutions. Conversely, sections labeled as “significant weakness” typically exhibit gaps in addressing requirements, lack supporting evidence, or fail to provide required details.)
calculate a winnability score to quantify a likelihood of winning a particular RFP; (Kusch paragraph 35 teaching the process 100 may further include predicting the probability of winning a contract using the trained machine learning algorithm; and/or prioritizing a list of contracts based on the probability of winning the contract. The process 100 may utilize the trained machine learning model 102. The process 100 may optionally further include: receiving a selection of a contract to bid on. The selection may be based on the probability, such that a higher probability indicates a higher likelihood of successfully winning the bid for the contract.)
and for continuously updating the authoritative repository with content having a high winnability score for reuse for processing future RFPs. (Kusch paragraph 17 teaching in some embodiments, for example as shown in FIG. 1, the system 200 may include a neural network-based contract analyzer 208 that processes RFP documents and predicts bid/no-bid recommendations. The system 200 may further include a contract generator 210, as shown in FIG. 1. The contract generator 210 may include a distributed multi-agent system with specialized AI agents for different aspects of proposal development. The system may include a memory 242 that maintains context and enables coordination between agents. In some embodiments, the system 200 may include a flexible human-in-the-loop architecture allowing variable automation levels. In some embodiments, system 200 may include integrated quality control mechanisms at one or more process points. In some embodiments, system 200 may include a nested, hierarchical decision-making structure that may at least partially mimic traditional proposal generation processes. The system 200 may reduce manual effort while improving proposal quality and win rates through consistent, data-driven processes. The system 200, or one or more components of the system, may be dynamically retrained to incorporate new data and/or outcomes and/or based on performance monitoring. Kusch paragraph 23 teaching the systems and methods described herein may also use LLMs to develop and/or generate a competitive proposal based on the identified contract, prior company contracts, and/or other third party contracts. For example, the LLM may draft textual and/or graphical content for a proposal to ensure that the proposal meets (or exceeds) all the requirements of the identified contract. In some embodiments, two LLMs may be trained separately using model selection, fine tuning, and/or in-context learning to generate different capabilities per LLM. One LLM is trained and used to draft proposal content, the other LLM is trained and used to analyze the content of the proposal to determine whether it meets (or exceed) predefined requirements. In some embodiments, the LLM may obtain and use prior technical approaches performed by the company and/or prior performance of the company in other related contracts (e.g., based on fine tuning and/or in-context learning as described elsewhere herein). In some embodiments, the LLM may highlight particular team qualifications based on the prior technical approaches and/or prior performance of the company or the team member. The LLM may access a dataset that includes prior technical approaches and/or prior performance for the company or team member in order to highlight particular qualifications for the instant proposal. In some embodiments, the LLM may generate multiple bids for inclusion in a contract based on analysis of prior company-based contracts and/or contracts/proposals belonging to other third parties that may be unrelated to the company. For example, the LLM may learn an effectivity rate (e.g., by adjusting the weights of the fine tuning) for past contracts and/or for other third party contracts and use such learning as a basis for generating new competitive proposals. Kusch paragraph 63 teaching as used herein, “supervised learning” may include fine-tuning a model on a specific task using supervised learning. For example, in the context of RFP analysis and response generation, supervised learning may be implemented using a dataset of historical proposals paired with their outcomes and evaluation scores. The training data includes explicit labels such as “win” or “loss” for overall proposal outcomes, as well as numerical scores (e.g., 0-100) for specific evaluation criteria like technical approach, past performance, and cost realism. For instance, a proposal section responding to technical requirements may be labeled with both the agency's technical evaluation score and specific strengths or weaknesses identified during source selection.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen in view of Liu to incorporate wherein the content management system is configured to: self-learn or self-tag to identify winning content; calculate a winnability score to quantify a likelihood of winning a particular RFP; and for continuously updating the authoritative repository with content having a high winnability score for reuse for processing future RFPs as taught in Kusch with the motivation of identifying winning content to leverages in future RFPs. (Kusch paragraphs 23, 35, and 63).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Patent No. 12,493,634) in view of Liu et al. (US Patent No. 11,379,670) and Roberts (US 20160379285).
Referring to claim 12,
Chen in view of Liu does not teach or suggest further comprising a collaboration platform for enabling simultaneous stakeholder input and tailored responses and for supporting simultaneous editing and commenting on the RFP documents by multiple users.
However, Roberts, which is directed to obtaining a bid for services, teaches further comprising a collaboration platform for enabling simultaneous stakeholder input and tailored responses and for supporting simultaneous editing and commenting on the RFP documents by multiple users. (Roberts paragraph 31 teaching by way of example and not limitation, each user of web portal 10 is typically categorized such as an Administrative User, a Client User for client 2 (FIG. 1), a Service Provider User for service providers 3 (FIG. 1), or the like. Web portal 10 (FIG. 1) further typically comprises a set of user interface displays, generally described below, which implement and provide a user interface configured to provide a plurality of options to each user of web portal 10. Typically, the user interface selectively guides a user of web portal 10 in the steps needed to complete their required input into the RFP process. By way of further example and not limitation, web portal 10 may comprise or otherwise make various forms available such as browser-enabled or other Internet available forms, as illustrated in FIGS. 3-14. As illustrated in FIGS. 3-14, either an Administrative User or Client User with appropriate permissions can setup a Client User account (FIG. 3) and assign one or more roles/permissions to such a user (FIG. 4). Client Users can create and/or edit client RFP documents 100 (FIG. 2) and approve such client RFP documents 100 via one or more forms (FIGS. 5 and 6). Once submitted, Service Provider Users may be invited to view and respond to client RFP documents 100 via a form (FIG. 7). Client User and others may see a list of those Service Provider Users who have been invited, along with status information on their bidding, if any (FIG. 8). Client Users and/or Service Provider Users may retrieve (FIG. 9) client RFP documents 100 and modify various aspects of their response (e.g., FIG. 10). As illustrated in FIG. 10, Client Users can add descriptive or disclaiming qualifiers to portions of client RFP documents 100 (FIG. 11). Service Provider Users can use one or more forms (e.g., FIG. 12) to qualify and/or create a response to client RFP documents 100. Once bidding ceases, as described below, a weighted report (FIG. 13) may be generated and presented to client 2 such as on demand via web portal 10.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen to incorporate further comprising a collaboration platform for enabling simultaneous stakeholder input and tailored responses and for supporting simultaneous editing and commenting on the RFP documents by multiple users as taught in Roberts with the motivation of enabling a plurality of users to access a RFP and review and edit before finalization. (Roberts paragraph 31)
Claims 13 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Patent No. 12,493,634) in view of Liu et al. (US Patent No. 11,379,670) and Conway et al. (US 20250190459).
Referring to claims 13 and 19,
Chen in view of Liu does not teach or suggest further comprising a blind spot detection module configured to: detect a blind spot in the system by calculating a blind spot score; perform an evaluation, based on the blind spot score, to determine a drafting performance or decision-making capabilities of the system; and identify, based on the evaluation, features of the system that indicate a lack of understanding or awareness of the RFP document.
However Conway, which is directed to developing, assessing, and monitoring of a generative ai system, teaches further comprising a blind spot detection module configured to: detect a blind spot in the system by calculating a blind spot score; (Conway paragraph 22 teaching in some aspects, the techniques described herein relate to a method, wherein, for each generative AI system in the plurality of generative AI systems, the one or more quantitative metrics include: a factual accuracy metric indicating an extent to which completions generated by the respective generative AI system in response to the plurality of queries are factual; a faithfulness metric indicating an extent to which the completions generated by the respective generative AI system include hallucinated information; a grounded-ness metric indicating an extent to which the completions generated by the respective generative AI system are based on context data extracted from the knowledge base of the respective generative AI system; a toxicity metric indicating an extent to which the completions generated by the respective generative AI system include toxic content; a latency metric indicating a latency associated with the processing of the queries and/or generation of the completions by the respective generative AI system; a token count metric derived from a number of tokens included in the completions generated by the respective generative AI system; and/or a cost metric indicative a cost incurred by using the generative model of the respective generative AI system to generate the completions.)
perform an evaluation, based on the blind spot score, to determine a drafting performance or decision-making capabilities of the system; (Conway paragraphs 112-113 teaching existing Gen AI technology often has significant problems with bias and accuracy, and a propensity to hallucinate (e.g., produce erroneous results) or generate content having low relevance to the user's prompt (e.g., content in a different language, or information that is not responsive to the prompt). When such errors arise, they can be very difficult to diagnose and correct, due to the complexity of underlying models and the unsupervised or semi-supervised techniques by which the models are trained. This disclosure describes the application of automated machine learning (Auto ML) and/or predictive modeling techniques to the development, assessment, and/or monitoring of Gen AI systems to address these and other problems. In addition, this disclosure describes techniques for developing and deploying AI applications that include both generative and predictive models. Large language models (LLMs) have demonstrated robust performance in a wide variety of natural language tasks, including text summarization, extraction of relevant information, disambiguation of entities, and language translation. In addition, the LLMs used in many Gen AI systems are generalized LLMs, which can be used to perform such tasks across a wide variety of knowledge domains. However, even generalized generative models (e.g., LLMs) can suffer from “knowledge cutoff” or a “knowledge gap,” whereby a generative model (e.g., an LLM) is unaware of events that occurred after the date of its training and unaware of events or information not represented in its training dataset. For example, a generalized generative model (e.g., LLM) trained using publicly available data and documents found on the Internet would be unaware of private or confidential information, such as the information contained in an organization's private, internal documents. Conway paragraphs 123-124 teaching in some embodiments, monitoring models (e.g., predictive models trained to monitor various aspects of an AI system's performance) are used to monitor the performance of an AI system's components. For example, monitoring models can be used to automatically monitor (e.g., measure and/or track) and report the values of quantitative and/or qualitative metrics indicative of the performance of the AI system (or its components). Some examples of quantitative metrics indicative of the performance of an AI system (or its components) may include factualness/factual accuracy, faithfulness, grounded-ness, toxicity/appropriateness, correctness, and/or topicality of generated content (e.g., one or more “completions”) provided by the generative model; latency and/or cost (e.g., the time and/or financial cost associated with submitting a particular prompt and/or receiving a specific completion from the generative model 140), token counts (e.g., prompt tokens, response tokens, document tokens, and/or total tokens used to generate a response), the risk that a completion contains personally identifiable information (PII), etc.)
and identify, based on the evaluation, features of the system that indicate a lack of understanding or awareness of the RFP document. (Conway paragraphs 123-124 teaching in some embodiments, monitoring models (e.g., predictive models trained to monitor various aspects of an AI system's performance) are used to monitor the performance of an AI system's components. For example, monitoring models can be used to automatically monitor (e.g., measure and/or track) and report the values of quantitative and/or qualitative metrics indicative of the performance of the AI system (or its components). Some examples of quantitative metrics indicative of the performance of an AI system (or its components) may include factualness/factual accuracy, faithfulness, grounded-ness, toxicity/appropriateness, correctness, and/or topicality of generated content (e.g., one or more “completions”) provided by the generative model; latency and/or cost (e.g., the time and/or financial cost associated with submitting a particular prompt and/or receiving a specific completion from the generative model 140), token counts (e.g., prompt tokens, response tokens, document tokens, and/or total tokens used to generate a response), the risk that a completion contains personally identifiable information (PII), etc. Conway paragraphs 126-127 teaching in some embodiments, reporting tools can use monitoring models and/or additional generative models can be used to explain changes detected by the monitoring models. For example, when drift in the content (e.g., completions) generated by an AI system's generative model 140 is detected, the outputs of other monitoring models may indicate whether the completions drifted in response to drift in the user inputs, the constructed prompts, or the information provided by the knowledge base. In some embodiments, monitoring models may be used to generate embeddings for the AI system's completions and identify the topics associated with those embeddings. In some embodiments, monitoring models may assess the AI system's sensitivity to various words (e.g., in the user input) and alert the user when high-sensitivity words are used. In some embodiments, one or more generative models may be used to automatically generate text explaining the outputs of the monitoring models using natural language. Visualizations of the monitored metric values and/or the corresponding explanations can be provided to the user in real time (e.g., in a system monitoring dashboard). In some embodiments, the AI monitoring system may be configured with thresholds or ranges related to the metric values, and the AI monitoring system may alert the user when the value of a metric exceeds a corresponding threshold or departs from a specified range. One advantage of inserting monitoring models into the AI system at the component level (rather than simply monitoring inputs to the AI system and outputs from the AI system) is that the outputs of such models make it easier to determine which components of the AI system are causing the AI system to operate in unexpected or undesirable ways. In addition or in the alternative, when a monitoring model detects a potential problem (e.g., drift or an anomaly in a monitored input or output, an off-topic input, etc.), the AI monitoring system can alert the user to the presence of the issue, provide an explanation of the issue, and/or recommend remedial action (e.g., avoiding sensitive words in prompts, removing a portion of the knowledge base that is returning information of low relevance, etc.).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen in view of Liu to incorporate further comprising a blind spot detection module configured to: detect a blind spot in the system by calculating a blind spot score; perform an evaluation, based on the blind spot score, to determine a drafting performance or decision-making capabilities of the system; and identify, based on the evaluation, features of the system that indicate a lack of understanding or awareness of the RFP document as taught in Conway with the motivation of assessing the quality of answers and determining and identifying the underlying issues with an incorrect response. (Conway paragraphs 112-113, 123-124, and 126-127).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Patent No. 12,493,634) in view of Meditz et al. (US 20260187122).
Referring to claim 20,
Chen teaches:
A method for automating answer generation in a request for proposal (RFP) process, comprising: receiving an RFP document as input at a generative artificial intelligence (GenAI) module; (Chen column 1 lines 15-23 teaching the present invention relates generally to the field of artificial intelligence and, more particularly, to an apparatus and method for automating the process of responding to Requests for Proposals (RFPs), Requests for Information (RFIs), Security Questionnaires (SQs), and other B2B questionnaires (collectively referred to as RFPs in this filing) using advanced data management and AI-based answer generation techniques. Chen column 7 lines 26-31 teaching FIG. 3 illustrates processing associated with the answer module 144. A user at customer machine 102 receives from the answer module 144 a user interface with prompts to specify a new project 300. The user uploads an RFP document and provides relevant metadata (e.g., customer name, due date, assignment details) 302.)
identifying questions in the RFP document; (Chen column 2 lines 21-37 teaching the data management module 142 and answer module 144 automate the RFP response process using advanced data management techniques and artificial intelligence. The data management module 142 is capable of ingesting various types of customer data, including product documentation, sales materials, security policies, and competitor information. This module analyzes the imported data, deploys different strategies for “chunking” the data, and implements context string embedding techniques to optimize data storage and retrieval.
The answer module 144 utilizes the processed data to create appropriate responses to RFP questions. This module employs sophisticated algorithms to analyze user inputs, determine document structure, extract relevant questions, and generate accurate answers using a combination of semantic matching, prompt engineering, and language model integration.)
decomposing the identified questions into sub-questions; (Chen column 6 line 40 to column 7 line 5 teaching the parsing system is a combination of a heuristic-based or OCR-based reader that extracts the raw text from the file. The raw text is then analyzed, organized, and chunked up (separated into smaller, organized pieces of text) based on the contents of the file. In this case, since the file is a technical specification document, the raw text will be broken up into a combination of chunks that contain narrative text, chunks that contain the parsed specification table text, and chunks that contain any other kind of text. All chunks are then organized by detected title elements so that each chunk contains information that is semantically similar and is labeled with additional semantic context. Finally, the chunks are augmented with metadata based on the contents of the chunk. For example, the chunks that contain narrative text are augmented with summaries of the text in the chunk, while table-based chunks are augmented with aggregate information based on the parsed table—this could be high level aggregations of the data provided in table columns, or row level totals appended to the end of rows, etc. All chunks are augmented with “discovery questions”, meaning questions that are generated based on the text in the chunk which aid in semantic similarity matching further in the process, when the chunk is discovered for use as part of answer generation. All chunks are also scanned for customer names that appear within the text, which are then removed to provide better generic performance results, and to minimize the likelihood that undesired references to other customers end up in autogenerated text. The chunks are then embedded directly within the system's vector database. Based on the type of information contained within the chunks, we choose the right embedding model—i.e. tabular data versus narrative text versus other informational text)
rephrasing the sub-questions; (Chen column 6 line 40 to column 7 line 5 teaching the parsing system is a combination of a heuristic-based or OCR-based reader that extracts the raw text from the file. The raw text is then analyzed, organized, and chunked up (separated into smaller, organized pieces of text) based on the contents of the file. In this case, since the file is a technical specification document, the raw text will be broken up into a combination of chunks that contain narrative text, chunks that contain the parsed specification table text, and chunks that contain any other kind of text. All chunks are then organized by detected title elements so that each chunk contains information that is semantically similar and is labeled with additional semantic context. Finally, the chunks are augmented with metadata based on the contents of the chunk. For example, the chunks that contain narrative text are augmented with summaries of the text in the chunk, while table-based chunks are augmented with aggregate information based on the parsed table—this could be high level aggregations of the data provided in table columns, or row level totals appended to the end of rows, etc. All chunks are augmented with “discovery questions”, meaning questions that are generated based on the text in the chunk which aid in semantic similarity matching further in the process, when the chunk is discovered for use as part of answer generation. All chunks are also scanned for customer names that appear within the text, which are then removed to provide better generic performance results, and to minimize the likelihood that undesired references to other customers end up in autogenerated text. The chunks are then embedded directly within the system's vector database. Based on the type of information contained within the chunks, we choose the right embedding model—i.e. tabular data versus narrative text versus other informational text.)
classifying each rephrased sub-questions into one or more predefined topics; (Chen column 7 lines 39-47 teaching the answer module 144 then analyzes the extracted questions to determine if they require RFP/RFI compatibility analysis (e.g., checking compliance with requested features) 314. Based on the analysis, the system establishes categories 316. The vector database of the data management module 142 (step 212 of FIG. 2 ) is used to form categories. Categories may be based upon semantic similarity, Q&A pairs that are semantically similar and other designated categories.)
expanding each rephrased sub-question into multiple queries to cover different aspects of the sub-questions; (Chen column 6 line 40 to column 7 line 5 teaching the parsing system is a combination of a heuristic-based or OCR-based reader that extracts the raw text from the file. The raw text is then analyzed, organized, and chunked up (separated into smaller, organized pieces of text) based on the contents of the file. In this case, since the file is a technical specification document, the raw text will be broken up into a combination of chunks that contain narrative text, chunks that contain the parsed specification table text, and chunks that contain any other kind of text. All chunks are then organized by detected title elements so that each chunk contains information that is semantically similar and is labeled with additional semantic context. Finally, the chunks are augmented with metadata based on the contents of the chunk. For example, the chunks that contain narrative text are augmented with summaries of the text in the chunk, while table-based chunks are augmented with aggregate information based on the parsed table—this could be high level aggregations of the data provided in table columns, or row level totals appended to the end of rows, etc. All chunks are augmented with “discovery questions”, meaning questions that are generated based on the text in the chunk which aid in semantic similarity matching further in the process, when the chunk is discovered for use as part of answer generation. All chunks are also scanned for customer names that appear within the text, which are then removed to provide better generic performance results, and to minimize the likelihood that undesired references to other customers end up in autogenerated text. The chunks are then embedded directly within the system's vector database. Based on the type of information contained within the chunks, we choose the right embedding model—i.e. tabular data versus narrative text versus other informational text.)
embedding each query to create vector representation for performing similarity searches to calculate a similarity score; (Chen column 9 lines 3-37 teaching when we receive a question to autogenerate an answer for, we first go through our context finding step. The context finding step performs the following actions: Semantic search across our indexed embeddings (as described in 204 through 208). The system first generates an embedding on the question using the same embeddings model that was used to index the content in the indexing step described above. The system then performs a cosine similarity look up search to fetch the 10 closest embeddings. We then use the text associated with those embeddings as part of our context in subsequent steps. Another technique we use is called Hypothetical Document Embeddings (HyDE). Because embeddings can sometimes be in paragraph format, it can lead to a bad fit for questions, which tend to be shorter and pithier. To combat this, we leverage LLMs to generate hypothetical paragraph texts that could answer the question at hand. We then perform regular semantic matching with those hypothetical paragraphs. We also perform regular keyword and ranking searches, which leverage keywords in the question to match embedded documents. If we have matched against a Question-and-Answer from our library, and our semantic cosine similarity is extremely high (>0.95), we assume that the Question from our Question-and-Answer library is roughly the same question. In these cases, we return the Answer from our Question-and-Answer library verbatim. Otherwise, we then send the question, along with the semantically matched question through two metadata generating steps: The first is to leverage an LLM call to hypothesize the intent of the question based on the context available)
retrieving relevant documents for each query based on the similarity score; (Chen column 9 lines 3-37 teaching when we receive a question to autogenerate an answer for, we first go through our context finding step. The context finding step performs the following actions: Semantic search across our indexed embeddings (as described in 204 through 208). The system first generates an embedding on the question using the same embeddings model that was used to index the content in the indexing step described above. The system then performs a cosine similarity look up search to fetch the 10 closest embeddings. We then use the text associated with those embeddings as part of our context in subsequent steps. Another technique we use is called Hypothetical Document Embeddings (HyDE). Because embeddings can sometimes be in paragraph format, it can lead to a bad fit for questions, which tend to be shorter and pithier. To combat this, we leverage LLMs to generate hypothetical paragraph texts that could answer the question at hand. We then perform regular semantic matching with those hypothetical paragraphs. We also perform regular keyword and ranking searches, which leverage keywords in the question to match embedded documents. If we have matched against a Question-and-Answer from our library, and our semantic cosine similarity is extremely high (>0.95), we assume that the Question from our Question-and-Answer library is roughly the same question. In these cases, we return the Answer from our Question-and-Answer library verbatim. Otherwise, we then send the question, along with the semantically matched question through two metadata generating steps: The first is to leverage an LLM call to hypothesize the intent of the question based on the context available)
Chen does not teach or suggest re-ranking, using a generative pre-trained transformer, the relevant documents to identify most relevant documents based on the similarity score; processing the most relevant documents to generate AI answers for each sub-question; and compiling and merging the generated AI answers to generate a comprehensive response for each sub-question.
However, Meditz, which is directed to providing improved RAG for query response, teaches
re-ranking, using a generative pre-trained transformer, the relevant documents to identify most relevant documents based on the similarity score; (Meditz paragraphs 48-49 teaching in embodiments of the systems and methods described herein, an enhanced RAG system may incorporate a re-ranking mechanism that revisits and adjusts the ranking of retrieved information, improving the alignment of the retrieved results with the incoming query. Such embodiments implement a re-ranker that processes and reorganizes the initially ranked information based on additional criteria, thus refining the quality of the supporting evidence provided to the generative AI model. The enhanced RAG structure in these embodiments employs a multi-stage pipeline that aggregates several RAGs, each designed for specific functions, and introduces interactive processes among them. This pipeline may serve various purposes, such as retrieving potential answers, refining answer quality so that the final selection of evidence is optimally ranked according to relevance to the query. In certain embodiments, a re-ranking mechanism may output the most relevant supporting evidence from among multiple sources, leveraging both historical data and new LLM-generated content. This approach provides a flexible framework capable of adjusting dynamically between the use of previously provided human data and fresh AI-generated responses. The pipeline and re-ranking combination may further enhance the quality of model-generated answers, offering a more adaptable system suited for integration with downstream applications. Embodiments thus extend beyond simple RAG by allowing threshold-based decision-making, whereby the system may identify specific conditions under which it is beneficial to transition from historical data to AI-generated data. In such embodiments, the system may adjust the retrieval and ranking processes in real-time based on the quality threshold, thereby helping responses meet high standards of relevance and reliability. This enhanced approach to RAG facilitates higher-quality outputs and greater customization capabilities, addressing performance limitations found in simple RAG implementations. Meditz paragraph 67 teaching it should be noted that, in various embodiments, LLM 180 may include or employ other and/or additional types of models that are able to process large data sets. As understood herein, LLMs are machine learning models that are characterized by a massive number of parameters and typically leverage substantial computational resources for training and inference. These models are often designed to handle complex and high-dimensional data, enabling them to capture intricate patterns and relationships within the data. LLMs are often based on deep learning architectures like deep neural networks, convolutional neural networks (CNNs), or transformer models.)
processing the most relevant documents to generate AI answers for each sub-question, (Meditz paragraphs 48-49 teaching in embodiments of the systems and methods described herein, an enhanced RAG system may incorporate a re-ranking mechanism that revisits and adjusts the ranking of retrieved information, improving the alignment of the retrieved results with the incoming query. Such embodiments implement a re-ranker that processes and reorganizes the initially ranked information based on additional criteria, thus refining the quality of the supporting evidence provided to the generative AI model. The enhanced RAG structure in these embodiments employs a multi-stage pipeline that aggregates several RAGs, each designed for specific functions, and introduces interactive processes among them. This pipeline may serve various purposes, such as retrieving potential answers, refining answer quality so that the final selection of evidence is optimally ranked according to relevance to the query. In certain embodiments, a re-ranking mechanism may output the most relevant supporting evidence from among multiple sources, leveraging both historical data and new LLM-generated content. This approach provides a flexible framework capable of adjusting dynamically between the use of previously provided human data and fresh AI-generated responses. The pipeline and re-ranking combination may further enhance the quality of model-generated answers, offering a more adaptable system suited for integration with downstream applications. Embodiments thus extend beyond simple RAG by allowing threshold-based decision-making, whereby the system may identify specific conditions under which it is beneficial to transition from historical data to AI-generated data. In such embodiments, the system may adjust the retrieval and ranking processes in real-time based on the quality threshold, thereby helping responses meet high standards of relevance and reliability. This enhanced approach to RAG facilitates higher-quality outputs and greater customization capabilities, addressing performance limitations found in simple RAG implementations. Meditz paragraph 56 teaching in some example embodiments, value generator application 115 may be configured to coordinate with one or more repositories (e.g., databases or data stores), e.g., external repository 140 and/or other vector data stores such as, e.g., keys data store 150, values data store 160, and/or value chunks data store 170, as described in detail herein. External repository 140 may be one or a collection of databases, containing key-value pairs, e.g., Q&A pairs. In some embodiments, incorporating a key-value pairs database in a RAG system may serve as a useful component for storing, organizing, and retrieving contextual data to enhance the accuracy and reliability of responses generated by an LLM, such as LLM 180, described herein. The key-value pair database may take various forms, be populated with diverse types of data, and be implemented using multiple architectures and techniques. Meditz paragraph 66 teaching in some embodiments, value generator application 115 may query the various vector data stores, and may perform similarity searches to find vectors that closely match the content or meaning of the query being evaluated. Combining data from the data store with the user query, value generator application 115 may be configured to compose or otherwise prepare an augmented prompt for LLM 180, to generate a response to the query (as described in more detail herein). In some embodiments, the LLM response may be used to retrieve the top full answers from vector data store 160, e.g., using cosine similarity scores as the ranking metric in the retrieval. In some embodiments, as described in detail herein, value generator application 115 may be further configured to execute a post-processing step that provides the user the flexibility to alter the contents of vector data store 160 and/or incorporate additional criteria for the re-ranking.)
and compiling and merging the generated AI answers to generate a comprehensive response for each sub-question. (Meditz paragraphs 113-114 teaching in some embodiments, both the top ranking value and the proposed response may be provided to the user for selection as the final response. In some embodiments, the system may combine the top full value retrieved from the revised list of values and the LLM-generated proposed response to generate a final response that improves on either individual input. This combination process may leverage the strengths of both the curated value from the database and the generative capabilities of the LLM to produce a response that is more comprehensive, accurate, and contextually nuanced. The combination process may begin by aligning the content of the top full value and the proposed response. The processor may analyze the semantic content of both inputs to identify overlapping, complementary, or divergent information. For example, if the top full value provides detailed, fact-based information and the proposed response adds contextual or explanatory details, the system may merge these elements to enhance the depth and clarity of the final response. In some embodiments, the system may structure the combination by prioritizing key information from the top full value and supplementing it with additional insights from the proposed response. This structured approach may involve segmenting both inputs into smaller units, such as sentences or phrases, and selecting the most relevant or unique elements from each. Meditz paragraph 117 teaching the final response may also incorporate annotations or explanations that highlight the origins of specific elements, particularly in applications where transparency is important. For example, the system might indicate that factual information is sourced from the curated database, while supplementary context is generated by the LLM. By combining the top full value and the LLM-generated proposed response, in some embodiments, the system may produce a final response that integrates the factual reliability of the curated database with the contextual adaptability of generative AI. This approach may not only enhance the quality and relevance of the response but also allow the system to address complex queries that require a blend of structured data and nuanced interpretation.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the manner of automating the generation of a response to a RFP using LLMs or generative AI technology as taught in Chen in view of Meditz to incorporate re-ranking, using a generative pre-trained transformer, the relevant documents to identify most relevant documents based on the similarity score; processing the most relevant documents to generate AI answers for each sub-question; and compiling and merging the generated AI answers to generate a comprehensive response for each sub-question as taught in Meditz with the motivation of assessing retrieved documents to ensure the most pertinent documents are leveraged in answering the questions provided. (Meditz paragraphs 48-49, 113-114, and 117).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Lamba (US 20260037724) -directed processing contract documents using LLMs.
Dent et al. (US 20200057799) – directed to generating a proposal based on a RFP.
Any inquiry concerning this communication or earlier communications from the - examiner should be directed to MICHAEL J MONAGHAN whose telephone number is (571)270-5523. The examiner can normally be reached on Monday- Friday 8:30 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sarah Monfeldt can be reached on (571) 270-1833. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Michael J. Monaghan/Examiner, Art Unit 3629
Prosecution Insights