DETAILED ACTION
This office action is in response to the communication filed on August 13, 2026. Claims 1-20 are currently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed on August 13, 2026 have been fully considered but they are not persuasive for the following reasons:
Applicant in Pages 9-11 of the Remarks argues that the claims are not directed to any abstract idea, even if the claims were deemed to involve an abstract idea to some extent, the claims integrate any alleged abstract idea into a practical application.
Examiner respectfully disagrees.
It is important to note that the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements (MPEP 2106.05(a)).
If the specification sets forth an improvement in technology, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement (MPEP 2106.04(d)(1)).
Independent claim 1 and similarly independent claims 12 and 20 covers several steps, such as the determining and generating steps, that recite an abstract idea within the “Mental Processes” grouping of abstract ideas, because a person can mentally or using a pen and paper perform the limitations recited in said steps, which are again discussed in detail in the current 101 rejection below.
The remaining steps in the claims that are identified as reciting additional elements, are only adding insignificant extra-solution activity to the judicial exception, are recognized as a well understood, routine, and conventional activity within the field of computer functions, and are applying the exception using generic computer components, which is not sufficient to amount to significantly more than the judicial exception and are not directed to any specific improvement in computer technology.
Additionally, dependent claims 2-10 and 13-19 cover additional steps that also recite an abstract idea within the “Mental Processes” grouping of abstract ideas, as discussed in detail in the 101 rejection below.
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Applicant 11-13 of the Remarks argues that Kotte does not teach or even suggest the amended features "determining a use case associated with the user query, the use case being one of at least a generic use case or a specific use case, the determining comprising: transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific use case", as recited in amended independent claim 1 and similarly recited in amended independent claims 12 and 20.
Examiner respectfully disagrees. The previously cited prior art Kotte and the newly cited prior art Ding alone and/or in combination discloses the argued features.
Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain.
Kotte in [0025] and [0074] discloses contextual query-answering system utilizing domain-specific context, determined via a context retrieval model, to interpret queries more accurately, context mapping model determines the context in which the terms of the contextual query are used and applies domain-specific knowledge to interpret the intent behind the query more accurately.
Kotte in [0044] and [0046] discloses a collection of machine learning models generate or encode query embeddings from the query to represent semantic content of the contextual query, utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain, determine a cosine similarity between the embeddings to retrieve right answer for the query.
Therefore, Kotte discloses determining a use case associated with the user query, the use case being one of at least a generic use case or a specific use case.
Kotte discloses determining a use case associated with a query and generating a request to a machine learning model or agent, however, Kotte does not explicitly disclose:
…the determining comprising:
transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific use case;
Ding in Column 2 lines 44-55 and in Column 7 line 52 – Column 8 line 61 discloses receiving prompt to be input to a machine learning model, such as a LLM, prompt used to predict a classification of the prompt, based on the prompt a second machine learning model selected from a plurality of machine learning models to generate an optimized prompt, determining general or specific prompt classification, resulting optimized prompt sent to a LLM.
Therefore, Ding discloses a determining comprising transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific use case.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teachings of Kotte and Ding, to have combined Kotte and Ding. The motivation to combine Kotte and Ding would be to improve the effectiveness of a prompt by improving the inputs of the prompt.
For the above reasons, Examiner states that rejection of the current Office action is proper.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
At step 1:
Independent claims 1, 12, and 20 respectively recite a system, a method, and a
non-transitory computer-readable storage medium, which are directed to a statutory
category such as a process, machine, or an article of manufacture.
At step 2A, prong one:
Independent claim 1 and similarly independent claims 12 and 20 recites the limitations:
“determining a use case associated with the user query, the use case being one of at least a generic use case or a specific use case”,
A person can mentally or using a pen and paper determine a use case associated with a user query, the use case being one of at least a generic use case or a specific use case.
“ based on determining that the use case associated with the user query is the specific use case:”;
A person can mentally or using a pen and paper perform a decision or action based on mentally or using a pen and paper determining that a use case associated with a user query is a specific use case.
“generating a request for a response to the user query, the request
comprising the user query and a subset of the context information relevant to the
specific use case”;
A person can mentally or using a pen and paper generate a request for a
response to a user query, the request comprising the user query and a subset of
context information relevant to a specific use case.
The limitations, as recited above in claims 1, 12, and 20, are processes that, under their broadest reasonable interpretation, cover steps that can be performed in the human mind or by a human using a pen and paper, but for recitation of generic computer components.
If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claims recite an abstract idea.
At step 2A, prong two:
This judicial exception is not integrated into a practical application.
Independent claim 1 and similarly independent claims 12 and 20 recites the limitations:
“receiving, at a computing device, a user query”, which is a step of receiving data. The step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
“the determining comprising: transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific user case”, which is a step of transmitting data. The step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
“retrieving context information relevant to the specific use case”, which is a step of retrieving data. The step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
“transmitting the generated request to a second machine learning (ML) agent”, which is a step of transmitting data. The step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
“retrieving, from the second ML agent, the response to the user query”, which is a step of retrieving data. The step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
“storing the response to the user query at the computing device”, which is a step of storing data. The step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
“presenting the response to a user via a user interface (UI) of the computing device”, which is a step of presenting or outputting data. The step is recited at a high level of generality, and amounts to mere data outputting, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
The additional elements “a system comprising: at least one processor; at least one memory component storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:”, “at a computing device”, “to a first machine learning (ML) agent”, “from the first ML agent”, “at the computing device”, and “via a user interface (UI) of the computing device” in the steps in claim 1 are recited at a high-level of generality, such that it amounts to no more than mere instructions to apply the exception using generic computer components.
The additional elements “at a computing device”, “to a first machine learning (ML) agent”, “from the first ML agent”, “at the computing device”, and “via a user interface (UI) of the computing device” in the steps in claim 12 are recited at a high-level of generality, such that it amounts to no more than mere instructions to apply the exception using generic computer components.
The additional elements “a non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:”, “at a computing device”, “to a first machine learning (ML) agent”, “from the first ML agent”, “at the computing device”, and “via a user interface (UI) of the computing device” in the steps in claim 20 are recited at a high-level of generality, such that it amounts to no more than mere instructions to apply the exception using generic computer components.
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
At step 2B:
Independent claim 1 and similarly independent claims 12 and 20 recites the limitations:
“receiving, at a computing device, a user query”, which is a step of receiving data, and is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i)).
“the determining comprising: transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific user case”, which is a step of transmitting data, and is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i)).
“retrieving context information relevant to the specific use case”, which is a step of retrieving data, and is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
“transmitting the generated request to a second machine learning (ML) agent”, which is a step of transmitting data, and is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i)).
“retrieving, from the second ML agent, the response to the user query”, which is a step of retrieving data, and is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
“storing the response to the user query at the computing device”, which is a step of storing data, and is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
“presenting the response to a user via a user interface (UI) of the computing device”, which is a step of presenting or outputting data, and is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of presenting offers and gathering statistics (MPEP 2106.05(d)(II)(iv)).
Accordingly, the additional limitations are not sufficient to amount to significantly more than the judicial exception. Therefore, the claims are directed to an abstract idea and are not patent eligible.
Dependent claim 2 and similarly dependent claim 13 recites additional limitations, such as:
“wherein the request comprises a prompt and a context, the prompt comprising the user query, and the context comprising the subset of the context information relevant to the use case”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper generate a request comprising a prompt and a context, the prompt comprising a user query, and the context comprising a subset of a context information relevant to a use case, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 3 and similarly dependent claim 14 recites additional limitations, such as:
“wherein transmitting, to a first machine learning (ML) agent, a classification request comprises:”, which is a step of transmitting data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i)).
“generating a classification request associated with the user query”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper generate a classification request associated with a user query, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
and wherein determining the use case associated with the user query further comprises:
“transmitting the classification request to a first ML agent”, which is a step of transmitting data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i)).
“retrieving, from the first ML agent, the use case associated with the user query”, which is a step of retrieving data
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 4 and similarly dependent claim 15 recites additional limitations, such as:
wherein retrieving context information relevant to the specific use case further comprises:
“computing a query embedding associated with the user query”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper compute a query embedding associated with a user query, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“extracting a set of keywords from the user query”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper extract a set of keywords from a user query, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“retrieving, based on the query embedding or the set of keywords, a set of documents relevant to the specific use case for the user query”, which is a step of retrieving data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 5 and similarly dependent claim 16 recites additional limitations, such as:
“wherein the subset of the context information is determined based on selecting one or more documents from the set of documents relevant to the specific use case”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper determine a subset of context information based on mentally or using a pen and paper selecting one or more documents from a set of documents relevant to a specific use case, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 6 and similarly dependent claim 17 recites additional limitations, such as:
wherein retrieving the set of documents relevant to the specific use case further comprises:
“accessing stored documents, each stored document associated with a stored document embedding”, which is a step of accessing of retrieving stored data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
“computing relevance scores, each relevance score based on computing a relevance measure based on the query embedding and a stored document embedding associated with one of the stored documents”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper compute relevance scores, each relevance score based on mentally or using a pen and paper computing a relevance measure based on a query embedding and a stored document embedding associated with one of the stored documents, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“ranking the relevance scores”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper rank relevance scores, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“determining a set of highest ranked relevance scores, a size of the set corresponding to a pre-determined threshold”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper determine a set of highest ranked relevance scores, a size of the set corresponding to a pre-determined threshold, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“retrieving the stored documents associated with the relevance scores in the set of highest ranked relevance scores”, which is a step of retrieving data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 7 recites additional limitations, such as:
“wherein the relevance measure is a cosine similarity between the query embedding and the stored document embedding”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claim 1, because a person can mentally or using a pen and paper compute a relevance measure based on a cosine similarity between a query embedding and a stored document embedding, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 8 and similarly dependent claim 18 recites additional limitations, such as:
the operations further comprising:
“generating an additional prompt based on the prompt and a prompt modification”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper generate an additional prompt based on a prompt and a prompt modification, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“generating an additional request for an additional response to the user query, the additional request comprising the additional prompt and the context”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper generating an additional request for an additional response to a user query, the additional request comprising an additional prompt and a context, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“transmitting the generated additional request to the second ML agent”, which is a step of transmitting data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of receiving or transmitting data over a network (MPEP 2106.05(d)(II)(i)).
“retrieving, from the second ML agent, the additional response to the user query”, which is a step of retrieving data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
“storing the additional response to the user query at the computing device”, which is a step of storing data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 9 and similarly dependent claim 19 recites additional limitations, such as:
“retrieving a ground truth value associated with the user query and an evaluation metric”, which is a step of retrieving data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
“accessing the response and the additional response”, which is a step of accessing or retrieving data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
“computing a first evaluation score associated with the prompt based on one or more of the user query, the ground truth value, and the response”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper compute a first evaluation score associated with a prompt based on one or more of a user query, a ground truth value, and a response, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“computing a second evaluation score associated with the additional prompt based on one or more the user query, the ground truth value, and the additional response”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper compute a second evaluation score associated with an additional prompt based on one or more a user query, a ground truth value, and an additional response, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“selecting an evaluation score of the first evaluation score and the second evaluation score based on a predetermined criterion”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claims 1 and 12, because a person can mentally or using a pen and paper select an evaluation score of a first evaluation score and a second evaluation score based on a predetermined criterion, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“storing, at the computing device, the selected evaluation score and an associated one of the prompt or additional prompt”, which is a step of storing data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 10 recites additional limitations, such as:
“accessing the response and the additional response”, which is a step of accessing or retrieving data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
“presenting, via a first visual element of a second UI, the user query and the response”, which is a step of presenting or outputting data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data outputting, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of presenting offers and gathering statistics (MPEP 2106.05(d)(II)(iv)).
“presenting, via second visual element of the second UI, the user query and the additional response”, which is a step of presenting or outputting data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data outputting, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of presenting offers and gathering statistics (MPEP 2106.05(d)(II)(iv)).
“selecting, based on receiving user input, one of the response or the additional response”;
This limitation is directed to the same abstract idea under the mental processes grouping as independent claim 1, because a person can mentally or using a pen and paper select, based on receiving user input, one of a response or an additional response, and because the limitation does not recite any additional elements that are sufficient to amount to significantly more.
“storing the prompt or the additional prompt respectively associated with the selected one of the response or the additional response”, which is a step of storing data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Dependent claim 11 recites additional limitations, such as:
“the operations further comprising storing the prompt modification applied to the prompt to generate the additional prompt”, which is a step of storing data.
At step 2A prong two, the step is recited at a high level of generality, and amounts to mere data gathering, which is a form of insignificant extra-solution activity (MPEP 2106.05(g)).
At step 2B, the step is recognized as a well understood, routine, and conventional activity within the field of computer functions as an element of storing and retrieving information in memory (MPEP 2106.05(d)(II)(iv)).
Accordingly, the additional elements, individually or in combination, do not integrate the abstract idea into a practical application, even viewing the claims a whole, because it does not impose any meaningful limits on practicing the abstract idea.
Accordingly, dependent claims 2-11 and 13-19 are also directed to abstract idea without significantly more and are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kotte (US Pub 2025/0252265) in view of Ding (US Pat 12,505,288).
With respect to claim 1, Kotte discloses a system comprising:
at least one processor (Kotte in [0110] and [0132] and in Figure 12 discloses system comprising one or more processors);
at least one memory component storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations (Kotte in [0110] and [0132] and in Figure 12 discloses memory storing instructions executed by one or more processors to perform operations) comprising:
receiving, at a computing device, a user query (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain);
determining a use case associated with the user query, the use case being one of at least a generic use case or a specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0025] and [0074] discloses contextual query-answering system utilizing domain-specific context, determined via a context retrieval model, to interpret queries more accurately, context mapping model determines the context in which the terms of the contextual query are used and applies domain-specific knowledge to interpret the intent behind the query more accurately; Kotte in [0044] and [0046] discloses a collection of machine learning models generate or encode query embeddings from the query to represent semantic content of the contextual query, utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain, determine a cosine similarity between the embeddings to retrieve right answer for the query),…
based on determining that the use case associated with the user query is the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0044] and [0046] discloses utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain):
retrieving context information relevant to the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
generating a request for a response to the user query, the request comprising the user query and a subset of the context information relevant to the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
transmitting the generated request to a second machine learning (ML) agent (Kotte in [0023] and [0038] discloses a contextual query answering system utilizing machine learning models to provide accurate domain-specific contextual answers to user questions within software-specific domains; Kotte in [0029] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
retrieving, from the second ML agent, the response to the user query (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space);
storing the response to the user query at the computing device (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries); and
presenting the response to a user via a user interface (UI) of the computing device (Kotte in [0029] discloses transmitting content to client device to cause the device to display content associated with contextual queries, providing a query prompt for display, receiving contextual queries and providing contextual responses to the client device; Kotte in [0147] discloses presenting output to a user via I/O interfaces such as a display, I/O interfaces configured to provide graphical data to a display for presentation to a user).
Kotte discloses determining a use case associated with a query and generating a request to a machine learning model or agent, however, Kotte does not explicitly disclose:
…the determining comprising:
transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific use case;
The Ding reference discloses a determining comprising transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific use case (Ding in Column 2 lines 44-55 and in Column 7 line 52 – Column 8 line 61 discloses receiving prompt to be input to a machine learning model, such as a LLM, prompt used to predict a classification of the prompt, based on the prompt a second machine learning model selected from a plurality of machine learning models to generate an optimized prompt, determining general or specific prompt classification, resulting optimized prompt sent to a LLM).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teachings of Kotte and Ding, to have combined Kotte and Ding. The motivation to combine Kotte and Ding would be to improve the effectiveness of a prompt by improving the inputs of the prompt (Ding: Column 1 lines 7-13 and 56-63).
With respect to claim 2, Kotte in view of Ding discloses the system of claim 1, wherein the request comprises a prompt and a context, the prompt comprising the user query, and the context comprising the subset of the context information relevant to the use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0091] and [0092] discloses providing a prompt to the response generator model to initiate the generation of a contextual response, prompt includes a contextual query, relevant question-answer pairs, and context).
With respect to claim 3, Kotte in view of Ding discloses the system of claim 1, wherein determining the use case associated with the user query further comprises:
generating a classification request associated with the user query, the classification request comprising a classification prompt comprising descriptions of at least the generic use case and the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0043] and [0044] discloses a neural network includes or refers to a collection of machine learning models trained and/or tuned based on inputs to determine classifications, scores, or approximate unknown functions; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0091] and [0092] discloses providing a prompt to the response generator model to initiate the generation of a contextual response, prompt includes a contextual query, relevant question-answer pairs, and context);
transmitting the classification request to a second ML agent (Kotte in [0043] and [0048] discloses providing prompt to response generator model, response generator model refers to a collection of machine learning models for generating a contextual response for a contextual query from the prompt, a machine learning model trained and/or tuned based on inputs to determine classifications); and
retrieving, from the second ML agent, the use case associated with the user query (Kotte in [0043] and [0048] discloses providing prompt to response generator model, response generator model refers to a collection of machine learning models for generating a contextual response for a contextual query from the prompt, a machine learning model trained and/or tuned based on inputs to determine classifications).
With respect to claim 4, Kotte in view of Ding discloses the system of claim 1, wherein retrieving context information relevant to the specific use case further comprises:
computing a query embedding associated with the user query (Kotte in [0044] discloses context retrieval model comprising one or more machine learning models generating or encoding query embeddings from contextual query to represent semantic content of the contextual query, generating data segment embeddings from data segments to represent semantic content of the data segments; Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space);
extracting a set of keywords from the user query (Kotte in [0054] and [0069] discloses parsing to extract features including segments of text, keyword frequency, semantic features, entities, topics, summaries, and/or metadata; Kotte in [0126] and [0128] discloses extracting, using a context retrieval model trained to determine relevant contextual data, a query embedding from a contextual query); and
retrieving, based on the query embedding or the set of keywords, a set of documents relevant to the specific use case for the user query (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space).
With respect to claim 5, Kotte in view of Ding discloses the system of claim 4, wherein the subset of the context information is determined based on selecting one or more documents from the set of documents relevant to the specific use case (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space).
With respect to claim 6, Kotte in view of Ding discloses the system of claim 4, wherein retrieving the set of documents relevant to the specific use case further comprises:
accessing stored documents, each stored document associated with a stored document embedding (Kotte in [0029] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses; Kotte in [0044] and [0046] discloses a collection of machine learning models generate or encode query embeddings from the query to represent semantic content of the contextual query, utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain, determine a cosine similarity between the embeddings to retrieve right answer for the query, utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings);
computing relevance scores, each relevance score based on computing a relevance measure based on the query embedding and a stored document embedding associated with one of the stored documents (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list);
ranking the relevance scores (Kotte in [0071] and [0072] discloses ranking stored document based on context scores to determine a subset of relevant documents, considering the position of a document in the ranked list of documents; Kotte in [0079] and [0087] discloses generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries, adding a ranking inside a set of relevant documents derived by using a log of click ratio of query-document pair, consider the position of relevant documents in a ranked list, giving higher scores to documents at the top of the list);
determining a set of highest ranked relevance scores, a size of the set corresponding to a pre-determined threshold (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list); and
retrieving the stored documents associated with the relevance scores in the set of highest ranked relevance scores (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space).
With respect to claim 7, Kotte in view of Ding discloses the system of claim 6, wherein the relevance measure is a cosine similarity between the query embedding and the stored document embedding (Kotte in [0046] and [0047] discloses context retrieval model determines a cosine similarity between the query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query, selecting top-k relevant answers from a vector embedding repository; Kotte in [0070] and [0086] discloses utilizing a cosine similarity to measure the cosine of the angle between embedding vectors to determine the relationship between the query embedding and the data segment embeddings, computing a context score from cosine similarity; Kotte in [0116] and [0126] discloses determining a context score for stored documents based on comparing query embedding and plurality of data segment embeddings, selecting stored document as a relevant document based on the context score, determining a cosine similarity between a query embedding and a plurality of data segment embeddings extracted from stored digital documents).
With respect to claim 8, Kotte in view of Ding discloses the system of claim 2, the operations further comprising:
generating an additional prompt based on the prompt and a prompt modification (Kotte in [0018], [0023], and [0066] discloses training a model to generate and compare embeddings from contextual user queries and data segments to determine relevant contextual data, extracting portions of digital content item, such as question-answer pairs, document titles, and/or document descriptions from stored documents, such as help documents, community question records, and/or help videos, to use as context for generating responses to queries, learning from user interactions to provide contextual answers that improve in accuracy and relevance to a domain, taking a contextual query and enriching it by additional context; Kotte in [0047] and [0048] discloses generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator, such as a machine learning model, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0096] discloses training the response generator model prompted by a sample question, ground truth document(s), and negative document(s));
generating an additional request for an additional response to the user query, the additional request comprising the additional prompt and the context (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
transmitting the generated additional request to the first ML agent (Kotte in [0029] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
retrieving, from the first ML agent, the additional response to the user query (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space); and
storing the additional response to the user query at the computing device (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries).
With respect to claim 9, Kotte in view of Ding discloses the system of claim 8, the operations further comprising:
retrieving a ground truth value associated with the user query and an evaluation metric (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries; Kotte in [0097] and [0098] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space, contextual query answering system selects negative document(s) such that they satisfy a dis-similarity threshold based on cosine similarity compared to a ground truth document and satisfy a similarity threshold based on cosine similarity compared to the ground truth document; Kotte in [0099] and [0108] discloses during comparison using performance metrics to assess how well predicted response matches a ground truth answer, fine tuning a model for increasing accuracy between predicted responses and ground truth answers over successive iterations, fine tuning the model using training data that includes sample questions, ground truth documents including a question-answer pair, and negative documents);
accessing the response and the additional response (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries);
computing a first evaluation score associated with the prompt based on one or more of the user query, the ground truth value, and the response (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list; Kotte in [0097] and [0098] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space, contextual query answering system selects negative document(s) such that they satisfy a dis-similarity threshold based on cosine similarity compared to a ground truth document and satisfy a similarity threshold based on cosine similarity compared to the ground truth document);
computing a second evaluation score associated with the additional prompt based on one or more the user query, the ground truth value, and the additional response (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list; Kotte in [0097] and [0098] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space, contextual query answering system selects negative document(s) such that they satisfy a dis-similarity threshold based on cosine similarity compared to a ground truth document and satisfy a similarity threshold based on cosine similarity compared to the ground truth document);
selecting an evaluation score of the first evaluation score and the second evaluation score based on a predetermined criterion (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list); and
storing, at the computing device, the selected evaluation score and an associated one of the prompt or additional prompt (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries; Kotte in [0116] and [0126] discloses determining a context score for stored documents based on comparing query embedding and plurality of data segment embeddings, selecting stored document as a relevant document based on the context score, determining a cosine similarity between a query embedding and a plurality of data segment embeddings extracted from stored digital documents).
With respect to claim 10, Kotte in view of Ding discloses the system of claim 8, the operations further comprising:
accessing the response and the additional response (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries);
presenting, via a first visual element of a second UI, the user query and the response (Kotte in [0029] discloses transmitting content to client device to cause the device to display content associated with contextual queries, providing a query prompt for display, receiving contextual queries and providing contextual responses to the client device; Kotte in [0147] discloses presenting output to a user via I/O interfaces such as a display, I/O interfaces configured to provide graphical data to a display for presentation to a user);
presenting, via second visual element of the second UI, the user query and the additional response (Kotte in [0029] discloses transmitting content to client device to cause the device to display content associated with contextual queries, providing a query prompt for display, receiving contextual queries and providing contextual responses to the client device; Kotte in [0147] discloses presenting output to a user via I/O interfaces such as a display, I/O interfaces configured to provide graphical data to a display for presentation to a user);
selecting, based on receiving user input, one of the response or the additional response (Kotte in [0029] discloses transmitting content to client device to cause the device to display content associated with contextual queries, providing a query prompt for display, receiving contextual queries and providing contextual responses to the client device; Kotte in [0034] and [0041] discloses client device provides input, such as selected content items, generating or identifying content items in response to user interactions such as text queries and button selections; Kotte in [0147] discloses presenting output to a user via I/O interfaces such as a display, I/O interfaces configured to provide graphical data to a display for presentation to a user); and
storing the prompt or the additional prompt respectively associated with the selected one of the response or the additional response (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries).
With respect to claim 11, Kotte in view of Ding discloses the system of claim 10, the operations further comprising storing the prompt modification applied to the prompt to generate the additional prompt (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0029] and [0109] discloses transmitting content to client device to cause the device to display content associated with contextual queries, providing a query prompt for display, receiving contextual queries and providing contextual responses to the client device, digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0034] and [0041] discloses client device provides input, such as selected content items, generating or identifying content items in response to user interactions such as text queries and button selections).
With respect to claim 12, Kotte discloses a method comprising:
receiving, at a computing device, a user query (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain);
determining a use case associated with the user query, the use case being one of at least a generic use case or a specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0025] and [0074] discloses contextual query-answering system utilizing domain-specific context, determined via a context retrieval model, to interpret queries more accurately, context mapping model determines the context in which the terms of the contextual query are used and applies domain-specific knowledge to interpret the intent behind the query more accurately; Kotte in [0044] and [0046] discloses a collection of machine learning models generate or encode query embeddings from the query to represent semantic content of the contextual query, utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain, determine a cosine similarity between the embeddings to retrieve right answer for the query),…
based on determining that the use case associated with the user query is the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0044] and [0046] discloses utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain):
retrieving context information relevant to the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
generating a request for a response to the user query, the request comprising the user query and a subset of the context information relevant to the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
transmitting the generated request to a second machine learning (ML) agent (Kotte in [0023] and [0038] discloses a contextual query answering system utilizing machine learning models to provide accurate domain-specific contextual answers to user questions within software-specific domains; Kotte in [0029] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
retrieving, from the second ML agent, the response to the user query (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space);
storing the response to the user query at the computing device (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries); and
presenting the response to a user via a user interface (UI) of the computing device (Kotte in [0029] discloses transmitting content to client device to cause the device to display content associated with contextual queries, providing a query prompt for display, receiving contextual queries and providing contextual responses to the client device; Kotte in [0147] discloses presenting output to a user via I/O interfaces such as a display, I/O interfaces configured to provide graphical data to a display for presentation to a user).
Kotte discloses determining a use case associated with a query and generating a request to a machine learning model or agent, however, Kotte does not explicitly disclose:
…the determining comprising:
transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific use case;
The Ding reference discloses a determining comprising transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific use case (Ding in Column 2 lines 44-55 and in Column 7 line 52 – Column 8 line 61 discloses receiving prompt to be input to a machine learning model, such as a LLM, prompt used to predict a classification of the prompt, based on the prompt a second machine learning model selected from a plurality of machine learning models to generate an optimized prompt, determining general or specific prompt classification, resulting optimized prompt sent to a LLM).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teachings of Kotte and Ding, to have combined Kotte and Ding. The motivation to combine Kotte and Ding would be to improve the effectiveness of a prompt by improving the inputs of the prompt (Ding: Column 1 lines 7-13 and 56-63).
With respect to claim 13, Kotte in view of Ding discloses the method of claim 12, wherein the request comprises a prompt and a context, the prompt comprising the user query, and the context comprising the subset of the context information relevant to the use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0091] and [0092] discloses providing a prompt to the response generator model to initiate the generation of a contextual response, prompt includes a contextual query, relevant question-answer pairs, and context).
With respect to claim 14, Kotte in view of Ding discloses the method of claim 12, wherein determining the use case associated with the user query further comprises:
generating a classification request associated with the user query, the classification request comprising a classification prompt comprising descriptions of at least the generic use case and the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0043] and [0044] discloses a neural network includes or refers to a collection of machine learning models trained and/or tuned based on inputs to determine classifications, scores, or approximate unknown functions; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0091] and [0092] discloses providing a prompt to the response generator model to initiate the generation of a contextual response, prompt includes a contextual query, relevant question-answer pairs, and context);
transmitting the classification request to a second ML agent (Kotte in [0029] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt); and
retrieving, from the second ML agent, the use case associated with the user query (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space).
With respect to claim 15, Kotte in view of Ding discloses the method of claim 12, wherein retrieving the context information relevant to the specific use case further comprises:
computing a query embedding associated with the user query (Kotte in [0044] discloses context retrieval model comprising one or more machine learning models generating or encoding query embeddings from contextual query to represent semantic content of the contextual query, generating data segment embeddings from data segments to represent semantic content of the data segments; Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space);
extracting a set of keywords from the user query (Kotte in [0054] and [0069] discloses parsing to extract features including segments of text, keyword frequency, semantic features, entities, topics, summaries, and/or metadata; Kotte in [0126] and [0128] discloses extracting, using a context retrieval model trained to determine relevant contextual data, a query embedding from a contextual query); and
retrieving, based on the query embedding or the set of keywords, a set of documents relevant to the specific use case for the user query (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space).
With respect to claim 16, Kotte in view of Ding discloses the method of claim 15, wherein the subset of the context information is determined based on selecting one or more documents from the retrieved set of documents relevant to the specific use case (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space).
With respect to claim 17, Kotte in view of Ding discloses the method of claim 15, wherein retrieving the set of documents relevant to the determined use case further comprises:
accessing stored documents, each stored document associated with a stored document embedding (Kotte in [0029] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses; Kotte in [0044] and [0046] discloses a collection of machine learning models generate or encode query embeddings from the query to represent semantic content of the contextual query, utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain, determine a cosine similarity between the embeddings to retrieve right answer for the query, utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings);
computing relevance scores, each relevance score based on computing a relevance measure based on the query embedding and a stored document embedding associated with one of the stored documents (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list);
ranking the relevance scores (Kotte in [0071] and [0072] discloses ranking stored document based on context scores to determine a subset of relevant documents, considering the position of a document in the ranked list of documents; Kotte in [0079] and [0087] discloses generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries, adding a ranking inside a set of relevant documents derived by using a log of click ratio of query-document pair, consider the position of relevant documents in a ranked list, giving higher scores to documents at the top of the list);
determining a set of highest ranked relevance scores, a size of the set corresponding to a pre-determined threshold (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list); and
retrieving documents associated with the relevance scores in the determined set of highest ranked relevance scores (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space).
With respect to claim 18, Kotte in view of Ding discloses the method of claim 13, further comprising:
generating an additional prompt based on the prompt and a prompt modification (Kotte in [0018], [0023], and [0066] discloses training a model to generate and compare embeddings from contextual user queries and data segments to determine relevant contextual data, extracting portions of digital content item, such as question-answer pairs, document titles, and/or document descriptions from stored documents, such as help documents, community question records, and/or help videos, to use as context for generating responses to queries, learning from user interactions to provide contextual answers that improve in accuracy and relevance to a domain, taking a contextual query and enriching it by additional context; Kotte in [0047] and [0048] discloses generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator, such as a machine learning model, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0096] discloses training the response generator model prompted by a sample question, ground truth document(s), and negative document(s));
generating an additional request for an additional response to the user query, the additional request comprising the additional prompt and the context (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
transmitting the generated additional request to the first ML agent (Kotte in [0029] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
retrieving, from the first ML agent, the additional response to the user query (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space); and
storing the additional response to the user query at the computing device (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries).
With respect to claim 19, Kotte in view of Ding discloses the method of claim 18, further comprising:
retrieving a ground truth value associated with the user query and an evaluation metric (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries; Kotte in [0097] and [0098] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space, contextual query answering system selects negative document(s) such that they satisfy a dis-similarity threshold based on cosine similarity compared to a ground truth document and satisfy a similarity threshold based on cosine similarity compared to the ground truth document; Kotte in [0099] and [0108] discloses during comparison using performance metrics to assess how well predicted response matches a ground truth answer, fine tuning a model for increasing accuracy between predicted responses and ground truth answers over successive iterations, fine tuning the model using training data that includes sample questions, ground truth documents including a question-answer pair, and negative documents);
accessing the response and the additional response (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries);
computing a first evaluation score associated with the prompt based on one or more of the user query, the ground truth value, and the response (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list; Kotte in [0097] and [0098] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space, contextual query answering system selects negative document(s) such that they satisfy a dis-similarity threshold based on cosine similarity compared to a ground truth document and satisfy a similarity threshold based on cosine similarity compared to the ground truth document);
computing a second evaluation score associated with the additional prompt based on one or more the user query, the ground truth value, and the additional response (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list; Kotte in [0097] and [0098] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space, contextual query answering system selects negative document(s) such that they satisfy a dis-similarity threshold based on cosine similarity compared to a ground truth document and satisfy a similarity threshold based on cosine similarity compared to the ground truth document);
selecting an evaluation score of the first evaluation score and the second evaluation score based on a predetermined criterion (Kotte in [0043], [0070], and [0072] discloses a machine learning model trained and/or tunes based on inputs to determine scores, performing a comparison to generate context scores to determine relevant documents, such as relevant question-answer pairs, generate context score representing similarity, such as cosine similarity or an embedding space distance, between a query embedding and data segment embeddings, ranking the stored documents based on the context scores to determine a subset of documents that are relevant documents, filtering to exclude documents that do not meet a threshold context score; Kotte in [0072] and [0079] discloses utilizing an evaluation metric for the context scores, assign higher relevance scores to documents that appear at top of a relevancy list, generating relevance metric based on how relevant a document is to a query based by a frequency of user selection of the document for similar queries; Kotte in [0086] and [0087] discloses determine context score based on similarity, enhance alignment between similarity scores of a query and a document to determine context score, such as cosine similarity, comparing the context score to a reference similarity score, give higher score to relevant documents that appear at the top of a ranked list); and
storing, at the computing device, the selected evaluation score and an associated one of the prompt or additional prompt (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries).
With respect to claim 20, Kotte discloses a non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer (Kotte in [0110] and [0132] and in Figure 12 discloses instructions stored on a non-transitory computer-readable storage medium and executed by one or more processors to perform operations), cause the computer to:
receive, at a computing device, a user query (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain);
determine a use case associated with the user query, the use case being one of at least a generic use case or a specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0025] and [0074] discloses contextual query-answering system utilizing domain-specific context, determined via a context retrieval model, to interpret queries more accurately, context mapping model determines the context in which the terms of the contextual query are used and applies domain-specific knowledge to interpret the intent behind the query more accurately; Kotte in [0044] and [0046] discloses a collection of machine learning models generate or encode query embeddings from the query to represent semantic content of the contextual query, utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain, determine a cosine similarity between the embeddings to retrieve right answer for the query),…
based on determining that the use case associated with the user query is the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0044] and [0046] discloses utilizing query embeddings and data segment embeddings to determine relevant contextual data that pertains to the query within a specific domain):
retrieve context information relevant to the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
generate a request for a response to the user query, the request comprising the user query and a subset of the context information relevant to the specific use case (Kotte in [0002] and [0039] discloses by integrating contextual data producing responses that are highly relevant to a user query, receiving a contextual query from a client device, the query incorporates, either expressly or impliedly, terms or phrases specific to the functionality or features of a process, an operation, or an application in a specific domain; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models, to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
transmit the generated request to a second machine learning (ML) agent (Kotte in [0023] and [0038] discloses a contextual query answering system utilizing machine learning models to provide accurate domain-specific contextual answers to user questions within software-specific domains; Kotte in [0029] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt);
retrieve, from the second ML agent, the response to the user query (Kotte in [0045] and [0046] discloses based on a contextual query utilizing properties of a vector space to perform a similarity search of a vector embedding repository to retrieve relevant answers for a prompt, determine a cosine similarity between query embeddings and data segment embeddings to retrieve the relevant answers for the contextual query; Kotte in [0047] and [0048] discloses selecting relevant answers from similarity between query and vector embeddings, generating a prompt to include relevant answers and other relevant contextual data segments and text instructions for generating a contextual response, providing the prompt to a response generator referring a collection of machine learning models to generate the contextual response, utilizing a combination of the query, content from documents, and contextual learning to generate the prompt; Kotte in [0097] discloses contextual query answering system selects a top-k number of ground truth document(s) by retrieving documents that are similar to a selected ground truth document in a pre-indexed embedding space);
store the response to the user query at the computing device (Kotte in [0029] and [0109] discloses digital content management system generating, tracking, storing, processing, receiving, and transmitting date, including contextual queries, digital content, vector embeddings, and contextual responses, storing digital content used by the contextual query answering system including the digital input, contextual query, embeddings, and contextual responses; Kotte in [0041] and [0078] discloses identifying content items based on various contextual data, including graph information from a knowledge graph and/or historical user account behavior, generating a training dataset from stored user interaction logs that document user engagement after receiving responses to contextual queries, correlate the stored interaction logs with content of stored digital documents in relation to the contextual queries); and
present the response to a user via a user interface (UI) of the computing device (Kotte in [0029] discloses transmitting content to client device to cause the device to display content associated with contextual queries, providing a query prompt for display, receiving contextual queries and providing contextual responses to the client device; Kotte in [0147] discloses presenting output to a user via I/O interfaces such as a display, I/O interfaces configured to provide graphical data to a display for presentation to a user).
Kotte discloses determining a use case associated with a query and generating a request to a machine learning model or agent, however, Kotte does not explicitly disclose:
…the determining comprising:
transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific use case;
The Ding reference discloses a determining comprising transmitting, to a first machine learning (ML) agent, a classification request comprising a classification prompt, the classification prompt comprising descriptions of at least the generic use case and the specific use case (Ding in Column 2 lines 44-55 and in Column 7 line 52 – Column 8 line 61 discloses receiving prompt to be input to a machine learning model, such as a LLM, prompt used to predict a classification of the prompt, based on the prompt a second machine learning model selected from a plurality of machine learning models to generate an optimized prompt, determining general or specific prompt classification, resulting optimized prompt sent to a LLM).
Therefore, it would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, having the teachings of Kotte and Ding, to have combined Kotte and Ding. The motivation to combine Kotte and Ding would be to improve the effectiveness of a prompt by improving the inputs of the prompt (Ding: Column 1 lines 7-13 and 56-63).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Remarks
The relevant prior art of record that is not used in claim rejections but are pertinent to the claims or disclosure is:
Han (US Pub 2025/0181846), which discloses transmitting use-case specific prompts to LLM.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REZWANUL MAHMOOD whose telephone number is (571)272-5625. The examiner can normally be reached M-F 9-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ann J. Lo can be reached at 571-272-9767. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/R.M/Examiner, Art Unit 2159 /ALBERT M PHILLIPS, III/Primary Examiner, Art Unit 2159