DETAILED ACTION
This office action is in response to Applicant’s submission filed on 3/28/2025. Claims 1-20 are pending in the application of which Claims 1, 8, and 15 are independent and have been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1- 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter without significantly more. The claims as whole, considering all claim elements both individually and in combination, do not amount to significantly more than an abstract idea.
The group of independent claims 1, 8, and 15 disclose the same inventive concept with slightly different claim language. In this analysis, claim 1 is analyzed as a representative of the other independent claims. The independent claim 1 recites: “ … analyzing the natural language input using a support-request analysis engine to determine a sufficiency condition; in accordance with a determination that the sufficiency condition is not satisfied, generating a set of recommended data item queries; analyzing the set of recommended data item queries to identify a group of data item queries for which answers were not included in the natural language input; analyzing the identified group of data item queries to identify redundant data item queries that are directed to eliciting same information; generating a representative data item query for the redundant data item queries; including the representative data item query in the identified group of data item queries; for a data item query of the identified group of data item queries, generating a natural language query string for eliciting, from a user, an answer to the data item query, comprising: generating a question generation prompt comprising predetermined query prompt text and the data item query; providing the question generation prompt to a generative output engine; and receiving the natural language query string from the generative output engine; causing display of the natural language query string to the user; receiving, from the user, a response including the answer to the data item query; generating an issue record including the answer to the data item query; and storing the issue record in association with an issue tracking system.
The aforementioned limitations, under its broadest reasonable interpretation, cover performance of the limitation in the mind but for the recitation of generic computer components. That is, other than reciting “support-request analysis engine”, “generative output engine” and “processor”, nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the “support-request analysis engine”, “generative output engine “and “processor” language “,
“receiving a natural language input” from a graphical user interface of a client application operating on a client device;
“analyzing the natural language input” using a support-request analysis engine to determine a sufficiency condition;
in accordance with a determination that the sufficiency condition is not satisfied, “generating a set of recommended data item queries”;
“analyzing the set of recommended data item queries” to identify a group of data item queries for which answers were not included in the natural language input;
“analyzing the identified group of data item queries” to identify redundant data item queries that are directed to eliciting same information;
“generating a representative data item query” for the redundant data item queries; including the representative data item query in the identified group of data item queries;
for a data item query of the identified group of data item queries, “generating a natural language query string for eliciting, from a user, an answer to the data item query,” comprising: generating a question generation prompt comprising predetermined query prompt text and the data item query;
“providing the question generation prompt” to a generative output engine; and
receiving the natural language query string from the generative output engine;
causing display of the natural language query string to the user;
receiving, from the user, a response including the answer to the data item query;
generating an issue record including the answer to the data item query; and
storing the issue record in association with an issue tracking system.
All of these steps (as noted above) can be performed in the mind and/or using a pen and paper. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
This judicial exception is not integrated into a practical application. In particular, the claim only recites additional elements - using a “support-request analysis engine”, “generative output engine” and “processor” to perform all of the above-mentioned steps. The use of a “support-request analysis engine”, “generative output engine”, and “processor” is recited at a high-level of generality (i.e., as a generic computer device performing a generic computer function) such that it amounts no more than mere instructions to apply the exception using a generic computer component. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to an abstract idea. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The only element mentioned is the usage of a “support-request analysis engine”, “generative output engine”, which due to lack of specificity can be considered as a generic processor. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of a processor is merely for the purpose of data gathering and/or insignificant extra-solution activity that amount to no more than mere instructions to apply the exception using a generic computer component. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The claim is not patent eligible.
The dependent Claims do not add limitations that could help the Claim as a whole to amount to significantly more than the Abstract idea identified for the Independent Claim:
Similarly, dependent claims 2- 7, 9-14, and 16-20 are also not patent eligible as they include additional steps that are directed towards an abstract idea as they can be practically performed in the mind without being integrated into a practical application or including any additional elements sufficient to amount to significantly more than the judicial exception.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-16 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Meteer et al. (US20250370992A1)(herein " Meteer "), and in further view of Cash et al. (US 20140365227A1)(herein "Cash").
Regarding claim 1, Meteer teaches a computer-implemented method of generating questions in response to a user input, the method comprising: receiving a natural language input from a graphical user interface of a client application operating on a client device; (Meteer, Par. 0066:” … queries submitted to the Q-A system [client application], contextual data associated to the query, and/or contextual data associated with a user submitting the query.”, and Par. 0095:” A user associated with the customer that provided that document library (e.g., an employee of the customer) can subsequently submit a query (e.g., a natural language query, such as “how many vacation days does an employee with 2 years seniority get a year?”) that is processed by the system 100, …”, and Par. 0170:” … This mechanism is referred to as visual QA. Because the answers for the next lower level are displayed, the user is given the option to select answers resulting from a deterministic process.”, and Par. 0254:” … data may be dynamically adjustable (e.g., can be specified during runtime) via an interactive user interface (such as interactive user query interface 130 of FIG. 1).”)
analyzing the natural language input using a support-request analysis engine to determine a sufficiency condition; (Meteer, Par. 0092:” … analyzing a query, prior to executing the query, to determine, and if possible, resolve, various issues of ambiguity related to the submitted query. For example, a relevance detector [support-request analysis engine] performs a quick ambiguity/relevance classifications on queries to determine if a particular query is too ambiguous to be processed, or has enough clarity to return answer results with sufficient confidence in the quality of the answers (e.g., confidence that the answers returned are not nonsensical).”) Note: “query is too ambiguous” reads on determining sufficiency condition.
analyzing the set of recommended data item queries to identify a group of data item queries for which answers were not included in the natural language input; (Meteer, Par. 0259:” … For example, an ambiguous in-domain query classification will cause the Q-A system to either advise the user that the query is missing information (e.g., context), or may launch a disambiguation procedure.”) Note: “query is missing information” reads on “answers were not included in the input”.
analyzing the identified group of data item queries to identify redundant data item queries that are directed to eliciting same information; (Meteer, Par. 0211:” … After executing the query chunks (by the Q-A system), some of those query chunks will yield relevant answers (with some of those relevant answers nevertheless containing redundant information resulting from potential overlap between parsed queries), but pooling the answers together will more likely give a relevant answer as opposed to getting a relevant answer from the original verbose query (in the hope that the question-answer system can infer meaning out of a long sentence).”)
generating a representative data item query for the redundant data item queries; (Meteer, Par. 0211:” … After executing the query chunks (by the Q-A system), some of those query chunks will yield relevant answers (with some of those relevant answers nevertheless containing redundant information resulting from potential overlap between parsed queries), but pooling the answers together will more likely give a relevant answer as opposed to getting a relevant answer from the original verbose query (in the hope that the question-answer system can infer meaning out of a long sentence).“) Note: “pooling the answer together will more likely give a relevant answer as opposed to getting a relevant answer from the original verbose query” reads on generating a representative data item query for the redundant data item.
including the representative data item query in the identified group of data item queries; ( Meteer, Par. 0157:” FIG. 3 further illustrates a drill down query modification in which, upon a determination that the query is not sufficiently specific, or is missing some query information (concepts), the drilldown framework determines (possibly using feedback from the user, such as disambiguation feedback) more detailed concepts that are needed to produce query answers satisfying matching criteria. For example, consider a query asking “how do I remove the battery in my 3000 Inspiron.” The backoff/drilldown framework is configured to determine (e.g., based on a machine learning implementation trained to determine the specific details that are usually associated with different concepts in an ontology) that battery removal (the concept identified from the query) corresponds to information typically given with respect to specific laptop model, and to further determine that the specific model number is missing from the query. The framework can then activate a disambiguation process that requests further specificity from the user, or alternatively may infer the missing information (e.g., the model number) based on contextual information (e.g., based on previously asked queries that included the model number, based on the type of laptop through which the query was submitted, etc.) In the example of FIG. 3, the framework executes an interactive disambiguation dialogue that asks the user to specify the model for which laptop's battery is to be removed (as illustrated by arrow 304).”)
for a data item query of the identified group of data item queries, generating a natural language query string for eliciting, from a user, an answer to the data item query, comprising: generating a question generation prompt comprising predetermined query prompt text and the data item query; (Meteer, Par. 0157:” … The framework can then activate a disambiguation process that requests [eliciting] further specificity from the user, or alternatively may infer the missing information [data item query] (e.g., the model number) based on contextual information (e.g., based on previously asked queries that included the model number, based on the type of laptop through which the query was submitted, etc.) In the example of FIG. 3, the framework executes an interactive disambiguation dialogue that asks the user to specify the model for which laptop's battery is to be removed (as illustrated by arrow 304).”) Note: “query prompt text” reads on question (string) for asking from the user.
providing the question generation prompt to a generative output engine; and (Meteer, Par. 0006:” … identifying possible alternative queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches …, and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”)
receiving the natural language query string from the generative output engine; causing display of the natural language query string to the user; (Meteer, Par. 0141:” … using natural language processing or a learning machine implementation) but have different concept/category values, intermediary output may be provided to the user (e.g., as a visual disambiguation prompt, or an audio disambiguation prompt) requesting the user to provide clarification information specifying which of the identified concepts is more relevant to the user's query.”, and Par. 0169:” … the user interacting with the Q-A system may cause another search to be executed at a lower level (e.g., providing to the system additional disambiguation information). This process is basically a natural language interaction.”, and Par. 0170:” … This mechanism is referred to as visual QA. Because the answers for the next lower level are displayed, the user is given the option to select answers resulting from a deterministic process. Every time one of the displayed answers is clicked/selected, another augmented FAQ (corresponding to queries covering concepts at the next child level) gets triggered. Through this iterative process the user may finally land at the list that includes the desired answer …”).
receiving, from the user, a response including the answer to the data item query; (Meteer, Par. 0021:” Obtaining the disambiguation information may include one or more of, for example, obtaining query contextual information for the first query and/or generating prompt data to prompt a user to provide clarification information. Selecting at least one of the multiple matches may include selecting at least one of the multiple matches based, at least in part, on the query contextual information and/or on the clarification information provided by the user in response to the generated prompt data.”) Note: “clarification information provided by the user” reads on user’s response including answer to the data item query.
generating an issue record including the answer to the data item query; and storing the issue record in association with an issue tracking system. (Meteer, Par. 0003:” Computer users often have access to vast amounts of data, whether accessible through public networks (such as the Internet) or private networks, that the users can a search to find answers and information to specific or general queries about some topic or issue. For example, organizations often collect large number of documents that constitute a repository of information, be it administrative of technical information, that the organization's employees may access and perform searches on.”, and Par. 0145:” … In some embodiments, the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates.”, and Par. 0146:” … Such derived questions can then be re-submitted to the query processing module to retrieve follow-up answers. This process can be iteratively repeated up to a pre-determined number of times.) Note: past questions (issues) read on issue record, furthermore, iteratively repeated up reads on issue tracking.
Meteer does not teach, however Cash teaches in accordance with a determination that the sufficiency condition is not satisfied, generating a set of recommended data item queries; (Cash, Par. 0085:” As described above, in order to complete a structured query, task flow processor 342 sometimes needs to initiate additional dialogue with the user in order to obtain additional information, and/or disambiguate potentially ambiguous utterances. When such interactions are necessary, task flow processor 342 invokes dialogue processing module 340 (sometimes herein also called a "dialogue processor") to engage in a dialogue with the user. In some embodiments, dialogue processor 340 determines how (and/or when) to ask the user for the additional information, and receives and processes the user responses. The questions are provided to and answers are received from the users through I/O processing module 328. In some embodiments, dialogue processor 340 presents dialogue output to the user via audio and/or visual output, and receives input from the user via spoken or physical (e.g., clicking) responses. Continuing with the example above, when task flow processor 342 invokes dialogue flow processor 340 to determine the "party size" and "date" information for the structured query associated with the domain "restaurant reservation," dialogue flow processor 335 generates questions, such as "For how many people?" and "On which day?," to pass to the user. Once answers are received from the user, the dialogue flow processor 340 can then populate the structured query with the missing information, or pass the information to task flow processor 342 to complete the missing information from the structured query.”)
Cash is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Meteer further in view of Cash to generate a set of recommended data item queries, when the sufficiency condition is not satisfied. Motivation to do so would prevent the system from guessing and help it to get exact, and correct information.
Regarding claim 2, Meteer, as modified above, teaches the method claim of 1.
Meteer, as modified above, further teaches the predetermined query prompt text is predetermined first query prompt text; and (Meteer, Par. 0217:” … the invocation of a generative language model may include generating a message (provided to a local LLM engine, or sent to a remote LLM server) that includes the query to be paraphrased, and further includes a command or a prompt (specifying a query rephrasing instruction) that instructs the generative large language model to modify the query text [prompt] according to edit (configuration) data specified in the command or prompt. For example, for messages sent to ChatGPT, the message containing the content to be modified (in this case, the query data) may include a bracketed prompt specifying certain required characteristics for the desired output, such as the level of articulation (e.g., “[rephrase at a medium of perplexity]”; other options for the perplexity may be “high” or “low”). “)
providing the representative data item query prompt to the generative output engine; and receiving the representative data item query from the generative output engine. (Meteer, Par. 0021:” Obtaining the disambiguation information [data item query] may include one or more of, for example, obtaining query contextual information for the first query and/or generating prompt data to prompt a user to provide clarification information. Selecting at least one of the multiple matches may include selecting at least one of the multiple matches based, at least in part, on the query contextual information and/or on the clarification information provided by the user in response to the generated prompt data.”)
Meteer, as modified above, does not teach, however Cash further teaches generating the representative data item query for the redundant data item queries comprises: generating a representative data item query prompt comprising predetermined second query prompt text and text from the redundant data item queries; (Cash, Par. 0085:” … Continuing with the example above, when task flow processor 342 invokes dialogue flow processor 340 to determine the "party size" and "date" information for the structured query associated with the domain "restaurant reservation," dialogue flow processor 335 generates questions, such as "For how many people?" and "On which day?," to pass to the user. Once answers are received from the user, the dialogue flow processor 340 can then populate the structured query with the missing information, or pass the information to task flow processor 342 to complete the missing information from the structured query.”) Note: "For how many people?" read on the first query prompt, and "On which day?" read on the second query prompt.
Regarding claim 3, Meteer, as modified above, teaches the method claim of 1.
Meteer, as modified above, further teaches removing, from the identified group of data item queries, data item queries that are not supported by a support-request interaction type associated with the natural language input, thereby creating a list of candidate data item queries; and selecting the data item query from the list of candidate data item queries. (Meteer, Par. 0191:” … may include identifying multiple question-answer pairs relevant to the first query that are additionally associated with concepts at a next lower hierarchical level than the first concept, and presenting to the user at least some of the multiple question-answer pairs. Modifying the query to include drilled-down query terms may include modifying the query based on a question portion of a question-answer pair selected from the at least some of the multiple question-answer pairs.”) Note: “presenting to the user at least some of the multiple question-answer pairs” read on removal of queries.
Regarding claim 4, Meteer, as modified above, teaches the method claim of 1.
Meteer, as modified above, further teaches wherein the predetermined query prompt text is predetermined first query prompt text; and (Meteer, Par. 0095:” A user associated with the customer that provided that document library (e.g., an employee of the customer) can subsequently submit a query (e.g., a natural language query, such as “how many vacation days does an employee with 2 years seniority get a year?”) that is processed by the system 100, …”.)
submitting a second request to a knowledge-base service to obtain a second subset of recommended data item queries, the second subset of recommended data item queries generated by: identifying a knowledge base document of a set of knowledge base documents; ( Meteer, Par. 0006:” … identifying possible alternative queries [second request] to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches (e.g., imposing requirement that at least a certain percentage of query answers be determined from particular data collections, etc.), and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinements read on submitting a second request to a knowledge-base service to obtain a second subset of recommended data item queries
generating a data item query prompt comprising predetermined second query prompt text, text from the knowledge base document, and the natural language input; (Meteer, Par. 0006:” … identifying possible alternative [second] queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches (e.g., imposing requirement that at least a certain percentage of query answers be determined from particular data collections, etc.), and mitigating various ambiguity issues.”, and Par. 0021:” … generating prompt data to prompt a user to provide clarification information. Selecting at least one of the multiple matches may include selecting at least one of the multiple matches based, at least in part, on the query contextual information and/or on the clarification information provided by the user in response to the generated prompt data.”, and Par. 0022:” … determine at a subsequent time instance one or more subsequent search results [text from the knowledge base] for the modified query [second query prompt] comprising the one or more modified query terms.”)
providing the data item query prompt to a generative output engine; and (Meteer, Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”)
receiving, from the generative output engine, the second subset of recommend data item queries; and (Meteer, Par. 0006:” … identifying possible alternative queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches …, and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinement reads on answer to the composite data item query.
submitting a third request to a second service to obtain a third subset of recommended data item queries, the third subset of recommended data item queries associated with a term included in the natural language input. (Meteer, Par. 0006:” … identifying possible alternative queries [second request] to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches (e.g., imposing requirement that at least a certain percentage of query answers be determined from particular data collections, etc.), and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinements read on 2nd, 3rd requests.
Meteer, as modified above, does not teach, however, Cash further teaches generating the set of recommended data item queries comprises: submitting a first request to a first service to obtain a first subset of recommended data item queries; (Cash, Par. 0085:” As described above, in order to complete a structured query, task flow processor 342 sometimes needs to initiate additional dialogue with the user in order to obtain additional information, and/or disambiguate potentially ambiguous utterances. When such interactions are necessary, task flow processor 342 invokes dialogue processing module 340 (sometimes herein also called a "dialogue processor") to engage in a dialogue with the user. In some embodiments, dialogue processor 340 determines how (and/or when) to ask the user for the additional information, and receives and processes the user responses. The questions are provided to and answers are received from the users through I/O processing module 328. In some embodiments, dialogue processor 340 presents dialogue output to the user via audio and/or visual output, and receives input from the user via spoken or physical (e.g., clicking) responses. Continuing with the example above, when task flow processor 342 invokes dialogue flow processor 340 to determine the "party size" and "date" information for the structured query associated with the domain "restaurant reservation," dialogue flow processor 335 generates questions, such as "For how many people?" and "On which day?," to pass to the user. Once answers are received from the user, the dialogue flow processor 340 can then populate the structured query with the missing information, or pass the information to task flow processor 342 to complete the missing information from the structured query.”)
Regarding claim 5, Meteer, as modified above, teaches the method claim of 4.
Meteer, as modified above, further teaches receiving an identifier of the client application; and (Meteer, Par. 0066:” … queries submitted to the Q-A system [client application], contextual data associated to the query, and/or contextual data associated with a user submitting the query.”, and Par. 0095:” A user associated with the customer that provided that document library (e.g., an employee of the customer) can subsequently submit a query (e.g., a natural language query, such as “how many vacation days does an employee with 2 years seniority get a year?”) that is processed by the system 100, …”). Note: (Q-A) system represents the identifier.
the first service selects the first set of recommended data item queries based on the identifier of the client application. (Meteer, Par. 0175:” … Alternatively, if the output is presented as candidate Q-A pairs, selection of one of the presented choices will cause modification of the query to a modified query that may include, or is based on, the pre-determined question of the selected Q-A pair.”, and Par. 0182:” Selecting from a list of prompts that have been written in advance, …”).
Regarding claim 6, Meteer, as modified above, teaches the method claim of 1.
Meteer, as modified above, further teaches providing the natural language input as an input to a machine learning model that is trained on a dataset including respective natural language inputs associated with respective sufficiency determinations. (Meteer, Par. 0005:” … a machine-learning question-answering (“Q-A”) platform with improved document processing and information retrieval achieved by improved processing of data and users' queries during runtime (search-time).”, and Par. 0145:” … the query stack is configured to determine whether received query data matches (semantically, as may be determined by a machine learning semantic matching model, or is some other sense) one of pre-determined questions (which may be stored in the answer cache), and to generate the output data based on one or more answer data records (possibly stored within the answer cache) in response to determining that the received query data matches one of the pre-determined questions. In some embodiments, the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates.”, and Par. 0157:”FIG. 3 further illustrates a drill down query modification in which, upon a determination that the query is not sufficiently specific, or is missing some query information (concepts), the drilldown framework determines (possibly using feedback from the user, such as disambiguation feedback) more detailed concepts that are needed to produce query answers satisfying matching criteria. For example, consider a query asking “how do I remove the battery in my 3000 Inspiron.” The backoff/drilldown framework is configured to determine (e.g., based on a machine learning implementation trained to determine the specific details that are usually associated with different concepts in an ontology) that battery removal (the concept identified from the query) corresponds to information typically given with respect to specific laptop model, and to further determine that the specific model number is missing from the query. The framework can then activate a disambiguation process that requests further specificity from the user, or alternatively may infer the missing information (e.g., the model number) based on contextual information (e.g., based on previously asked queries that included the model number, based on the type of laptop through which the query was submitted, etc.) In the example of FIG. 3, the framework executes an interactive disambiguation dialogue that asks the user to specify the model for which laptop's battery is to be removed (as illustrated by arrow 304).”, and Par. 0164:” The process query box 410 next searches the content repository (e.g., the repository 140 of FIG. 1) to identify matches between the transformed query 402 and the content arranged in the repository, and returns answers 412, which may include answers presented in semantic form, a particular visual form, and/or the vector-transformed form. The identified answers may be those portions of the content being searched that satisfy a minimal matching criterion (for example, the identified answers are those with a relevance score exceeding some pre-determined threshold). Relevance/matching scores may be computed, for example, according to a distance measurements between semantic content of an answer and a corresponding question, according to a predicted output of a trained machine learning engine that assesses relevance of a question and a candidate answer, according to a computed Transform-Based-Distance (TBD) score between a question and an answer, according to a posterior probability equivalent of the TBD, or according to any other relevance derivation process. A particular question and answer pair with a high relevance score indicates a likelihood of a match between the query submitted and the content searched.”, and Par. 0244:” The identified answers in block 830 may be those portions of the content being searched that satisfy a minimal matching criterion (for example, the identified answers are those with a relevance score exceeding some pre-determined threshold). As noted above (e.g., in relation to FIG. 4) relevance/matching scores may be computed, for example, according to a distance measurements between semantic content of an answer and a corresponding question, according to a predicted output of a trained machine learning engine that assesses relevance of a question and a candidate answer, according to a computed Transform-Based-Distance (TBD) score between a question and an answer, according to a posterior probability equivalent of the TBD, or according to any other relevance derivation process. A particular question and answer pair with a high relevance score indicates a likelihood of a match between the query submitted and the content searched.”) Note: “query is not sufficiently specific”, or “missing some query information” read on sufficiency determination.
Regarding claim 7, Meteer, as modified above, teaches the method claim of 1.
Meteer, as modified above, further teaches wherein the predetermined query prompt text is predetermined first query prompt text; and (Meteer, Par. 0095:” A user associated with the customer that provided that document library (e.g., an employee of the customer) can subsequently submit a query (e.g., a natural language query, such as “how many vacation days does an employee with 2 years seniority get a year?”) that is processed by the system 100, …”.)
the method further comprises, prior to generating the natural language query string for eliciting the answer to the data item query: ranking the identified group of data item queries for which answers were not included in the natural language input based on relevance of the data item queries to a subject of the natural language input, comprising: (Meteer, Par. 0141:” … provide clarifying information to resolve ambiguity present in two or more of the query results. For example, when two answers are associated with the same or similar concept/category of information (be it an entity name, associated contextual information, or some abstract concept derived using natural language processing or a learning machine implementation) but have different concept/category values, intermediary output may be provided to the user (e.g., as a visual disambiguation prompt, or an audio disambiguation prompt) requesting the user to provide clarification information specifying which of the identified concepts is more relevant to the user's query. The disambiguation information returned by the user is then used to select one or more of the initial matches (and may eliminate some other matches), and/or to rank (based on computed relevance determined using the returned input from the user) the initial or remaining matches. …”, and Par. 0145:” … the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates.”) Note: Ranking as taught can be applied to queries for which answers were not included in the natural language input based on relevance.
generating a relevance inquiry prompt comprising predetermined second query prompt text, text from the identified group of data item queries, and the natural language input; (Meteer, Par. 0157:” … The framework can then activate a disambiguation process that requests [query prompt] further specificity from the user, or alternatively may infer the missing information (e.g., the model number) based on contextual information (e.g., based on previously asked queries that included the model number, based on the type of laptop through which the query was submitted, etc.) In the example of FIG. 3, the framework executes an interactive disambiguation dialogue that asks the user to specify the model for which laptop's battery is to be removed (as illustrated by arrow 304).”) Note: “second query prompt” reads on question (string) for asking from the user, which is a relevance-based query.
providing the relevance inquiry prompt to the generative output engine; and (Meteer, Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”)
receiving, from the generative output engine, a ranking of the data item queries of the identified group of data item queries; and (Meteer, Par. 0002:” … to question-answer systems to generate responses to queries submitted by a user, …”, and Par. 0141:” … rank (based on computed relevance determined using the returned input from the user) the initial or remaining matches. “)
selecting, based on the ranking, at least one data item query from the identified group of data item queries as the data item query. (Meteer, Par. 0145:” … In some embodiments, the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates.”) Note: ranking to identify matching candidate reads on selecting based on the ranking.
Regarding claim 8, Meteer teaches a computer-implemented method of generating questions in response to a user input, the method comprising: receiving a natural language input from a graphical user interface of a client application operating on a client device; (Meteer, Par. 0066:” … queries submitted to the Q-A system [client application], contextual data associated to the query, and/or contextual data associated with a user submitting the query.”, and Par. 0095:” A user associated with the customer that provided that document library (e.g., an employee of the customer) can subsequently submit a query (e.g., a natural language query, such as “how many vacation days does an employee with 2 years seniority get a year?”) that is processed by the system 100, …”, and Par. 0170:” … This mechanism is referred to as visual QA. Because the answers for the next lower level are displayed, the user is given the option to select answers resulting from a deterministic process.”, and Par. 0254:” … data may be dynamically adjustable (e.g., can be specified during runtime) via an interactive user interface (such as interactive user query interface 130 of FIG. 1).”)
ranking the group of recommended data item queries based on a relevance of the recommended data item queries to a subject of the natural language input, comprising: (Meteer, Par. 0141:" ... when two answers are associated with the same or similar concept/category of information (be it an entity name, associated contextual information, or some abstract concept derived using natural language processing or a learning machine implementation) but have different concept/category values, intermediary output may be provided to the user (e.g., as a visual disambiguation prompt, or an audio disambiguation prompt) requesting the user to provide clarification information specifying which of the identified concepts is more relevant to the user's query. The disambiguation information returned by the user is then used to select one or more of the initial matches (and may eliminate some other matches), and/or to rank (based on computed relevance determined using the returned input from the user) the initial or remaining matches.
generating a first prompt comprising predetermined first query prompt text, text from the group of recommended data item queries, and the natural language input; (Meteer, Par. 0095:” … submit a query (e.g., a natural language query, such as “how many vacation days does an employee with 2 years seniority get a year?”) that is processed by the system 100, and, in situations where a quick answer is not otherwise available from a cache for commonly-asked-questions, the query is processed and transformed into a format compatible with the format of ingested documents to identify portions in one or more of the ingested documents that may contain the answer to the user's query.”, and Par. 0169:” … So the user interacting with the Q-A system may cause another search to be executed at a lower level … This process is basically a natural language interaction.”)
providing the first prompt to a generative output engine; and (Meteer, Par. 0039:” … providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics. “, and Par. 0220:” … performed by one or more generative LLM engines configured to receive a first query [prompt] and a command …”).
receiving, from the generative output engine, a ranking of the recommended data item queries of the group of recommend data item queries; (Meteer, Par. 0002:” … to question-answer systems to generate responses to queries [prompt] submitted by a user, …”, and Par. 0141:” … rank (based on computed relevance determined using the returned input from the user) the initial or remaining matches. “)
selecting, based on the ranking, at least one data item query from the recommended data item queries; (Meteer, Par. 0145:” … In some embodiments, the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates.”) Note: ranking to identify matching candidate reads on selecting based on the ranking.
for the selected at least one data item query, generating a natural language query string for eliciting, from a user, an answer to the at least one data item query, comprising: generating a question generation prompt comprising predetermined second query prompt text and the at least one data item query; (Meteer, Par. 0157:” … The framework can then activate a disambiguation process that requests [eliciting] further specificity from the user, or alternatively may infer the missing information [data item query] (e.g., the model number) based on contextual information (e.g., based on previously asked queries that included the model number, based on the type of laptop through which the query was submitted, etc.) In the example of FIG. 3, the framework executes an interactive disambiguation dialogue that asks the user to specify the model for which laptop's battery is to be removed (as illustrated by arrow 304).”) Note: “second query prompt text” reads on question (string) for asking from the user.
providing the question generation prompt to the generative output engine; and (Meteer, Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”)
receiving the natural language query string from the generative output engine; causing display of the natural language query string to the user; (Meteer, Par. 0141:” … using natural language processing or a learning machine implementation) but have different concept/category values, intermediary output may be provided to the user (e.g., as a visual disambiguation prompt, or an audio disambiguation prompt) requesting the user to provide clarification information specifying which of the identified concepts is more relevant to the user's query.”, and Par. 0169:” … the user interacting with the Q-A system may cause another search to be executed at a lower level (e.g., providing to the system additional disambiguation information). This process is basically a natural language interaction.”, and Par. 0170:” … This mechanism is referred to as visual QA. Because the answers for the next lower level are displayed, the user is given the option to select answers resulting from a deterministic process. Every time one of the displayed answers is clicked/selected, another augmented FAQ (corresponding to queries covering concepts at the next child level) gets triggered. Through this iterative process the user may finally land at the list that includes the desired answer …”).
receiving, from the user, a response including the answer to the at least one data item query; and (Meteer, Par. 0021:” Obtaining the disambiguation information may include one or more of, for example, obtaining query contextual information for the first query and/or generating prompt data to prompt a user to provide clarification information. Selecting at least one of the multiple matches may include selecting at least one of the multiple matches based, at least in part, on the query contextual information and/or on the clarification information provided by the user in response to the generated prompt data.”) Note: “clarification information provided by the user” reads on user’s response including answer to the data item query.
storing the answer to the at least one data item query in association with the natural language input. (Meteer, Par. 0003:” Computer users often have access to vast amounts of data, whether accessible through public networks (such as the Internet) or private networks, that the users can a search to find answers and information to specific or general queries about some topic or issue. For example, organizations often collect large number of documents that constitute a repository of information, be it administrative of technical information, that the organization's employees may access and perform searches on.”, and Par. 0145:” … In some embodiments, the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates.”, and Par. 0146:” … Such derived questions can then be re-submitted to the query processing module to retrieve follow-up answers. This process can be iteratively repeated up to a pre-determined number of times.) Note: past questions (queries) read on natural language input, furthermore, iteratively repeated up reads on storing the answer.
Meteer, does not teach, however, Cash teaches obtaining a group of recommended data item queries, the group of recommended data item queries including text extracted from at least one knowledge base document of a set of knowledge base documents; (Cash, Par. 0004:” The ability of a digital assistant system to produce satisfactory responses to user requests depends on the natural language processing, knowledge base, and artificial intelligence available to the digital assistant system.”, and Par. 0085:” As described above, in order to complete a structured query, task flow processor 342 sometimes needs to initiate additional dialogue with the user in order to obtain additional information, and/or disambiguate potentially ambiguous utterances. When such interactions are necessary, task flow processor 342 invokes dialogue processing module 340 (sometimes herein also called a "dialogue processor") to engage in a dialogue with the user. In some embodiments, dialogue processor 340 determines how (and/or when) to ask the user for the additional information, and receives and processes the user responses. The questions are provided to and answers are received from the users through I/O processing module 328. In some embodiments, dialogue processor 340 presents dialogue output to the user via audio and/or visual output, and receives input from the user via spoken or physical (e.g., clicking) responses. Continuing with the example above, when task flow processor 342 invokes dialogue flow processor 340 to determine the "party size" and "date" information for the structured query associated with the domain "restaurant reservation," dialogue flow processor 335 generates questions, such as "For how many people?" and "On which day?," to pass to the user. Once answers are received from the user, the dialogue flow processor 340 can then populate the structured query with the missing information, or pass the information to task flow processor 342 to complete the missing information from the structured query.”, Par. 0086:” … For example, the task flow model for the actionable intent of “restaurant reservation,” optionally, includes steps and instructions for contacting a restaurant and actually requesting a reservation for a particular party size at a particular time. For example, using a structured query, such as {restaurant reservation, restaurant=ABC Café, date=3/12/2012, time=7 PM, party size=5}, task flow processor 342, optionally, performs the steps of: (1) logging onto a server [knowledge base] of the ABC Café or a restaurant reservation system such as OPENTABLE®; (2) entering the date, time, and party size information in a form on the website; (3) submitting the form; and (4) making a calendar entry for the reservation in the user's calendar.)”)
Cash is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Meteer further in view of Cash to obtain a group of recommended data item queries, the group of recommended data item queries including text extracted from at least one knowledge base document of a set of knowledge base documents. Motivation to do so would prevent the system from guessing and help it to get exact, and correct information.
Regarding claim 9, Meteer, as modified above, teaches the method of claim 8.
Meteer, as modified above, further teaches wherein obtaining the group of recommended data item queries comprises: identifying a knowledge base document of the set of knowledge base documents; (Meteer, Par. 0006:” … identifying possible alternative queries [second request] to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches (e.g., imposing requirement that at least a certain percentage of query answers be determined from particular data collections, etc.), and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinements read on identifying a knowledge base document.
generating a data item query prompt comprising predetermined third query prompt text, text from the knowledge base document, and the natural language input; (Meteer, Par. 0006:” … identifying possible alternative queries [second request] to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches (e.g., imposing requirement that at least a certain percentage of query answers be determined from particular data collections, etc.), and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinements read on 2nd, 3rd queries.
providing the data item query prompt to the generative output engine; and (Meteer, Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”)
receiving, from the generative output engine, at least a subset of the group of recommend data item queries. (Meteer, Par. 0006:” … identifying possible alternative queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches …, and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinement reads on answer to the composite data item query.
Regarding claim 10, Meteer, as modified above, teaches the method of claim 8.
Meteer, as modified above, further teaches prior to ranking the group of recommended data item queries, removing, from the group of recommended data item queries, data item queries for which answers were included in the natural language input. (Meteer, Par. 0191:” … may include identifying multiple question-answer pairs relevant to the first query that are additionally associated with concepts at a next lower hierarchical level than the first concept, and presenting to the user at least some of the multiple question-answer pairs. Modifying the query to include drilled-down query terms may include modifying the query based on a question portion of a question-answer pair selected from the at least some of the multiple question-answer pairs.”) Note: “presenting to the user at least some of the multiple question-answer pairs” read on removal of queries and deciding to remove those including or not is a design choice.
Regarding claim 11, Meteer, as modified above, teaches the method of claim 8.
Meteer, as modified above, further teaches prior to ranking the group of recommended data item queries: analyzing the group of recommended data item queries to identify redundant data item queries that are directed to eliciting same information; (Meteer, Par. 0211:” … After executing the query chunks (by the Q-A system), some of those query chunks will yield relevant answers (with some of those relevant answers nevertheless containing redundant information resulting from potential overlap between parsed queries), but pooling the answers together will more likely give a relevant answer as opposed to getting a relevant answer from the original verbose query (in the hope that the question-answer system can infer meaning out of a long sentence).”)
generating a representative data item query for the redundant data item queries; and (Meteer, Par. 0211:” … After executing the query chunks (by the Q-A system), some of those query chunks will yield relevant answers (with some of those relevant answers nevertheless containing redundant information resulting from potential overlap between parsed queries), but pooling the answers together will more likely give a relevant answer as opposed to getting a relevant answer from the original verbose query (in the hope that the question-answer system can infer meaning out of a long sentence).“) Note: “pooling the answer together will more likely give a relevant answer as opposed to getting a relevant answer from the original verbose query” reads on generating a representative data item query for the redundant data item.
including the representative data item query in the group of recommended data item queries. (Meteer, Par. 0157:” FIG. 3 further illustrates a drill down query modification in which, upon a determination that the query is not sufficiently specific, or is missing some query information (concepts), the drilldown framework determines (possibly using feedback from the user, such as disambiguation feedback) more detailed concepts that are needed to produce query answers satisfying matching criteria. For example, consider a query asking “how do I remove the battery in my 3000 Inspiron.” The backoff/drilldown framework is configured to determine (e.g., based on a machine learning implementation trained to determine the specific details that are usually associated with different concepts in an ontology) that battery removal (the concept identified from the query) corresponds to information typically given with respect to specific laptop model, and to further determine that the specific model number is missing from the query. The framework can then activate a disambiguation process that requests further specificity from the user, or alternatively may infer the missing information (e.g., the model number) based on contextual information (e.g., based on previously asked queries that included the model number, based on the type of laptop through which the query was submitted, etc.) In the example of FIG. 3, the framework executes an interactive disambiguation dialogue that asks the user to specify the model for which laptop's battery is to be removed (as illustrated by arrow 304).”)
Regarding claim 12, Meteer, as modified above, teaches the method of claim 11.
Meteer, as modified above, further teaches wherein generating the representative data item query for the redundant data item queries comprises: generating a representative data item query prompt comprising predetermined third query prompt text and text from the redundant data item queries; (Meteer, Par. 0211:” … After executing the query chunks (by the Q-A system), some of those query chunks will yield relevant answers (with some of those relevant answers nevertheless containing redundant information resulting from potential overlap between parsed queries), but pooling the answers together will more likely give a relevant answer as opposed to getting a relevant answer from the original verbose query (in the hope that the question-answer system can infer meaning out of a long sentence).“) Note: “pooling the answer together will more likely give a relevant answer as opposed to getting a relevant answer from the original verbose query” reads on generating a representative data item query for the redundant data item.
providing the representative data item query prompt to the generative output engine; and (Meteer, Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”)
receiving the representative data item query from the generative output engine. (Meteer, Par. 0141:” … using natural language processing or a learning machine implementation) but have different concept/category values, intermediary output may be provided to the user (e.g., as a visual disambiguation prompt, or an audio disambiguation prompt) requesting the user to provide clarification information specifying which of the identified concepts is more relevant to the user's query.”, and Par. 0169:” … the user interacting with the Q-A system may cause another search to be executed at a lower level (e.g., providing to the system additional disambiguation information). This process is basically a natural language interaction.”).
Regarding claim 13, Meteer, as modified above, teaches the method of claim 8.
Meteer, as modified above, further teaches wherein the graphical user interface is a support request graphical user interface associated with the client application. (Meteer, Par. 0066:” … queries submitted to the Q-A system [client application], contextual data associated to the query, and/or contextual data associated with a user submitting the query.”, and Par. 0095:” A user associated with the customer that provided that document library (e.g., an employee of the customer) can subsequently submit a query (e.g., a natural language query, such as “how many vacation days does an employee with 2 years seniority get a year?”) that is processed by the system 100, …”, and Par. 0170:” … This mechanism is referred to as visual QA. Because the answers for the next lower level are displayed, the user is given the option to select answers resulting from a deterministic process.”, and Par. 0254:” … data may be dynamically adjustable (e.g., can be specified during runtime) via an interactive user interface (such as interactive user query interface 130 of FIG. 1).”)
Regarding claim 14, Meteer, as modified above, teaches the method of claim 8.
Meteer, as modified above, further teaches generating an issue record including the answer to the at least one data item query; and storing the issue record in association with an issue tracking system. (Meteer, Par. 0003:” Computer users often have access to vast amounts of data, whether accessible through public networks (such as the Internet) or private networks, that the users can a search to find answers and information to specific or general queries about some topic or issue. For example, organizations often collect large number of documents that constitute a repository of information, be it administrative of technical information, that the organization's employees may access and perform searches on.”, and Par. 0145:” … In some embodiments, the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates.”, and Par. 0146:” … Such derived questions can then be re-submitted to the query processing module to retrieve follow-up answers. This process can be iteratively repeated up to a pre-determined number of times.) Note: past questions (issues) read on issue record, furthermore, iteratively repeated up reads on issue tracking.
Regarding claim 15, Meteer teaches a system of a support platform, the system comprising: one or more processing units; computer readable memory storing computer readable instructions that when executed by the one or more processing units cause the system to: (Meteer, Par. 0023:” … a first non-transitory computer readable media is provided, that is programmed with instructions, executable on one or more processors of a computing system, to determine by a searching system, …”).
receive a natural language input, the natural language input provided to a support request graphical user interface of a client application operating on a client device; (Meteer, Par. 0066:” … queries submitted to the Q-A system [client application], contextual data associated to the query, and/or contextual data associated with a user submitting the query.”, and Par. 0095:” A user associated with the customer that provided that document library (e.g., an employee of the customer) can subsequently submit a query (e.g., a natural language query, such as “how many vacation days does an employee with 2 years seniority get a year?”) that is processed by the system 100, …”, and Par. 0170:” … This mechanism is referred to as visual QA. Because the answers for the next lower level are displayed, the user is given the option to select answers resulting from a deterministic process.”, and Par. 0254:” … data may be dynamically adjustable (e.g., can be specified during runtime) via an interactive user interface (such as interactive user query interface 130 of FIG. 1).”)
analyze the set of recommended data item queries to identify a subset of data item queries for which answers were not included in the natural language input, the analyzing comprising, for a recommended data item query of the set of recommended data item queries: (Meteer, Par. 0157:” … The framework can then activate a disambiguation process that requests [eliciting] further specificity from the user, or alternatively may infer the missing information [data item query] (e.g., the model number) based on contextual information (e.g., based on previously asked queries that included the model number, based on the type of laptop through which the query was submitted, etc.) In the example of FIG. 3, the framework executes an interactive disambiguation dialogue that asks the user to specify the model for which laptop's battery is to be removed (as illustrated by arrow 304).”) Note: “second prompt” reads on question (string) for asking from the user.
generating a second prompt comprising predetermined query prompt text, the recommended data item query, and the natural language input; (Meteer, Par. 0157:” … The framework can then activate a disambiguation process that requests [query prompt] further specificity from the user, or alternatively may infer the missing information (e.g., the model number) based on contextual information (e.g., based on previously asked queries that included the model number, based on the type of laptop through which the query was submitted, etc.) In the example of FIG. 3, the framework executes an interactive disambiguation dialogue that asks the user to specify the model for which laptop's battery is to be removed (as illustrated by arrow 304).”) Note: “second prompt” reads on question (string) for asking from the user.
providing the second prompt to a generative output engine; (Meteer, Par. 0006:” … identifying possible alternative queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches …, and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”)
receiving, from the generative output engine, a generative response; and (Meteer, Par. 0006:” … identifying possible alternative queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches …, and mitigating various ambiguity issues.”, and Par. 0141:” … using natural language processing or a learning machine implementation) but have different concept/category values, intermediary output may be provided to the user (e.g., as a visual disambiguation prompt, or an audio disambiguation prompt) requesting the user to provide clarification information specifying which of the identified concepts is more relevant to the user's query.”, and Par. 0169:” … the user interacting with the Q-A system may cause another search to be executed at a lower level (e.g., providing to the system additional disambiguation information). This process is basically a natural language interaction.”) Note: iterative refinement reads on receiving from the generative output engine.
analyzing the generative response to determine whether an answer to the recommended data item query was included in the natural language input; (Meteer, Par. 0092:” … analyzing a query, prior to executing the query, to determine, and if possible, resolve, various issues of ambiguity related to the submitted query. For example, a relevance detector performs a quick ambiguity/relevance classifications on queries to determine if a particular query is too ambiguous to be processed, or has enough clarity to return answer results with sufficient confidence in the quality of the answers (e.g., confidence that the answers returned are not nonsensical).”) Note: “query is too ambiguous” reads on answer to the recommended data item query was included.
generate a composite data item query from at least a first data item query and a second data item query from the subset of data item queries, the composite data item query configured to elicit information for the first data item query and the second data item query; generate a natural language query string for eliciting an answer to the composite data item query; (Meteer, Par. 0006:” … identifying possible alternative queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches …, and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinement reads on composite query.
receive a response including the answer to the composite data item query; and (Meteer, Par. 0006:” … identifying possible alternative queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches …, and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinement reads on answer to the composite data item query.
store the answer to the composite data item query in association with the natural language input. (Meteer, Par. 0003:” Computer users often have access to vast amounts of data, whether accessible through public networks (such as the Internet) or private networks, that the users can a search to find answers and information to specific or general queries about some topic or issue. For example, organizations often collect large number of documents that constitute a repository of information, be it administrative of technical information, that the organization's employees may access and perform searches on.”, and Par. 0145:” … In some embodiments, the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates.”, and Par. 0146:” … Such derived questions can then be re-submitted to the query processing module to retrieve follow-up answers. This process can be iteratively repeated up to a pre-determined number of times.) Note: past questions (queries) read on natural language input, furthermore, iteratively repeated up reads on storing the composite data item query.
Meteer, does not teach, however Cash teaches obtain a set of recommended data item queries; (Cash, Par. 0085:” As described above, in order to complete a structured query, task flow processor 342 sometimes needs to initiate additional dialogue with the user in order to obtain additional information, and/or disambiguate potentially ambiguous utterances. When such interactions are necessary, task flow processor 342 invokes dialogue processing module 340 (sometimes herein also called a "dialogue processor") to engage in a dialogue with the user. In some embodiments, dialogue processor 340 determines how (and/or when) to ask the user for the additional information, and receives and processes the user responses. The questions are provided to and answers are received from the users through I/O processing module 328. In some embodiments, dialogue processor 340 presents dialogue output to the user via audio and/or visual output, and receives input from the user via spoken or physical (e.g., clicking) responses. Continuing with the example above, when task flow processor 342 invokes dialogue flow processor 340 to determine the "party size" and "date" information for the structured query associated with the domain "restaurant reservation," dialogue flow processor 335 generates questions, such as "For how many people?" and "On which day?," to pass to the user. Once answers are received from the user, the dialogue flow processor 340 can then populate the structured query with the missing information, or pass the information to task flow processor 342 to complete the missing information from the structured query.”)
Cash is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Meteer further in view of Cash to obtain a set of recommended data item queries. Motivation to do so would prevent the system from guessing and help it to get exact, and correct information.
Regarding claim 16, Meteer, as modified above, teaches the system claim of 15.
Meteer, as modified above, further teaches generating the natural language query string comprises: generating a question generation prompt comprising predetermined second query prompt text and the composite data item query; (Meteer, Par. 0006:” … identifying possible alternative queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches …, and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinement reads on composite query.
providing the question generation prompt to the generative output engine; and (Meteer, Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”)
receiving the natural language query string from the generative output engine. (Meteer, Par. 0006:” … identifying possible alternative queries to those submitted by the user, or correcting deficiencies in the submitted queries. Those solutions include iterative refinements through disambiguation techniques, re-writing queries to generate supplemental queries, using preference information to focus on particular data collections when performing searches …, and mitigating various ambiguity issues.”, and Par. 0218:” … provide the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”, and Par. 0220:” … generative LLM engines configured to receive a first query and a command …”, and Par. 0225:” … For the generative LLM approach, deriving one or more secondary queries may include generating a paraphrase message comprising the first query and one or more required output characteristics for a resultant query paraphrasing of the first query, and providing the paraphrase message to a generative large language model system configured to generate a paraphrased query based on the first query and the one or more required output characteristics.”) Note: iterative refinement reads on answer to the composite data item query.
Regarding claim 19, Meteer, as modified above, teaches the system claim of 15.
Meteer, as modified above, further teaches wherein the computer readable instructions further cause the system to: generate an issue record including the answer to the recommended data item query; and store the issue record in association with an issue tracking system. (Meteer, Par. 0003:” Computer users often have access to vast amounts of data, whether accessible through public networks (such as the Internet) or private networks, that the users can a search to find answers and information to specific or general queries about some topic or issue. For example, organizations often collect large number of documents that constitute a repository of information, be it administrative of technical information, that the organization's employees may access and perform searches on.”, and Par. 0145:” … In some embodiments, the matching of query data to the past questions and associated answers stored in cache is performed by computing a score that is based on the combination of the questions and their answers, and ranking the computed scores to identify one or more likely matching candidates.”, and Par. 0146:” … Such derived questions can then be re-submitted to the query processing module to retrieve follow-up answers. This process can be iteratively repeated up to a pre-determined number of times.) Note: past questions (issues) read on issue record, furthermore, iteratively repeated up reads on issue tracking.
Regarding claim 20, Meteer, as modified above, teaches the system claim of 15.
Meteer, as modified above, further teaches prior to analyzing the set of recommended data item queries to identify the subset of data item queries for which answers were not included in the natural language input, remove, from the set of recommended data item queries, recommended data item queries that are not supported by a support request interaction type associated with the natural language input. (Meteer, Par. 0191:” … may include identifying multiple question-answer pairs relevant to the first query that are additionally associated with concepts at a next lower hierarchical level than the first concept, and presenting to the user at least some of the multiple question-answer pairs. Modifying the query to include drilled-down query terms may include modifying the query based on a question portion of a question-answer pair selected from the at least some of the multiple question-answer pairs.”) Note: “presenting to the user at least some of the multiple question-answer pairs” read on removal of queries.
Claims 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Meteer and Cash, and in further view of Abboud et al. (US 20140258050 A1)(herein " Abboud ").
Regarding claim 17, Meteer, as modified above, teaches the system claim of 15.
Meteer, as modified above, further teaches wherein obtaining the set of recommended data item queries comprises: receiving an identifier of the client application; and (Meteer, Par. 0066:” … queries submitted to the Q-A system [client application], contextual data associated to the query, and/or contextual data associated with a user submitting the query.”, and Par. 0095:” A user associated with the customer that provided that document library (e.g., an employee of the customer) can subsequently submit a query (e.g., a natural language query, such as “how many vacation days does an employee with 2 years seniority get a year?”) that is processed by the system 100, …”). Note: (Q-A) system represents the identifier.
the service selects the set of recommended data item queries based on the identifier of the client application. (Meteer, Par. 0175:” … Alternatively, if the output is presented as candidate Q-A pairs, selection of one of the presented choices will cause modification of the query to a modified query that may include, or is based on, the pre-determined question of the selected Q-A pair.”, and Par. 0182:” Selecting from a list of prompts that have been written in advance, …”).
Meteer, as modified above, does not teach, however, Abboud teaches submitting a request to a service to obtain the set of recommended data item queries, the request including the identifier of the client application; and (Abboud, Par. 0030:” … submit a request for item recommendations through a computing device, for example, a mobile smart phone, programmed with an application program configured to submit such requests. In response to this request, a server system analyzes the inventory in search of types of items with sufficient quantity on hand to stock the display and, optionally, that satisfy other selection criteria.”)
Abboud is considered to be analogous to the claimed invention because it is in the same field of endeavor. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Meteer, as modified above, further in view of Abboud to submit a request to a service to obtain the set of recommended data item queries, the request including the identifier of the client application. Motivation to do so would provide an automated optimization and personalization, which eliminates the need for users or client applications to manually construct complex query parameters.
Regarding claim 18, Meteer, as modified above, teaches the system claim of 17.
Meteer, as modified above, does not teach, however, Abboud further teaches wherein obtaining the set of recommended data item queries further comprises submitting an additional request to an additional service to obtain additional recommended data item queries, the additional recommended data item queries associated with a term included in the natural language input. (Abboud, Par. 0030:” … submit a request [additional request] for item [additional] recommendations through a computing device, for example, a mobile smart phone, programmed with an application program configured to submit such requests [input]. In response to this request, a server system analyzes the inventory in search of types of items with sufficient quantity on hand to stock the display and, optionally, that satisfy other selection criteria.”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Amthor et al. (US 20260177801 A1) teaches in Par. 0064:” In the event of a negative evaluation (that is to say, if the information is insufficient), the LLM asks the user follow-up questions in order to obtain the missing experiment-specific information needed to determine the parameters of the focus strategy. Alternatively or additionally, the LLM can ask follow-up questions to clarify the focus strategy or to clarify previous information. As part of the follow-up question, the model can specify a list of the required sample properties or, alternatively, query characteristics of the experiment or of the sample carrier from which the LLM or the machine-learned model can derive the required parameters.”
Examiner's Note: Examiner has cited particular columns and line numbers and/or paragraph numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
In the case of amending the Claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DARIOUSH AGAHI whose telephone number is (408)918-7689. The examiner can normally be reached Monday - Thursday and alternate Fridays, 7:30-4:30 PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached on 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DARIOUSH AGAHI, P.E.
Primary Examiner
/DARIOUSH AGAHI/Primary Examiner, Art Unit 2656
Prosecution Insights