DETAILED ACTION
This communication is in response to the Amendments and Arguments filed on 02/02/2026.
Claims 3-4 have been canceled by the Applicant.
Claim(s) 1-2 and 5-20 are pending and have been examined. Hence, this action has been made FINAL.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments and Amendments
Amendments to the claims by the Applicant have been considered and addressed below.
With respect to the 35 USC § 101 and 102/103 rejections, the Applicant provides several arguments in which the Examiner will respond accordingly, below.
35 USC § 101 rejection(s)
Arguments:
Claims 1-20 stand rejected under 35 U.S.C. § 101 as allegedly being directed to an abstract idea without significantly more. In an effort to advance prosecution, and without conceding the propriety of the rejections, various claims are amended herein as set forth above. For example, independent claim 1 is amended to recite, in part, that "the LLM is trained or fine-tuned based on training examples of NL based inputs and corresponding refined prompts," as suggested by the Examiner during the interview. Along similar lines, dependent claims 13-16 are amended to make the training or fine- tuning affirmative steps. The undersigned respectfully requests reconsideration of the Office Action's 101 rejection at least in view of the amendments included herein.
Examiner’s Response to Arguments:
Arguments have been considered but these are not persuasive. The Examiner respectfully disagrees with the arguments of the limitation of "the LLM is trained or fine-tuned based on training examples of NL based inputs and corresponding refined prompts," overcoming the 35 U.S.C. § 101 (abstract idea) rejection(s). During the interview held in 01/27/2026, the Examiner indicated that more details regarding how the training of the LLM is performed along with tying the trained model performing specific actions recited in the claims would help overcoming the rejection. However, the claim language as currently amended does not provide such details.
For more details, please refer to updated 35 U.S.C. § 101 rejections for claims 1-2 and 5-20 below.
35 USC § 102/103 rejection(s)
Arguments:
35 U.S.C. § 102 Rejection(s)
Claims 1-5, 7-9, 11-13, 15-16, and 19-20 currently stand rejected under 35 U.S.C. § 102(a)(1) as allegedly being anticipated by Zhang (U.S. Pat. No. 12,038,918). As noted previously, the Examiner indicated during the interview that if the independent claims were amended as discussed during the interview, beyond the original amendments proposed prior to the interview, the rejections of the independent claims might be overcome, pending further consideration and/or searching. Without conceding the propriety of these rejections, the independent claims are amended as set forth above consistent with those discussions. Accordingly, the undersigned respectfully requests that the rejections of the independent claims be withdrawn. The undersigned requests that the rejections of the dependent claims be withdrawn at least by virtue of their dependencies from the independent claims.
35 U.S.C. § 103 Rejection(s)
Claim 18 stands rejected under 35 U.S.C. § 103 as allegedly being unpatentable over Zhang, in view of Fu (U.S. Pub. No. 2025/0190604). Claim 17 stands rejected under 35 U.S.C. § 103 as allegedly being unpatentable over Zhang, in view of Fu. Claims 6, 10, and 14 stand rejected under 35 U.S.C. § 103 as allegedly being unpatentable over Zhang, in view of Shen ("Promptor: A Conversational and Autonomous Prompt Generation Agent for Intelligent Text Entry Techniques"). The undersigned requests that the rejections of the dependent claims be withdrawn at least by virtue of their dependencies from the independent claims.
Examiner’s Response to Arguments:
Applicant’s arguments with respect to independent claim(s) 1 and 18-20 under 35 U.S.C. § 102/103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Zhang et al. (US 12038918 B1) and further in view of Hong et al. (US 20240354513 A1)
For more details, please refer to updated 35 U.S.C. § 103 rejections for claims 1-2 and 5-20, below.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1-2 and 5-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. More specifically directed to the abstract idea grouping of: mental process.
The independent claim(s) 1 and 19-20 recite(s):
1. A method implemented by one or more processors, the method comprising:
receiving natural language (NL) based input associated with a client device, wherein the NL based input lacks one or more details for completing an intended task;
generating a refined input prompt corresponding to the NL based input based on first large language model (LLM) output generated based on processing at least the NL based input and context data using an LLM, wherein the refined input prompt comprises a generative rewrite of the NL based input that includes one or more of the details that were lacking from the NL based input, and wherein the LLM is trained or fine-tuned based on training examples of NL based inputs and corresponding refined prompts;
causing the refined input prompt to be rendered at the client device;
responsive to user input received at the client device indicative of an acceptance of the refined input prompt, generating responsive content to the NL based input based on second LLM output generated based on processing the refined input prompt using the LLM; and
causing the responsive content to the NL based input to be rendered at the client device.
19. A system comprising:
one or more hardware processors; and
memory storing instructions that, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations according to the method of claim 1.
20. A non-transitory computer-readable storage medium storing instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to perform operations according to the method of claim 1.
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving text or speech (e.g., question/query) from another human, wherein the received text or speech is missing specific details (e.g., required step, task, or instructions);
Re-writing or repeating the speech in other words (e.g., paraphrased) using a predetermined set of rules considering context of the text/speech to include the missing specific details, and wherein the re-writing or repeating the speech in other words (e.g., paraphrased) is done using a predetermined set of rules;
Displaying the re-written text on paper as a form of question for clarification from the other human;
Receiving a response/clarification from the other human and determining a response to the question/query/text;
Writing down the response to said question/query/text.
The independent claim(s) 18 recite(s):
18. A method implemented by one or more processors, the method comprising:
obtaining one or more training examples wherein each training example comprises a NL based input and a corresponding refined input prompt;
fine-tuning, based on the one or more training examples, an LLM to generate, based on a given NL based input associated with a client device, a corresponding refined input prompt usable to, in response to user input received at the client device indicative of an acceptance of the corresponding refined input prompt, generate responsive content to the given NL based input based on LLM output generated based on processing the corresponding refined input prompt using the LLM, the responsive content to be caused to be rendered at the client device, wherein the refined input prompt comprises a generative rewrite of the NL based input that includes one or more of details lacking from the given NL based input.
of claim 1.
This reads on a human (e.g., mentally and/or using pen and paper):
Using predefined examples of questions that can be asked and their corresponding variations;
Redefining the predetermined set of rules to:
Receiving text or speech (e.g., question/query) from another human;
Re-writing or repeating the speech in other words (e.g., paraphrased) using a predetermined set of rules;
Displaying the re-written text on paper as a form of question for clarification from the other human;
Receiving a response/clarification from the other human and determining a response to the question/query/text;
Writing down the response to said question/query/text to include any missing specific details from the received text/speech.
This judicial exception is not integrated into a practical application because for example: claims 1 and 18-20 recite at least “one or more processors”, “a memory”, and “a client device”. As an example, in [0020 and 0068] of the as filed specification, it is disclosed: “[0020] The client device 110 can be, for example, one or more of: a desktop computer, a laptop computer, a tablet, a mobile phone, a computing device of a vehicle (e.g., an in-vehicle communications system, an in-vehicle entertainment system, an in-vehicle navigation system), a standalone interactive speaker (optionally having a display), a smart appliance such as a smart television, and/or a wearable apparatus of the user that includes a computing device (e.g., a watch of the user having a computing device, glasses of the user having a computing device, a virtual or augmented reality computing device). Additional and/or alternative client devices may be provided. […] [0068] … This system of the method 700A includes one or more processors, memory, and/or other component(s) of computing device(s) (e.g., client device 110, NL based response system 120, computing device 810, one or more servers, and/or other computing devices)…” . Therefore, a general-purpose computer or computing device is described and mainly used as an application thereof. Accordingly, these additional elements do not integrate the abstract idea into a practical idea because it does not impose any meaningful limits on practicing the abstract idea.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of using a computer is listed as a general computing device as noted. The claim is not patent eligible.
With respect to claim 2, the claim(s) recite:
2. The method of claim 1, wherein generating the refined input prompt is based on processing at least (i) the NL based input and (ii) one or more exemplary refined input prompts using the LLM.
This reads on a human (e.g., mentally and/or using pen and paper):
Performing actions wherein the re-writing or repeating the speech in other words (e.g., paraphrased) is done using a predetermined set of rules
No additional limitations are present.
With respect to claim 5, the claim(s) recite:
5. The method of claim 1, wherein the context data comprises user data associated with a user of the client device.
This reads on a human (e.g., mentally and/or using pen and paper):
Performing actions wherein the predetermined set of rules used are editing according to specific/pre-known information.
No additional limitations are present.
With respect to claim 6, the claim(s) recite:
6. The method of claim 1, wherein the NL based input is received as part of a multi-turn dialog with a user of the client device, and wherein the context data comprises historical data associated with one or more of the previous turns of the multi-turn dialog.
This reads on a human (e.g., mentally and/or using pen and paper):
Considering multiple replies between two human over a period of time.
No additional limitations are present.
With respect to claim 7, the claim(s) recite:
7. The method of claim 1, further comprising:
selecting one or more terms of the refined input prompt to be user selectable terms; and
causing an indication of the user selectable terms from among the refined input prompt to be rendered at the client device.
This reads on a human (e.g., mentally and/or using pen and paper):
Giving options to the other human for selection and
Displaying said options in a piece of paper.
No additional limitations are present.
With respect to claim 8, the claim(s) recite:
8. The method of claim 7, wherein selecting the one or more terms of the refined input prompt to be user selectable terms is based on metadata included in the first LLM output.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the options are selected from a predefined source (e.g., book, dictionary).
No additional limitations are present.
With respect to claim 9, the claim(s) recite:
9. The method of claim 7, further comprising:
responsive to detecting user interaction with a particular user selectable term of the one or more user selectable terms at the client device, causing one or more alternative terms corresponding to the particular user selectable term to be rendered at the client device; and
responsive to detecting user selection of a particular alternative term at the client device, replacing the particular user selectable term with the particular alternative term in the refined input prompt to generate an updated refined input prompt,
wherein the responsive content is generated based on processing the updated refined input prompt using the LLM in response to user input received at the client device indicative of an acceptance of the updated refined input prompt.
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving a selection as a response/clarification from the other human and determining a response to the question/query/text;
Displaying said selection on a piece of paper;
Receiving another selection as a response/clarification from the other human and determining a response to the question/query/text;
Displaying said updated selection on a piece of paper;
Writing down the response to said question/query/text.
No additional limitations are present.
With respect to claim 10, the claim(s) recite:
10. The method of claim 9, further comprising:
storing an indication of the selection of the alternative term for use in generating subsequent refined input prompts using the LLM.
This reads on a human (e.g., mentally and/or using pen and paper):
Writing down the selection of the other human to consider it in the future.
No additional limitations are present.
With respect to claim 11, the claim(s) recite:
11. The method of claim 1, further comprising:
causing a graphical user interface element to be rendered at the client device, wherein the refined input prompt is generated in response to user selection of the graphical user interface element at the client device.
This reads on a human (e.g., mentally and/or using pen and paper):
Writing down on a piece of paper for display the response/clarification from the other human.
No additional limitations are present.
With respect to claim 12, the claim(s) recite:
12. The method of claim 11, further comprising:
determining whether to cause rendering of the graphical user interface element based on a quality metric of the NL based input and/or based on determining that the NL based input comprises harmful content.
This reads on a human (e.g., mentally and/or using pen and paper):
Determining whether to write down the response/clarification from the other human based on a predefined set of rules.
No additional limitations are present.
With respect to claim 13, the claim(s) recite:
13. The method of claim 1, wherein the responsive content is first responsive content, the method further comprising:
generating second responsive content to the NL based input based on third LLM output generated based on processing the NL based input using the LLM;
causing the second responsive content to the NL based input to be rendered at the client device;
detecting user input indicative of a selection of one of the first responsive content and the second responsive content at the client device; and
in response to the first responsive content being selected:
based on the refined input prompt and the NL based input together as a training example, training or fine-tuning the LLM to generate, based on a given NL based input, a corresponding refined input prompt.
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving a selection as a response/clarification from the other human and determining a response to the question/query/text;
Displaying said selection on a piece of paper;
Receiving another selection as a response/clarification from the other human and determining a response to the question/query/text;
Displaying said updated selection on a piece of paper;
Writing down the response to said question/query/text.
No additional limitations are present.
With respect to claim 14, the claim(s) recite:
14. The method of claim 1, further comprising:
modifying the refined input prompt based on user input received at the client device to generate an updated refined input prompt;
responsive to user input received at the client device indicative of an acceptance of the updated refined input prompt, generating responsive content to the NL based input based on processing the updated refined input prompt using the LLM;
causing the responsive content to the NL based input to be rendered at the client device; and
based on the updated refined input prompt and the NL based input together as a training example, training or in fine-tuning the LLM to generate, based on a given NL based input, a corresponding refined input prompt.
This reads on a human (e.g., mentally and/or using pen and paper):
Modifying the received response/clarification from the other human to determine an updated response;
Receiving another response/clarification from the other human and determining a response to the question/query/text;
Writing down the response to said question/query/text.
No additional limitations are present.
With respect to claim 15, the claim(s) recite:
15. The method of claim 1, further comprising:
responsive to the user input received at the client device indicative of the acceptance of the refined input prompt;
based on the refined input prompt and the NL based input together as a training example, training or fine-tuning the LLM to generate, based on a given NL based input, a corresponding refined input prompt.
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving another response/clarification from the other human and determining a response to the question/query/text;
Writing down the response to said question/query/text.
No additional limitations are present.
With respect to claim 16, the claim(s) recite:
16. The method of claim 15, further comprising:
responsive to a user input received at the client device indicative of a request to revert to the NL based input, bypassing the training or fine-tuning based on storing the refined input prompt and the NL based input as a training example.
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving another response/clarification from the other human (i.e., request to revert);
Determining not to write down the response to said question/query/text.
No additional limitations are present.
With respect to claim 17, the claim(s) recite:
17. The method of claim 1, further comprising:
fine-tuning the LLM to generate, based on a given NL based input, a corresponding refined input prompt, based on one or more training examples, wherein each training example comprises a NL based input and a corresponding refined input prompt.
This reads on a human (e.g., mentally and/or using pen and paper):
Using predefined examples of questions that can be asked and their corresponding variations and redefining the predetermined set of rules.
No additional limitations are present.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 1-2, 5, 7-9, 11-13, 15-16, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 12038918 B1) and further in view of Hong et al. (US 20240354513 A1).
As to independent claim 1, Zhang et al. teaches:
1. A method implemented by one or more processors (see ¶ Col. 1, lines 30-67: “(3) In general, in one aspect, one or more embodiments relate to a method that includes receiving an original query in a user interface, generating an ambiguity query from the original query, and sending, via an application programming interface (API) of a large language model, the ambiguity query to the large language model... […] (5) In general, in one aspect, one or more embodiments relate to a system that includes at least one computer processor, a user interface comprising a query input widget to receive an original query, and a prompt manager executing on the at least one computer processor configured to perform operations…”), the method comprising:
receiving natural language (NL) based input associated with a client device (see ¶ Col. 2, lines 59-63: “(5) In general, in one aspect, one or more embodiments relate to a system that includes at least one computer processor, a user interface comprising a query input widget to receive an original query, and a prompt manager executing on the at least one computer processor configured to perform operations...” and ¶ Col. 10, lines 44-51: “(50) The input devices (510) may include a touchscreen, keyboard, mouse, microphone, touchpad, electronic pen, or any other type of input device. The input devices (510) may receive inputs from a user that are responsive to data and messages presented by the output devices (508). The inputs may include text input, audio input, video input, etc., which may be processed and transmitted by the computing system (500) in accordance with the disclosure…”);
generating a refined input prompt corresponding to the NL based input based on first large language model (LLM) output generated based on processing at least the NL based input using an LLM (see ¶ Col. 3, lines 38-44: “(15) A prompt manager (104) is interposed between the user interface (100) and the LLM (106). The prompt manager (104) is configured to detect an ambiguous query and suggest revisions for the ambiguous query. In one or more embodiments, the prompt manager includes an ambiguity example dataset (108), an ambiguity detector (110), a perturbed query creator (112), and a query coach (114).”,
¶ Col. 4, lines 19: “(18) The ambiguity detector (110) is further configured to receive a binary response (120) from the LLM (106) and detect, based at least in part on the binary response (120). The binary response (120) is the response that has one of two possible values. The first value indicates that the query encapsulated by the ambiguity query is detected as ambiguous by the LLM (106) while the second value indicates that the query encapsulated by the ambiguity query is detected as unambiguous by the LLM (106). The LLM (106) may further be configured to output a confidence value. The confidence value is detected probability that the binary value is correct. Specifically, the confidence value is the degree of confidence that the LLM is correct in predicting the query as ambiguous or unambiguous”
and ¶ Col. 4, line 47 – Col. 5, line 11: “(20) Continuing with FIG. 1, the prompt manager (104) further includes a query coach (114). The query coach (114) is configured to determine, using the perturbed query creator (112) and the ambiguity detector (110), the locations of the original query that are ambiguous and obtain clarification for the ambiguous query. Specifically, the query coach (114) is configured to obtain a clarification for the original query (118) to generate a revised query (122). The clarification may have more specificity in the original query (118) through adjectives, different word choices, or clauses. In one or more embodiments, the query coach (114) is configured to generate a clarification request for the ambiguous location in the original query. The clarification request is a request to the user to clarify a portion of the original query. For example, the clarification request may be a question or a suggestion to clarify a particular term in the original query that is presented in the user interface (100). For example, the clarification request may be a list of suggestions from which the user may accept one or add the user's own clarification. As another example, the clarification request may be a question presented to the user.
(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”);
PNG
media_image1.png
696
420
media_image1.png
Greyscale
causing the refined input prompt to be rendered at the client device (see ¶ Col. 2, lines 48-53: “(8) …Based on the location, the middle tier populates an interface to request clarification from the user ad generate a revised query. The revised query is then transmitted to the LLM. Because the revised query corrects the ambiguity, the output of the LLM is more likely to be correct.”);
responsive to user input received at the client device indicative of an acceptance of the refined input prompt, generating responsive content to the NL based input based on second LLM output generated based on processing the refined input prompt using the LLM (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above, more specifically: “(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”); and
causing the responsive content to the NL based input to be rendered at the client device (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above, more specifically: “(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”).
However, Zhang et al. does not explicitly teach, but Hong et al. does teach:
wherein the NL based input lacks one or more details for completing an intended task (see ¶ [0070]: “As discussed, the prompt engineering model 260 is an artificial intelligence system designed to refine textual prompts to obtain detailed and tailored outputs from a generative AI model. The prompt engineering model 260 consists of several interconnected modules, each dedicated to refining the text input to align with the user's concept. The system begins with the detection module 500, which receives and channels the text prompt to specific processing modules. The visual concept detection module 502 identifies elements suitable for visual representation, while the specificity detection module 504 and ambiguity detection module 506 find and highlight elements that lack detail or are prone to multiple interpretations, respectively. The anomaly detection module 508 flags elements that deviate from expected patterns. The context-aware recommendation engine 510 then uses the outputs from these modules to suggest plausible keywords or phrases that refine the prompt. The Prompt UI module 512 and personalized recommendation engine 514 enhance the user experience by creating an intuitive interface and tailored suggestions. The prompt engineering model 260 also includes UI modules 516, 518, and 520 that assist users in clarifying ambiguous terms, rectifying anomalous entries, and enriching under-detailed concepts. Finally, the text prompt variation module 522 synthesizes the user's decisions to generate prompt variations closely aligned with the user's intended visual concept. These prompts can be used to generate highly accurate visual outputs or returned to the system for further refinement. By detecting and addressing ambiguities, anomalies, or areas lacking specificity within text prompts and providing personalized recommendations, this model ensures high-quality visual outputs that resonate with user intentions.”);
wherein the refined input prompt comprises a generative rewrite of the NL based input that includes one or more of the details that were lacking from the NL based input, and wherein the LLM is trained or fine-tuned based on training examples of NL based inputs and corresponding refined prompts (see ¶ [0070] citation as in limitation above, more specifically: “. The visual concept detection module 502 identifies elements suitable for visual representation, while the specificity detection module 504 and ambiguity detection module 506 find and highlight elements that lack detail or are prone to multiple interpretations, respectively. […] These prompts can be used to generate highly accurate visual outputs or returned to the system for further refinement. By detecting and addressing ambiguities, anomalies, or areas lacking specificity within text prompts and providing personalized recommendations, this model ensures high-quality visual outputs that resonate with user intentions.”);
Zhang et al. and Hong et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language query processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. to incorporate the teachings of Hong et al. of wherein the NL based input lacks one or more details for completing an intended task and wherein the refined input prompt comprises a generative rewrite of the NL based input that includes one or more of the details that were lacking from the NL based input, and wherein the LLM is trained or fine-tuned based on training examples of NL based inputs and corresponding refined prompts which provides the benefit of enhancing the user experience ([0070] of Hong et al.).
As to independent claim 19, Zhang et al. further teaches:
19. A system (see ¶ Col. 1, lines 59-64: “(5) In general, in one aspect, one or more embodiments relate to a system that includes at least one computer processor, a user interface comprising a query input widget to receive an original query, and a prompt manager executing on the at least one computer processor configured to perform operations…”) comprising:
one or more hardware processors (see ¶ Col. 1, lines 59-64: “(5) In general, in one aspect, one or more embodiments relate to a system that includes at least one computer processor,…”); and
memory storing instructions (see ¶ Col. 11, lines 4-16: “(52) Software instructions in the form of computer readable program code to perform embodiments may be stored, in whole or in part, temporarily or permanently, on a non-transitory computer readable medium such as a CD, DVD, storage device, a diskette, a tape, flash memory, physical memory, or any other computer readable storage medium. Specifically, the software instructions may correspond to computer readable program code that, when executed by a processor(s), is configured to perform one or more embodiments, which may include transmitting, receiving, presenting, and displaying data and messages described in the other figures of the disclosure.”) that, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations (see ¶ Col. 11, lines 4-16 citation as in limitation above: “…when executed by a processor(s), is configured to perform one or more embodiments…”) according to the method of claim 1 [taught by Zhang et al. in combination with Hong et al.].
As to independent claim 20, Zhang et al. further teaches:
20. A non-transitory computer-readable storage medium storing instructions (see ¶ Col. 11, lines 4-16: “(52) Software instructions in the form of computer readable program code to perform embodiments may be stored, in whole or in part, temporarily or permanently, on a non-transitory computer readable medium such as a CD, DVD, storage device, a diskette, a tape, flash memory, physical memory, or any other computer readable storage medium. Specifically, the software instructions may correspond to computer readable program code that, when executed by a processor(s), is configured to perform one or more embodiments, which may include transmitting, receiving, presenting, and displaying data and messages described in the other figures of the disclosure.”) that, when executed by one or more hardware processors, cause the one or more hardware processors to perform operations (see ¶ Col. 11, lines 4-16 citation as in limitation above.) according to the method of claim 1 [taught by Zhang et al. in combination with Hong et al.].
Regarding claim 2, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
2. The method of claim 1, wherein generating the refined input prompt is based on processing at least (i) the NL based input and (ii) one or more exemplary refined input prompts using the LLM (see Fig. 1 (118: original query, 116: ambiguity query (associated with the original query and input to LLM), 122: revised query (input to LLM), 106: LLM) and ¶ Col. 3, line 65 – Col. 4, line 6: “(17) Continuing with the prompt manager (104), the ambiguity detector (110) is configured to generate an ambiguity query (116) from another query (e.g., the original query (118), and perturbed versions of the original query). The ambiguity query (116) is a query to the LLM that encapsulates another query into a query asking if the other query is an ambiguous query. The ambiguity query (116) may encapsulate the original query (118) or encapsulate perturbed versions of the original query…”).
Regarding claim 5, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
5. The method of claim 1, wherein the context data comprises user data associated with a user of the client device (see Fig. 1 (118: original query, 116: ambiguity query (associated with the original query and input to LLM), 122: revised query (input to LLM), 106: LLM) and ¶ Col. 3, line 65 – Col. 4, line 6: “(17) Continuing with the prompt manager (104), the ambiguity detector (110) is configured to generate an ambiguity query (116) from another query (e.g., the original query (118), and perturbed versions of the original query). The ambiguity query (116) is a query to the LLM that encapsulates another query into a query asking if the other query is an ambiguous query. The ambiguity query (116) may encapsulate the original query (118) or encapsulate perturbed versions of the original query…”
and ¶ Col. 5, lines 48-51: “(26) In Block 203, an ambiguity query is generated from the original query. The ambiguity detector encapsulates the original query into an ambiguity query that is configured to concurrently train the LLM…”).
Regarding claim 7, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
7. The method of claim 1, further comprising:
selecting one or more terms of the refined input prompt to be user selectable terms (see ¶ Col. 4, line 47 – Col. 5, line 11: “(20) … For example, the clarification request may be a list of suggestions from which the user may accept one or add the user's own clarification. As another example, the clarification request may be a question presented to the user.”); and
causing an indication of the user selectable terms from among the refined input prompt to be rendered at the client device (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above.).
Regarding claim 8, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 7, above.
Zhang et al. further teaches:
8. The method of claim 7, wherein selecting the one or more terms of the refined input prompt to be user selectable terms is based on metadata included in the first LLM output (see (¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above and further ¶ Col. 8, lines 44-61: “(40) In Block 311, a clarification for the original query is generated based on the ambiguous location. In one of more embodiments, the prompt manager interacts with the user via the user interface to generate the clarification. Specifically, the prompt manager may send a clarification request to the user via the user interface that asks the user to clarify the term. The prompt manager may add some suggestions to clarify the term. For example, the suggestions may be the perturbations that resulted in the query being less ambiguous as compared to the original query. As another example, the suggestions may be the types of perturbations that resulted in the query being less ambiguous (e.g., using a synonym versus adding an adjective or clause). (41) In order to generate the clarification request, the prompt manager may populate a template with a term at the ambiguous location. The template may be of the form: “What do you mean by “<term at ambiguous location>”? or “Please clarify: “<term at ambiguous location>.””).
Regarding claim 9, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 7, above.
Zhang et al. further teaches:
9. The method of claim 7, further comprising:
responsive to detecting user interaction with a particular user selectable term of the one or more user selectable terms at the client device, causing one or more alternative terms corresponding to the particular user selectable term to be rendered at the client device (see Fig. 1 and see ¶ Col. 3, lines 38-44: “(15) A prompt manager (104) is interposed between the user interface (100) and the LLM (106). The prompt manager (104) is configured to detect an ambiguous query and suggest revisions for the ambiguous query. In one or more embodiments, the prompt manager includes an ambiguity example dataset (108), an ambiguity detector (110), a perturbed query creator (112), and a query coach (114).”,
¶ Col. 4, lines 19: “(18) The ambiguity detector (110) is further configured to receive a binary response (120) from the LLM (106) and detect, based at least in part on the binary response (120). The binary response (120) is the response that has one of two possible values. The first value indicates that the query encapsulated by the ambiguity query is detected as ambiguous by the LLM (106) while the second value indicates that the query encapsulated by the ambiguity query is detected as unambiguous by the LLM (106). The LLM (106) may further be configured to output a confidence value. The confidence value is detected probability that the binary value is correct. Specifically, the confidence value is the degree of confidence that the LLM is correct in predicting the query as ambiguous or unambiguous”
and ¶ Col. 4, line 47 – Col. 5, line 11: “(20) Continuing with FIG. 1, the prompt manager (104) further includes a query coach (114). The query coach (114) is configured to determine, using the perturbed query creator (112) and the ambiguity detector (110), the locations of the original query that are ambiguous and obtain clarification for the ambiguous query. Specifically, the query coach (114) is configured to obtain a clarification for the original query (118) to generate a revised query (122). The clarification may have more specificity in the original query (118) through adjectives, different word choices, or clauses. In one or more embodiments, the query coach (114) is configured to generate a clarification request for the ambiguous location in the original query. The clarification request is a request to the user to clarify a portion of the original query. For example, the clarification request may be a question or a suggestion to clarify a particular term in the original query that is presented in the user interface (100). For example, the clarification request may be a list of suggestions from which the user may accept one or add the user's own clarification. As another example, the clarification request may be a question presented to the user.
(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”); and
responsive to detecting user selection of a particular alternative term at the client device, replacing the particular user selectable term with the particular alternative term in the refined input prompt to generate an updated refined input prompt (see Fig. 1 and ¶ Col. 3, lines 38-44, ¶ Col. 4, lines 19 and ¶ Col. 4, line 47 – Col. 5, line 11, more specifically: “…For example, the clarification request may be a question or a suggestion to clarify a particular term in the original query that is presented in the user interface (100). For example, the clarification request may be a list of suggestions from which the user may accept one or add the user's own clarification. As another example, the clarification request may be a question presented to the user...”),
wherein the responsive content is generated based on processing the updated refined input prompt using the LLM in response to user input received at the client device indicative of an acceptance of the updated refined input prompt (see Fig. 1 and ¶ Col. 3, lines 38-44, ¶ Col. 4, lines 19 and ¶ Col. 4, line 47 – Col. 5, line 11, more specifically: “…the clarification request may be a list of suggestions from which the user may accept one or add the user's own clarification…”).
Regarding claim 11, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
11. The method of claim 1, further comprising:
causing a graphical user interface element to be rendered at the client device, wherein the refined input prompt is generated in response to user selection of the graphical user interface element at the client device (see (¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above and further ¶ Col. 8, lines 44-61: “(40) In Block 311, a clarification for the original query is generated based on the ambiguous location. In one of more embodiments, the prompt manager interacts with the user via the user interface to generate the clarification. Specifically, the prompt manager may send a clarification request to the user via the user interface that asks the user to clarify the term. The prompt manager may add some suggestions to clarify the term. For example, the suggestions may be the perturbations that resulted in the query being less ambiguous as compared to the original query. As another example, the suggestions may be the types of perturbations that resulted in the query being less ambiguous (e.g., using a synonym versus adding an adjective or clause). (41) In order to generate the clarification request, the prompt manager may populate a template with a term at the ambiguous location. The template may be of the form: “What do you mean by “<term at ambiguous location>”? or “Please clarify: “<term at ambiguous location>.””).
Regarding claim 12, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 11, above.
Zhang et al. further teaches:
12. The method of claim 11, further comprising:
determining whether to cause rendering of the graphical user interface element based on a quality metric of the NL based input and/or based on determining that the NL based input comprises harmful content (see ¶ Col. 6, lines 31-46: “(29) In Block 209, a determination is whether the prompt manager detects the original query as an ambiguous query using the binary response and confidence value. In some embodiments, the prompt manager uses the binary response directly to detect that the original query is an ambiguous query. Namely, if the binary response is that the original query is ambiguous, then the prompt manager detects the original query as ambiguous. In some embodiments, the prompt manager further uses the confidence value to determine if the original query is ambiguous. For example, if the binary value is that the original query is ambiguous and the confidence value is greater than a minimum threshold, then the prompt manager may detect that the original query is ambiguous. The minimum threshold is a different threshold than a threshold used by the LLM to determine whether the ambiguous query is ambiguous.”
and ¶ Col. 6, line 66 – Col. 7, line 5: “(33) Returning to Block 209, if the original query is detected as ambiguous, the flow proceeds to Block 211. In Block 211, a revised query is generated from the original query. Generating the revised query may be performed by outputting in the user interface, a recommendation to revise the original query and receiving a response with the revised query via the user interface…”).
Regarding claim 13, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 11, above.
Zhang et al. further teaches:
13. The method of claim 1, wherein the responsive content is first responsive content (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above, more specifically: “(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”), the method further comprising:
generating second responsive content to the NL based input based on third LLM output generated based on processing the NL based input using the LLM (see Fig. 1 (118: original query, 116: ambiguity query (associated with the original query and input to LLM), 122: revised query (input to LLM), 120: binary response, 124: query response, 106: LLM) and ¶ Col. 3, line 65 – Col. 4, line 32: “(17) Continuing with the prompt manager (104), the ambiguity detector (110) is configured to generate an ambiguity query (116) from another query (e.g., the original query (118), and perturbed versions of the original query). The ambiguity query (116) is a query to the LLM that encapsulates another query into a query asking if the other query is an ambiguous query. The ambiguity query (116) may encapsulate the original query (118) or encapsulate perturbed versions of the original query […] (18) The ambiguity detector (110) is further configured to receive a binary response (120) from the LLM (106) and detect, based at least in part on the binary response (120). The binary response (120) is the response that has one of two possible values…”);
causing the second responsive content to the NL based input to be rendered at the client device (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above, more specifically: “(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”);
detecting user input indicative of a selection of one of the first responsive content and the second responsive content at the client device (see ¶ Col. 4, line 47 – Col. 5, line 11: “(20) Continuing with FIG. 1, the prompt manager (104) further includes a query coach (114). The query coach (114) is configured to determine, using the perturbed query creator (112) and the ambiguity detector (110), the locations of the original query that are ambiguous and obtain clarification for the ambiguous query. Specifically, the query coach (114) is configured to obtain a clarification for the original query (118) to generate a revised query (122). The clarification may have more specificity in the original query (118) through adjectives, different word choices, or clauses. In one or more embodiments, the query coach (114) is configured to generate a clarification request for the ambiguous location in the original query. The clarification request is a request to the user to clarify a portion of the original query. For example, the clarification request may be a question or a suggestion to clarify a particular term in the original query that is presented in the user interface (100). For example, the clarification request may be a list of suggestions from which the user may accept one or add the user's own clarification. As another example, the clarification request may be a question presented to the user.); and
in response to the first responsive content being selected (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above.): based on the refined input prompt and the NL based input together as a training example, training or fine-tuning the LLM to generate, based on a given NL based input, a corresponding refined input prompt (see (¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above and further ¶ Col. 8, lines 44-61: “(40) In Block 311, a clarification for the original query is generated based on the ambiguous location. In one of more embodiments, the prompt manager interacts with the user via the user interface to generate the clarification. Specifically, the prompt manager may send a clarification request to the user via the user interface that asks the user to clarify the term. The prompt manager may add some suggestions to clarify the term. For example, the suggestions may be the perturbations that resulted in the query being less ambiguous as compared to the original query. As another example, the suggestions may be the types of perturbations that resulted in the query being less ambiguous (e.g., using a synonym versus adding an adjective or clause). (41) In order to generate the clarification request, the prompt manager may populate a template with a term at the ambiguous location. The template may be of the form: “What do you mean by “<term at ambiguous location>”? or “Please clarify: “<term at ambiguous location>.””).
Regarding claim 15, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
15. The method of claim 1, further comprising:
responsive to the user input received at the client device indicative of the acceptance of the refined input prompt (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above, more specifically: “(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”);
based on the refined input prompt and the NL based input together as a training example, training or fine-tuning the LLM to generate, based on a given NL based input, a corresponding refined input prompt (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above, more specifically: “(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”).
Regarding claim 16, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 15, above.
Zhang et al. further teaches:
16. The method of claim 15, further comprising:
responsive to a user input received at the client device indicative of a request to revert to the NL based input, bypassing the training or fine-tuning based on storing the refined input prompt and the NL based input as a training example (see Fig. 2 (209, 211-215, and 221: bypassing the generation/processing/storing of a revised query when no ambiguity based) and ¶ Col. 6, line 58 – Col. 7, line 27: “(32) If a determination is made that the original query is not ambiguous, then the original query is transmitted to the LLM, and the query response transmitted to the user interface in Block 221. Then, the flow may proceed to end. Transmitting the original query and transmitting the binary response may be via the prompt manager or directly from the user interface to the LLM. (33) Returning to Block 209, if the original query is detected as ambiguous, the flow proceeds to Block 211. In Block 211, a revised query is generated from the original query…”).
Claim 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 12038918 B1) and further in view of Hong et al. (US 20240354513 A1) and Fu et al. (US 20250190604 A1).
As to independent claim 18, Zhang et al. teaches:
18. A method implemented by one or more processors (see ¶ Col. 1, lines 30-67: “[…] (5) In general, in one aspect, one or more embodiments relate to a system that includes at least one computer processor, a user interface comprising a query input widget to receive an original query, and a prompt manager executing on the at least one computer processor configured to perform operations…”), the method comprising:
obtaining one or more training examples wherein each training example comprises a NL based input (see ¶ Col. 3, lines 45-64: “(16) The ambiguity example dataset (108) is a set of labeled training data to train the LLM as how to classify an ambiguity. Specifically, the ambiguity example dataset (108) includes labeled queries that include ambiguous and unambiguous queries. Namely, each query in the ambiguity example dataset (108) is related to a label defining whether the query is ambiguous or unambiguous. The same label may be associated with multiple queries in which case the position of the query in the ambiguity example dataset (108) may indicate whether the query is ambiguous or unambiguous. In another example, a separate label may be associated with each query. The ambiguity example dataset (108) may include hundreds or thousands of example ambiguous queries and unambiguous queries. In one or more embodiments, the ambiguity example dataset (108) may be user labeled or derived from user feedback. By way of an example of being human derived, when a user provides feedback that a response to a query did not answer the user's question, then the query that the user submits may be labeled as an ambiguous query.”);
fine-tuning, based on the one or more training examples, an LLM (see ¶ Col. 3, lines 45-64: “(16) The ambiguity example dataset (108) is a set of labeled training data to train the LLM as how to classify an ambiguity…”) to generate, based on a given NL based input associated with a client device, a corresponding refined input prompt usable (see ¶ Col. 3, lines 38-44: “(15) A prompt manager (104) is interposed between the user interface (100) and the LLM (106). The prompt manager (104) is configured to detect an ambiguous query and suggest revisions for the ambiguous query. In one or more embodiments, the prompt manager includes an ambiguity example dataset (108), an ambiguity detector (110), a perturbed query creator (112), and a query coach (114).”,
¶ Col. 4, lines 19: “(18) The ambiguity detector (110) is further configured to receive a binary response (120) from the LLM (106) and detect, based at least in part on the binary response (120). The binary response (120) is the response that has one of two possible values. The first value indicates that the query encapsulated by the ambiguity query is detected as ambiguous by the LLM (106) while the second value indicates that the query encapsulated by the ambiguity query is detected as unambiguous by the LLM (106). The LLM (106) may further be configured to output a confidence value. The confidence value is detected probability that the binary value is correct. Specifically, the confidence value is the degree of confidence that the LLM is correct in predicting the query as ambiguous or unambiguous”
and ¶ Col. 4, line 47 – Col. 5, line 11: “(20) Continuing with FIG. 1, the prompt manager (104) further includes a query coach (114). The query coach (114) is configured to determine, using the perturbed query creator (112) and the ambiguity detector (110), the locations of the original query that are ambiguous and obtain clarification for the ambiguous query. Specifically, the query coach (114) is configured to obtain a clarification for the original query (118) to generate a revised query (122). The clarification may have more specificity in the original query (118) through adjectives, different word choices, or clauses. In one or more embodiments, the query coach (114) is configured to generate a clarification request for the ambiguous location in the original query. The clarification request is a request to the user to clarify a portion of the original query. For example, the clarification request may be a question or a suggestion to clarify a particular term in the original query that is presented in the user interface (100). For example, the clarification request may be a list of suggestions from which the user may accept one or add the user's own clarification. As another example, the clarification request may be a question presented to the user.
(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”) to, in response to user input received at the client device indicative of an acceptance of the corresponding refined input prompt (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above, more specifically: “(20)… The clarification request is a request to the user to clarify a portion of the original query. For example, the clarification request may be a question or a suggestion to clarify a particular term in the original query that is presented in the user interface (100). For example, the clarification request may be a list of suggestions from which the user may accept one or add the user's own clarification. As another example, the clarification request may be a question presented to the user.”), generate responsive content to the given NL based input based on LLM output generated based on processing the corresponding refined input prompt using the LLM (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above, more specifically: “(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”), the responsive content to be caused to be rendered at the client device (see ¶ Col. 4, line 47 – Col. 5, line 11 citation as in limitation above, more specifically: “(21) The result of the clarification is a revised query (122) that is transmitted to the LLM (106). The LLM (106) may respond to the revised query (122) with a query response (124), which is presented in the user interface (100). The query response is the answer to the user's query. The query response may be a free-text response. Namely, whereas the ambiguity query is a binary response query in which the result is binary response, the original query and revised query are free-text response queries. The query response may be any text, image, or multimedia response that is the output of the LLM to the revised query.”).
However, Zhang et al. does not explicitly teach, but Hong et al. does teach:
wherein the refined input prompt comprises a generative rewrite of the NL based input that includes one or more of details lacking from the given NL based input (see ¶ [0070]: “As discussed, the prompt engineering model 260 is an artificial intelligence system designed to refine textual prompts to obtain detailed and tailored outputs from a generative AI model. The prompt engineering model 260 consists of several interconnected modules, each dedicated to refining the text input to align with the user's concept. The system begins with the detection module 500, which receives and channels the text prompt to specific processing modules. The visual concept detection module 502 identifies elements suitable for visual representation, while the specificity detection module 504 and ambiguity detection module 506 find and highlight elements that lack detail or are prone to multiple interpretations, respectively. The anomaly detection module 508 flags elements that deviate from expected patterns. The context-aware recommendation engine 510 then uses the outputs from these modules to suggest plausible keywords or phrases that refine the prompt. The Prompt UI module 512 and personalized recommendation engine 514 enhance the user experience by creating an intuitive interface and tailored suggestions. The prompt engineering model 260 also includes UI modules 516, 518, and 520 that assist users in clarifying ambiguous terms, rectifying anomalous entries, and enriching under-detailed concepts. Finally, the text prompt variation module 522 synthesizes the user's decisions to generate prompt variations closely aligned with the user's intended visual concept. These prompts can be used to generate highly accurate visual outputs or returned to the system for further refinement. By detecting and addressing ambiguities, anomalies, or areas lacking specificity within text prompts and providing personalized recommendations, this model ensures high-quality visual outputs that resonate with user intentions.”)
Zhang et al. and Hong et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language query processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. to incorporate the teachings of Hong et al. of wherein the refined input prompt comprises a generative rewrite of the NL based input that includes one or more of details lacking from the given NL based input which provides the benefit of enhancing the user experience ([0070] of Hong et al.).
However, Zhang et al. in combination with Hong et al. do not explicitly teach, but Fu et al. does teach:
obtaining one or more training examples wherein each training example comprises a NL based input and a corresponding refined input prompt (see ¶ [0019]: “The database query generator 105 interfaces with an LLM 104 (e.g., a via an API of the LLM 104) that has been adapted to generate database queries from user queries. In this example, the LLM 104 has been adapted to generate SQL queries from user queries. Similar to the intent classifier 103, the LLM 104 may have been a publicly available and/or pretrained LLM that was adapted to perform the task of generating database queries compatible with the database 113 from user queries. Adapting the LLM 104 for this task used a training dataset of example user queries and corresponding database queries, and performance of the LLM 104 once adapted was evaluated to ensure that the LLM 104 generates valid, executable database queries as is described below in reference to FIG. 2.”)
Zhang et al., Hong et al. and Fu et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language query processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. in combination with Hong et al. to incorporate the teachings of Fu et al. of obtaining one or more training examples wherein each training example comprises a NL based input and a corresponding refined input prompt which provides the benefit of improving LLM’s reliability in generating valid, executable database queries from user queries ([0031] of Fu et al.).
Claim 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 12038918 B1) and further in view of Hong et al. (US 20240354513 A1) as applied to claim 1 and further in view of Fu et al. (US 20250190604 A1).
Regarding claim 17, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
17. The method of claim 1, further comprising:
fine-tuning the LLM to generate, based on a given NL based input, a corresponding refined input prompt, based on one or more training examples (see ¶ Col. 3, lines 45-64: “(16) The ambiguity example dataset (108) is a set of labeled training data to train the LLM as how to classify an ambiguity…”),
However, Zhang et al. in combination with Hong et al. do not explicitly teach, but Fu et al. does teach:
wherein each training example comprises a NL based input and a corresponding refined input prompt (see ¶ [0019]: “The database query generator 105 interfaces with an LLM 104 (e.g., a via an API of the LLM 104) that has been adapted to generate database queries from user queries. In this example, the LLM 104 has been adapted to generate SQL queries from user queries. Similar to the intent classifier 103, the LLM 104 may have been a publicly available and/or pretrained LLM that was adapted to perform the task of generating database queries compatible with the database 113 from user queries. Adapting the LLM 104 for this task used a training dataset of example user queries and corresponding database queries, and performance of the LLM 104 once adapted was evaluated to ensure that the LLM 104 generates valid, executable database queries as is described below in reference to FIG. 2.”)
Zhang et al., Hong et al., and Fu et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language query processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. in combination with Hong et al. to incorporate the teachings of Fu et al. of wherein each training example comprises a NL based input and a corresponding refined input prompt which provides the benefit of improving LLM’s reliability in generating valid, executable database queries from user queries ([0031] of Fu et al.).
Claims 6, 10, and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 12038918 B1) and further in view of Hong et al. (US 20240354513 A1) as applied to claim 1 above and further in view of Shen et al. (Shen, Junxiao, et al. "Promptor: A conversational and autonomous prompt generation agent for intelligent text entry techniques." arXiv preprint arXiv:2310.08101 (2023). https://arxiv.org/pdf/2310.08101).
Regarding claim 6, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 1, above.
However, Zhang et al. in combination with Hong et al. do not explicitly teach, but Shen et al. does teach:
6. The method of claim 1, wherein the NL based input is received as part of a multi-turn dialog with a user of the client device, and wherein the context data comprises historical data associated with one or more of the previous turns of the multi-turn dialog (see Figure 3 (multi-turn), Table 3 (multi-turn), and ¶ 5.2 Prompting GPT-3.5: “Our goal is to prompt GPT-3.5 to equip it with the core feature of predicting multiple sentence candidates based on conversation history, persona, and a user’s set of keywords…”).
Zhang et al., Hong et al., and Shen et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language query processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. in combination with Hong et al. to incorporate the teachings of Shen et al. of wherein the NL based input is received as part of a multi-turn dialog with a user of the client device, and wherein the context data comprises historical data associated with one or more of the previous turns of the multi-turn dialog which provides the benefit of improving sentence prediction and user personalization ([abstract] of Shen et al.).
Regarding claim 10, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 9, above.
However, Zhang et al. in combination with Hong et al. do not explicitly teach, but Shen et al. does teach:
10. The method of claim 9, further comprising:
storing an indication of the selection of the alternative term for use in generating subsequent refined input prompts using the LLM (see Figure 3 (multi-turn), Table 3 (multi-turn), and ¶ 5.2 Prompting GPT-3.5: “Our goal is to prompt GPT-3.5 to equip it with the core feature of predicting multiple sentence candidates based on conversation history, persona, and a user’s set of keywords…” and last ¶ of 6.3.3 Promptor-designed prompts generate better predictions than self-designed prompts: “…Some users only used the user’s input and neglected or ignored the previous conversation history. In contrast, Promptor provides users with a comprehensive background of different perspectives of information to leverage. Notably, all users in Group B utilized both the conversational history and user’s input.”).
Zhang et al., Hong et al. and Shen et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language query processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. in combination with Hong et al. to incorporate the teachings of Shen et al. of storing an indication of the selection of the alternative term for use in generating subsequent refined input prompts using the LLM which provides the benefit of improving sentence prediction and user personalization ([abstract] of Shen et al.).
Regarding claim 14, Zhang et al. in combination with Hong et al. teaches the limitations as in claim 9, above.
However, Zhang et al. in combination with Hong et al. do not explicitly teach, but Shen et al. does teach:
14. The method of claim 1, further comprising:
modifying the refined input prompt based on user input received at the client device to generate an updated refined input prompt (see ¶ 5.2 Prompting GPT-3.5: “Our goal is to prompt GPT-3.5 to equip it with the core feature of predicting multiple sentence candidates based on conversation history, persona, and a user’s set of keywords. Additionally, it should have the advanced capability of incorporating punctuation for users. In devising the optimal approach, we consulted with Promptor and finalized a specific prompt. We utilized this prompt to configure GPT-3.5 as KWickChat 2. Remarkably, this entire procedure was completed in under 20 minutes, showcasing its efficiency, especially when compared to the time-intensive process of fine-tuning a KWickChat model for system integration. Implementing GPT-3.5 is streamlined, necessitating only API calls.” and ¶ 5.3 Fine-tuning GPT-3.5: “OpenAI has recently introduced the fine-tuning functionality for GPT-3.5. We employed the OpenAI fine-tuning APIs to fine-tune GPT-3.5, utilizing the same prompt as we did for the Prompted GPT-3.5 version. The fine-tuning process follows the same training and testing split from Shen et al. [42].”);
responsive to user input received at the client device indicative of an acceptance of the updated refined input prompt, generating responsive content to the NL based input based on processing the updated refined input prompt using the LLM (see ¶ 5.2 Prompting GPT-3.5 and ¶ 5.3 Fine-tuning GPT-3.5 citations as in limitation above and further Fig. 3 and Table 3);
causing the responsive content to the NL based input to be rendered at the client device (see ¶ 5.2 Prompting GPT-3.5 and ¶ 5.3 Fine-tuning GPT-3.5 citations as in limitation above and further Fig. 3 and Table 3.); and
based on the updated refined input prompt and the NL based input together as a training example, training or fine-tuning the LLM to generate, based on a given NL based input, a corresponding refined input prompt (see ¶ 5.2 Prompting GPT-3.5 and ¶ 5.3 Fine-tuning GPT-3.5 citations as in limitation above and further Fig. 3 and Table 3.).
Zhang et al., Hong et al. and Shen et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in natural language query processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. in combination with Hong et al. to incorporate the teachings of Shen et al. of modifying the refined input prompt based on user input received at the client device to generate an updated refined input prompt; responsive to user input received at the client device indicative of an acceptance of the updated refined input prompt, generating responsive content to the NL based input based on processing the updated refined input prompt using the LLM; causing the responsive content to the NL based input to be rendered at the client device; and storing the updated refined input prompt and the NL based input together as a training example for use in fine-tuning the LLM to generate, based on a given NL based input, a corresponding refined input prompt which provides the benefit of improving sentence prediction and user personalization ([abstract] of Shen et al.).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Keisha Y Castillo-Torres whose telephone number is (571)272-3975. The examiner can normally be reached Monday - Friday, 9:00 am - 4:00 pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre-Louis Desir can be reached at (571)272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Keisha Y. Castillo-Torres
Examiner
Art Unit 2659
/Keisha Y. Castillo-Torres/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
Prosecution Insights