DETAILED ACTION
This office action is in response to Applicant’s Request for Continued Examination (RCE), received on 03/23/2026. Claims 1, 9-11, 14-15, 17-18, 20 have been amended. Claims 1-20 are pending and have been considered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/23/2026 has been entered.
Response to Arguments
Applicant’s arguments, see pgs. 9-10, filed 03/23/2026, with respect to the rejection(s) of claim(s) 1, 14, and 17 under 35 U.S.C. 103 (Zhu in view of Jensen) have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Long et al. (US-20200104427-A1), hereinafter Long. Long discloses “Techniques for providing a personalized neural query auto-completion pipeline are disclosed herein. In some embodiments, a computer system, in response to detecting user-entered text that has been entered by a user in a search field of a search engine, generates auto-completion candidates based on the user-entered text and a corresponding frequency level for each one of the auto-completion candidates, ranks the auto-completion candidates based on profile data of the user using a neural network model, and causes at least a portion of the plurality of auto-completion candidates to be displayed in an auto-complete user interface element of the search field within the user interface of the computing device of the user based on the ranking prior to the user-entered text being submitted by the user as part of a search query” (abstract). See updated rejections below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-6, 8, 10-11, 13-15, 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (US-20190324780-A1), hereinafter Zhu, in view of Jensen et al. (US-20250156451-A1), hereinafter Jensen, further in view of Long et al. (US-20200104427-A1), hereinafter Long.
Regarding claim 1, Zhu discloses: a method comprising:
obtaining, from a first model ([Fig. 4, Dialog Engine 235]), a typeahead suggestion of a list comprising a plurality of typeahead suggestions responsive to a partial search query ([Fig. 4, Candidate Hypotheses 415], [0064] The user input may comprise a partial request. In particular embodiments, the assistant system 140 may analyze, based on a personalized language model, the user input to generate one or more candidate hypotheses corresponding to the partial request [Generation of hypotheses indicates a required obtaining of the hypotheses, i.e. suggestions, wherein a plurality of hypotheses indicates a list of suggestions]).
Zhu does not disclose:
creating a prompt comprising a plurality of portions, wherein a first portion of the plurality of the portions comprises an instruction to rank the typeahead suggestion against the list of the plurality of typeahead suggestions and wherein a second portion of the plurality of portions comprises the typeahead suggestion.
Jensen discloses:
creating a prompt comprising a plurality of portions ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Defining a plurality of inputs to be used for generating a prompt indicates each distinct input to be a portion of the overall prompt, i.e. search query and/or user characteristics among other possible sections]), wherein a first portion of the plurality of the portions comprises an instruction to rank the typeahead suggestion against the list of the plurality of typeahead suggestions ([0067] After generating the autosuggest query candidates 415, one or more of them may be sent to the user device. To further refine which query candidates are sent to the user device, the query candidates may be scored and ranked before sending 420. In some embodiments, the autosuggest query candidates 415 that were generated by the language model 410 are evaluated along with a set of query candidates generated by a “traditional” autosuggest model that is trained directly on user search queries. The query candidates may also be evaluated with respect to a search space relevant to the search request. This may determine, for example, whether the query candidates correspond to potential results in the search space. For the online concierge system, for example, the search space may correspond to items available at one or more locations relevant to the user's search. In the example of FIG. 4, the query candidate “Application” may have no relevant result in the search space and may be removed accordingly from the set of query suggestions sent to the user. Thus, while the output token predictions (e.g., the token logits) may be used to initially select the query candidates 415, the scoring and ranking 420 may be used to further confirm that the output tokens, based on a more general language model, are relevant to the particular search space, [Performing a ranking operation to refine results sent to a user using a language model 410 indicates the relevant, selected candidates to have a higher ranking as compared to the other candidates in the generated tokens and/or candidates generated by the other “traditional” model which forms a list as seen in Fig. 4. The language model would necessarily have to be prompted to perform this task]) and wherein a second portion of the plurality of portions comprises the typeahead suggestion ([Fig. 4, Language Model 410 receiving Prompt and Initial Partial Query 400 as input], [Having a language model which receives a prompt for generating autosuggestions based on input partial queries indicates that the prompt must necessarily include the partial query to know what to generate autosuggestions off of]).
Zhu and Jensen are considered analogous art within user query completion. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu to incorporate the teachings of Jensen, because of the novel way to re-use previous candidates relevant to revised partial search queries to generate additional tokens that expand upon previous candidates, reducing required runtime latency for generating autosuggestions (Jensen, [0003]).
Zhu in view of Jensen does not disclose:
providing the prompt as an input to a large language model (LLM), wherein the LLM is different from the first model; and
causing the LLM to rank the typeahead suggestion against the list of the plurality of typeahead suggestions based on the prompt.
Long discloses:
providing the prompt as an input to a large language model (LLM), wherein the LLM is different from the first model ([0053] The candidate generation module 310 then generates auto-completion candidates 515, such as in the example embodiments of the candidate generation module 310 discussed in the present disclosure. The generated auto-completion candidates 515 are fed into the neural network module 320, which generates a ranking of the auto-completion candidates 525, such as in the example embodiments of the neural network module 320, [Ranking the candidates with a model different from that which generated the candidates indicates the neural network module to be different from the candidate generation module, wherein [0058] describes a language-model based evaluator indicating the neural network model to be a large language model in view of the large language model of Jensen. The examiner asserts that operations performed by a language model are necessarily performed through prompting said model in view of the previously disclosed prompt of Jensen]); and
causing the LLM to rank the typeahead suggestion against the list of the plurality of typeahead suggestions based on the prompt ([As previously cited, ranking auto-completions candidates against each other indicates comparing a typeahead suggestion against the list of the plurality of suggestions, wherein a language model is necessarily operated via prompt in view of the prompt input of Jensen]).
Zhu, Jensen, and Long are considered analogous art within query auto-completion suggestions. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen to incorporate the teachings of Long, because of the novel way to use a heuristic method to generate an initial set of auto-completion candidates based on a history of previously-submitted search queries to be ranked using a neural network, reducing the computational expense associated with employing a complex neural network to score auto-completion candidates (Long, [0017]).
Zhu further discloses:
providing, to a computing device ([0026] a client system 130 may be an electronic device), an evaluation output by the LLM in response to the prompt ([0073] dialog engine 235 may further use the personalized language model 410 to generate a ranked list of candidate slots, [0076] The assistant system 140 may present the suggested auto-completions 510 [Presenting top-ranked suggestions indicates the ranks/selected suggestions are representative of evaluation output]).
Regarding claim 2, Zhu in view of Jensen, further in view of Long discloses: the method of claim 1.
Zhu further discloses:
wherein the typeahead suggestion comprises an autocompleted search suggestion ([Fig. 5A, 510]) or at least one of an entity suggested search result ([Fig. 5B, 510, “Roger”]), a product suggested search result, a job entity suggested search result ([The examiner would like to note that due to the disjunctive nature of the claim, not all elements require a mapping]), or a knowledge suggested search result ([0074] The assistant system 140 may then generate a candidate hypothesis 415 as “send a message to [SL:people(name)] at [SL:time(time)].” If the user selects this candidate hypothesis 415, the assistant system 140 may further generate a list of highly relevant friends/family members for the slot of [SL:people(name)] and some possible time for the slot of [SL:time(time)], [Awareness of specific people and/or time slots indicates a knowledge suggested search]).
Regarding claim 3, Zhu in view of Jensen, further in view of Long discloses: the method of claim 2.
Jensen further discloses:
wherein the instructions to evaluate the typeahead suggestion in the first portion of the plurality of portions of the prompt further comprises at least one of an evaluation instruction for the autocompleted search suggestion ([0065] The language model 410 may assign probabilities to possible output tokens representing the likelihood of the output token given the input sequence and specified prior output sequence. The output tokens may output as a probability of particular tokens, or the probability of a token relative to other tokens (e.g., as a softmax). To determine tokens for the autosuggest query candidates 415, the tokens may be selected based on the output token probabilities), an evaluation instruction for the entity suggested search result, an evaluation instruction for the product suggested search result, an evaluation instruction for the job entity suggested search result, or an evaluation instruction for the knowledge suggested search result ([The examiner would like to note that due to the disjunctive nature of the claims, these elements do not require a mapping]).
Regarding claim 4, Zhu in view of Jensen, further in view of Long discloses: the method of claim 1.
Jensen further discloses:
wherein the plurality of portions of the prompt further comprises a third portion comprising user profile information associated with a user profile and the partial search query ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Context describing the user indicates a portion of the prompt dedicated to user profile information as compared to portions dedicated to the search space, etc.]).
Regarding claim 5, Zhu in view of Jensen, further in view of Long discloses: the method of claim 4.
Zhu further discloses:
wherein the evaluation output comprises an indication of the typeahead suggestion being a low-quality typeahead suggestion ([0071] the assistant system 140 may determine if at least one confidence score of the one or more confidence scores associated with the one or more candidate hypotheses 415 is smaller than a threshold score [Determining a confidence score to be smaller than a threshold confidence, in the context of predicting typeahead words, indicates a low confidence score is a low-quality typeahead suggestion]), further comprising:
obtaining a second typeahead suggestion responsive to a second partial search query, wherein the second typeahead suggestion is the same as the typeahead suggestion ([0064] The user input may comprise a partial request. In particular embodiments, the assistant system 140 may analyze, based on a personalized language model, the user input to generate one or more candidate hypotheses corresponding to the partial request [Receiving a second typeahead suggestion the same as the first indicates one typeahead suggestion]);
creating a second prompt based on the second typeahead suggestion and a second user profile information ([0039] the assistant system 140 may analyze the user input using natural-language understanding. The analysis may be based on the user profile for more personalized and context-aware understanding, [0050] the proactive agent 285 may also rank the generated candidate entities based on the user profile and the content associated with the candidate entities. The ranking may be based on the similarities between a user's interests and the candidate entities, [0065] generation of the one or more candidate hypotheses 415 may be further based on one or more context-specific language models 420) [Ranking candidate entities/hypotheses, i.e. typeahead suggestions, wherein the model is user-dependent (see assistant system (containing proactive agent 285) leveraging a “personalized user model”, ([0063])), indicates a required user-specific prompt (representing user interests) for the model to match similarities be able to rank the suggestions based on a specific user; wherein generating and then ranking hypotheses, i.e. user-specific ranking, indicates that the ranking operation is required to be dependent upon the user and typeahead suggestion, i.e. hypotheses, in the form of a ranking prompt. Extending this to a second prompt/typeahead/profile does not change the functionality of Zhu]), wherein the second user profile information is different from the user profile information ([0099] one or more second users of an online social network, [Defining second users of an online social network, indicating a required user profile for access to the social network, indicates this user/profile could be used to generate a second personalized model without a change in functionality to Zhu]); and,
causing the LLM to evaluate the second typeahead suggestion based on the second prompt to obtain a second evaluation output ([0050] The assistant system 140 may then calculate similarity scores (e.g., based on cosine similarity) between the feature vector representing the user's interest and the feature vectors representing the candidate entities, [Determining a similarity score, i.e. evaluation, between entities, i.e. typeahead suggestions, and user information, i.e. reasonably understood to represent a prompt, indicates an evaluation of the typeahead suggestion based on the prompt. Extending this to a second prompt/typeahead suggestion does not change the functionality of Zhu]), wherein the second evaluation output comprises an indication of the second typeahead suggestion being a high-quality typeahead suggestion ([Fig. 5A, 510], [0073] generate a ranked list of candidate slots, [Generation of a ranked list of typeahead suggestions indicates at least one high-quality typeahead to be presented to the user]).
Regarding claim 6, Zhu in view of Jensen, further in view of Long discloses: the method of claim 4.
Zhu further discloses:
wherein the partial search query and the user profile information is an input pair of a set of input pairs ([0050] the proactive agent 285 may generate candidate entities associated with the proactive task based on a user profile, [Generating candidate entities, i.e. suggestions, based on a user profile indicates the partial search query for which candidates are generated and the user profile are an input pair. Further, in view of the plurality of generated ranked entities/suggestions, see [Fig. 5A-B], indicating a set of input pairs, i.e. the user paired with each candidate entity/suggestion]).
Regarding claim 8, Zhu in view of Jensen, further in view of Long discloses: the method of claim 6.
Zhu further discloses:
wherein the set of input pairs comprises a distribution of pairs of partial search queries and user profile information associated with autocompleted search suggestions ([Fig. 5A, 510, “send a message to…”], [An autocompleted suggestion with the input “s”]), pairs of partial search queries and user profile information associated with entity suggested search results ([Fig. 5B, 510, “Send a message to Raymond”], [A contact is an entity]), pairs of partial search queries and user profile information associated with product suggested search results ([Fig. 5A, 510, “set a timer”], [A timer is reasonably understood to be a product]), pairs of partial search queries and user profile information associated with job entity suggested search results ([Fig. 5A, 510, “share”], [Sharing information tracks to a job, i.e. transmitting, sending, receiving, etc. and “share” can be representative of a label for a job entity tag]), and pairs of partial search queries and user profile information associated with knowledge suggested search results ([Fig. 5B, 510, “send a message to Roger”], [Knowing contacts to send messages to indicates associated knowledge with the partial query, wherein the contacts are dependent upon the user as previously disclosed], [Further, the examiner would like to note that, as Zhu discloses a plurality of generated results, even more of which are generated before presenting the ranked list to the user, Zhu clearly discloses a distribution of pairs of the above categories]).
Regarding claim 10, Zhu in view of Jensen, further in view of Long discloses: the method of claim 1.
Jensen further discloses:
creating the prompt comprising the plurality of portions ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Defining a plurality of inputs to be used for generating a prompt indicates each distinct input to be a portion of the overall prompt, i.e. search query and/or user characteristics among other possible sections]), wherein the prompt comprises a third portion comprising at least one of user activity or profile information associated with a user profile ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Context describing the user indicates a portion of the prompt dedicated to user profile information as compared to portions dedicated to the search space, etc.]), and wherein the second portion of the plurality of portions comprises the plurality of typeahead suggestions ([Fig. 4, Generating Token(s) 435 based on Maintained Autosuggest Query Candidates 430 as input to Language Model 410], [Passing autosuggested query candidates as input to the language model to generate additional suggestions indicates a required prompting of the language model to receive the autosuggested candidate(s) (plurality emphasized), i.e. typeahead suggestions, to know what to build off of]);
causing the LLM to evaluate the plurality of typeahead suggestions using the prompt ([Considering the previously cited probability determination of Jensen for evaluation, indicating that this element has already inherently been covered by Jensen as required by the prompt of Jensen]); and,
providing, to the computing device ([In view of the previously disclosed computing device of Zhu]), a plurality of evaluation outputs corresponding to the plurality of typeahead suggestions evaluated using the prompt ([Fig. 3, Display Query Candidates 320 to User Device 300 generated based on Search System 305, i.e. Fig. 4, consisting of the language model receiving a prompt]).
Regarding claim 11, Zhu in view of Jensen, further in view of Long discloses: the method of claim 1.
Zhu further discloses:
generating a second typeahead suggestion determined using the LLM ([0072] The assistant system 140 may further generate the one or more candidate hypotheses 415 based on the personalized language model 410), wherein the second typeahead suggestion is responsive to the partial search query ([Fig. 5A, 515], [Wherein the suggestions in 515 are clearly all for the same partial search query indicating at least a second typehead suggestion]);
creating a second prompt based on the second typeahead suggestion ([0039] the assistant system 140 may analyze the user input using natural-language understanding. The analysis may be based on the user profile for more personalized and context-aware understanding, [0050] the proactive agent 285 may also rank the generated candidate entities based on the user profile and the content associated with the candidate entities. The ranking may be based on the similarities between a user's interests and the candidate entities, [0065] generation of the one or more candidate hypotheses 415 may be further based on one or more context-specific language models 420 [Ranking candidate entities/hypotheses, i.e. typeahead suggestions, wherein the model is user-dependent (see assistant system (containing proactive agent 285) leveraging a “personalized user model”, ([0063])), indicates a required user-specific prompt (representing user interests) for the model to match similarities be able to rank the suggestions based on a specific user; wherein generating and then ranking hypotheses, i.e. user-specific ranking, indicates that the ranking operation is required to be dependent upon the user and typeahead suggestion, i.e. hypotheses, in the form of a ranking prompt. Extending this operation to a second prompt/suggestion does not change the functionality of prompting of Zhu as Zhu discloses generating multiple hypothesis using the same models]);
causing the LLM to evaluate the second typeahead suggestion based on the second prompt to obtain a second evaluation output ([0050] The assistant system 140 may then calculate similarity scores (e.g., based on cosine similarity) between the feature vector representing the user's interest and the feature vectors representing the candidate entities, [Determining a similarity score, i.e. evaluation, between entities, i.e. typeahead suggestions, and user information, i.e. reasonably understood to represent a prompt, indicates an evaluation of the typeahead suggestion based on the prompt. Extending this operation to a second suggestion does not change the functionality of Zhu, i.e. either suggestions generated through the personalized model or context-specific model could be used as candidate entities for evaluation]).
Regarding claim 13, Zhu in view of Jensen, further in view of Long discloses: the method of claim 1.
Zhu further discloses:
wherein causing the LLM to evaluate the typeahead suggestion based on the prompt further comprises:
receiving, by the LLM, an Application Program Interface (API) call comprising the prompt ([0034] allow users to interact with these entities through an application programming interfaces (API) or other communication channels, [Wherein one of the interacting entities is the assistant system, see Fig. 1]), wherein the prompt includes a plurality of typeahead suggestions ([0077] the assistant system 140 may analyze, based on a personalized language model 410, the user input 405 to generate one or more candidate hypotheses 415 corresponding to the partial request, [Generating candidate hypotheses based on the personalized language model, i.e. LLM, which is trained using the dialog engine, a part of the assistant system, see Fig. 2, indicating a required prompt for training the model to be personalized. Further, this information for personalization could be sent through API as disclosed in [0034] and contains a plurality of typeahead suggestions dependent upon the user. The generated prompt for personalization could also have typehead suggestions added in after the original prompt call without departing from the current construction of the claim]);
providing, to the computing device ([Fig. 4, auto-complete suggestion 460 sent to client system 130], [Wherein a client system can be reasonably understood to represent a computing device, see Figs. 5A/B]), an evaluation output for each of the plurality of typeahead suggestions in response to the API call ([Fig. 5A, 515], [Providing a n-best ranked list of typeahead suggestions indicates the suggestions to be an evaluation output]).
Regarding claim 14, Zhu discloses:
a system comprising:
at least one processor ([0110] a processor 1002); and,
at least one memory device coupled to the at least one processor ([Fig. 10, Memory 1004], [Wherein the memory and processor are coupled through bus 1012]), wherein the memory device comprises instructions that, when executed by the at least one processor ([0111] data in memory 1004 or storage 1006 for instructions executing at processor 1002), cause the at least one processor to perform at least one operation comprising:
obtaining, from a first model ([Fig. 4, Dialog Engine 235]), a typeahead suggestion of a list comprising a plurality of typeahead suggestions responsive to a partial search query ([Fig. 4, Candidate Hypotheses 415], [0064] The user input may comprise a partial request. In particular embodiments, the assistant system 140 may analyze, based on a personalized language model, the user input to generate one or more candidate hypotheses corresponding to the partial request [Generation of hypotheses indicates a required obtaining of the hypotheses, i.e. suggestions, wherein a plurality of hypotheses indicates a list of suggestions]).
Zhu does not disclose:
creating a prompt comprising a plurality of portions, wherein a first portion of the plurality of the portions comprises an instruction to rank the typeahead suggestion against the list of the plurality of typeahead suggestions and wherein a second portion of the plurality of portions comprises the typeahead suggestion.
Jensen discloses:
creating a prompt comprising a plurality of portions ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Defining a plurality of inputs to be used for generating a prompt indicates each distinct input to be a portion of the overall prompt, i.e. search query and/or user characteristics among other possible sections]), wherein a first portion of the plurality of the portions comprises an instruction to rank the typeahead suggestion against the list of the plurality of typeahead suggestions ([0067] After generating the autosuggest query candidates 415, one or more of them may be sent to the user device. To further refine which query candidates are sent to the user device, the query candidates may be scored and ranked before sending 420. In some embodiments, the autosuggest query candidates 415 that were generated by the language model 410 are evaluated along with a set of query candidates generated by a “traditional” autosuggest model that is trained directly on user search queries. The query candidates may also be evaluated with respect to a search space relevant to the search request. This may determine, for example, whether the query candidates correspond to potential results in the search space. For the online concierge system, for example, the search space may correspond to items available at one or more locations relevant to the user's search. In the example of FIG. 4, the query candidate “Application” may have no relevant result in the search space and may be removed accordingly from the set of query suggestions sent to the user. Thus, while the output token predictions (e.g., the token logits) may be used to initially select the query candidates 415, the scoring and ranking 420 may be used to further confirm that the output tokens, based on a more general language model, are relevant to the particular search space, [Performing a ranking operation to refine results sent to a user using a language model 410 indicates the relevant, selected candidates to have a higher ranking as compared to the other candidates in the generated tokens and/or candidates generated by the other “traditional” model which forms a list as seen in Fig. 4. The language model would necessarily have to be prompted to perform this task]) and wherein a second portion of the plurality of portions comprises the typeahead suggestion ([Fig. 4, Language Model 410 receiving Prompt and Initial Partial Query 400 as input], [Having a language model which receives a prompt for generating autosuggestions based on input partial queries indicates that the prompt must necessarily include the partial query to know what to generate autosuggestions off of]).
Zhu and Jensen are considered analogous art within user query completion. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu to incorporate the teachings of Jensen, because of the novel way to re-use previous candidates relevant to revised partial search queries to generate additional tokens that expand upon previous candidates, reducing required runtime latency for generating autosuggestions (Jensen, [0003]).
Zhu in view of Jensen does not disclose:
providing the prompt as an input to a large language model (LLM), wherein the LLM is different from the first model; and
causing the LLM to rank the typeahead suggestion against the list of the plurality of typeahead suggestions based on the prompt.
Long discloses:
providing the prompt as an input to a large language model (LLM), wherein the LLM is different from the first model ([0053] The candidate generation module 310 then generates auto-completion candidates 515, such as in the example embodiments of the candidate generation module 310 discussed in the present disclosure. The generated auto-completion candidates 515 are fed into the neural network module 320, which generates a ranking of the auto-completion candidates 525, such as in the example embodiments of the neural network module 320, [Ranking the candidates with a model different from that which generated the candidates indicates the neural network module to be different from the candidate generation module, wherein [0058] describes a language-model based evaluator indicating the neural network model to be a large language model in view of the large language model of Jensen. The examiner asserts that operations performed by a language model are necessarily performed through prompting said model in view of the previously disclosed prompt of Jensen]); and
causing the LLM to rank the typeahead suggestion against the list of the plurality of typeahead suggestions based on the prompt ([As previously cited, ranking auto-completions candidates against each other indicates comparing a typeahead suggestion against the list of the plurality of suggestions, wherein a language model is necessarily operated via prompt in view of the prompt input of Jensen]).
Zhu, Jensen, and Long are considered analogous art within query auto-completion suggestions. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen to incorporate the teachings of Long, because of the novel way to use a heuristic method to generate an initial set of auto-completion candidates based on a history of previously-submitted search queries to be ranked using a neural network, reducing the computational expense associated with employing a complex neural network to score auto-completion candidates (Long, [0017]).
Zhu further discloses:
providing, to a computing device ([0026] a client system 130 may be an electronic device), an evaluation output by the LLM in response to the prompt ([0073] dialog engine 235 may further use the personalized language model 410 to generate a ranked list of candidate slots, [0076] The assistant system 140 may present the suggested auto-completions 510 [Presenting top-ranked suggestions indicates the ranks/selected suggestions are representative of evaluation output]).
Regarding claim 15, Zhu in view of Jensen, further in view of Long discloses: the system of claim 14.
Jensen further discloses:
creating the prompt comprising the plurality of portions ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Defining a plurality of inputs to be used for generating a prompt indicates each distinct input to be a portion of the overall prompt, i.e. search query and/or user characteristics among other possible sections]), wherein the prompt comprises a third portion comprising at least one of user activity or profile information associated with a user profile ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Context describing the user indicates a portion of the prompt dedicated to user profile information as compared to portions dedicated to the search space, etc.]), and wherein the second portion of the plurality of portions comprises the plurality of typeahead suggestions ([Fig. 4, Generating Token(s) 435 based on Maintained Autosuggest Query Candidates 430 as input to Language Model 410], [Passing autosuggested query candidates as input to the language model to generate additional suggestions indicates a required prompting of the language model to receive the autosuggested candidate(s) (plurality emphasized), i.e. typeahead suggestions, to know what to build off of]);
causing the LLM to evaluate the plurality of typeahead suggestions using the prompt ([Considering the previously cited probability determination of Jensen for evaluation, indicating that this element has already inherently been covered by Jensen as required by the prompt of Jensen]); and,
providing, to the computing device ([In view of the previously disclosed computing device of Zhu]), a plurality of evaluation outputs corresponding to the plurality of typeahead suggestions evaluated using the prompt ([Fig. 3, Display Query Candidates 320 to User Device 300 generated based on Search System 305, i.e. Fig. 4, consisting of the language model receiving a prompt]).
Regarding claim 17, Zhu discloses: a non-transitory machine-readable storage medium comprising instructions that ([0117] a computer-readable non-transitory storage medium), when executed by at least one processor ([Fig. 10, Processor 1002]), cause the at least one processor to perform at least one operation comprising:
obtaining, from a first model ([Fig. 4, Dialog Engine 235]), a typeahead suggestion of a list comprising a plurality of typeahead suggestions responsive to a partial search query ([Fig. 4, Candidate Hypotheses 415], [0064] The user input may comprise a partial request. In particular embodiments, the assistant system 140 may analyze, based on a personalized language model, the user input to generate one or more candidate hypotheses corresponding to the partial request [Generation of hypotheses indicates a required obtaining of the hypotheses, i.e. suggestions, wherein a plurality of hypotheses indicates a list of suggestions]).
Zhu does not disclose:
creating a prompt comprising a plurality of portions, wherein a first portion of the plurality of the portions comprises an instruction to rank the typeahead suggestion against the list of the plurality of typeahead suggestions and wherein a second portion of the plurality of portions comprises the typeahead suggestion.
Jensen discloses:
creating a prompt comprising a plurality of portions ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Defining a plurality of inputs to be used for generating a prompt indicates each distinct input to be a portion of the overall prompt, i.e. search query and/or user characteristics among other possible sections]), wherein a first portion of the plurality of the portions comprises an instruction to rank the typeahead suggestion against the list of the plurality of typeahead suggestions ([0067] After generating the autosuggest query candidates 415, one or more of them may be sent to the user device. To further refine which query candidates are sent to the user device, the query candidates may be scored and ranked before sending 420. In some embodiments, the autosuggest query candidates 415 that were generated by the language model 410 are evaluated along with a set of query candidates generated by a “traditional” autosuggest model that is trained directly on user search queries. The query candidates may also be evaluated with respect to a search space relevant to the search request. This may determine, for example, whether the query candidates correspond to potential results in the search space. For the online concierge system, for example, the search space may correspond to items available at one or more locations relevant to the user's search. In the example of FIG. 4, the query candidate “Application” may have no relevant result in the search space and may be removed accordingly from the set of query suggestions sent to the user. Thus, while the output token predictions (e.g., the token logits) may be used to initially select the query candidates 415, the scoring and ranking 420 may be used to further confirm that the output tokens, based on a more general language model, are relevant to the particular search space, [Performing a ranking operation to refine results sent to a user using a language model 410 indicates the relevant, selected candidates to have a higher ranking as compared to the other candidates in the generated tokens and/or candidates generated by the other “traditional” model which forms a list as seen in Fig. 4. The language model would necessarily have to be prompted to perform this task]) and wherein a second portion of the plurality of portions comprises the typeahead suggestion ([Fig. 4, Language Model 410 receiving Prompt and Initial Partial Query 400 as input], [Having a language model which receives a prompt for generating autosuggestions based on input partial queries indicates that the prompt must necessarily include the partial query to know what to generate autosuggestions off of]).
Zhu and Jensen are considered analogous art within user query completion. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu to incorporate the teachings of Jensen, because of the novel way to re-use previous candidates relevant to revised partial search queries to generate additional tokens that expand upon previous candidates, reducing required runtime latency for generating autosuggestions (Jensen, [0003]).
Zhu in view of Jensen does not disclose:
providing the prompt as an input to a large language model (LLM), wherein the LLM is different from the first model; and
causing the LLM to rank the typeahead suggestion against the list of the plurality of typeahead suggestions based on the prompt.
Long discloses:
providing the prompt as an input to a large language model (LLM), wherein the LLM is different from the first model ([0053] The candidate generation module 310 then generates auto-completion candidates 515, such as in the example embodiments of the candidate generation module 310 discussed in the present disclosure. The generated auto-completion candidates 515 are fed into the neural network module 320, which generates a ranking of the auto-completion candidates 525, such as in the example embodiments of the neural network module 320, [Ranking the candidates with a model different from that which generated the candidates indicates the neural network module to be different from the candidate generation module, wherein [0058] describes a language-model based evaluator indicating the neural network model to be a large language model in view of the large language model of Jensen. The examiner asserts that operations performed by a language model are necessarily performed through prompting said model in view of the previously disclosed prompt of Jensen]); and
causing the LLM to rank the typeahead suggestion against the list of the plurality of typeahead suggestions based on the prompt ([As previously cited, ranking auto-completions candidates against each other indicates comparing a typeahead suggestion against the list of the plurality of suggestions, wherein a language model is necessarily operated via prompt in view of the prompt input of Jensen]).
Zhu, Jensen, and Long are considered analogous art within query auto-completion suggestions. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen to incorporate the teachings of Long, because of the novel way to use a heuristic method to generate an initial set of auto-completion candidates based on a history of previously-submitted search queries to be ranked using a neural network, reducing the computational expense associated with employing a complex neural network to score auto-completion candidates (Long, [0017]).
Zhu further discloses:
providing, to a computing device ([0026] a client system 130 may be an electronic device), an evaluation output by the LLM in response to the prompt ([0073] dialog engine 235 may further use the personalized language model 410 to generate a ranked list of candidate slots, [0076] The assistant system 140 may present the suggested auto-completions 510 [Presenting top-ranked suggestions indicates the ranks/selected suggestions are representative of evaluation output]).
Regarding claim 18, Zhu in view of Jensen, further in view of Long discloses: the non-transitory machine-readable storage medium of claim 17.
Jensen further discloses:
creating the prompt comprising the plurality of portions ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Defining a plurality of inputs to be used for generating a prompt indicates each distinct input to be a portion of the overall prompt, i.e. search query and/or user characteristics among other possible sections]), wherein the prompt comprises a third portion comprising at least one of user activity or profile information associated with a user profile ([0062] the prompt for the language model 410 is generated based on a particular context describing characteristics of the search query, such as the user, the search space, and other contextual data. For example, for an online concierge system, the context may also describe items currently in the user's order (e.g., in the user's shopping cart), the user's dietary preferences and conditions, and so forth, [Context describing the user indicates a portion of the prompt dedicated to user profile information as compared to portions dedicated to the search space, etc.]), and wherein the second portion of the plurality of portions comprises the plurality of typeahead suggestions ([Fig. 4, Generating Token(s) 435 based on Maintained Autosuggest Query Candidates 430 as input to Language Model 410], [Passing autosuggested query candidates as input to the language model to generate additional suggestions indicates a required prompting of the language model to receive the autosuggested candidate(s) (plurality emphasized), i.e. typeahead suggestions, to know what to build off of]);
causing the LLM to evaluate the plurality of typeahead suggestions using the prompt ([Considering the previously cited probability determination of Jensen for evaluation, indicating that this element has already inherently been covered by Jensen as required by the prompt of Jensen]); and,
providing, to the computing device ([In view of the previously disclosed computing device of Zhu]), a plurality of evaluation outputs corresponding to the plurality of typeahead suggestions evaluated using the prompt ([Fig. 3, Display Query Candidates 320 to User Device 300 generated based on Search System 305, i.e. Fig. 4, consisting of the language model receiving a prompt]).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu in view of Jensen, further in view of Long, further in view of Botros et al. (US-20220188361-A1), hereinafter Botros, further in view of Podgorny et al. (US-20180108093-A1), hereinafter Podgorny.
Regarding claim 7, Zhu in view of Jensen, further in view of Long discloses: the method of claim 6.
Zhu further discloses:
wherein the set of input pairs comprises a distribution of pairs of partial search queries and user profile information associated with successful search sessions ([0050] the proactive agent 285 may generate candidate entities associated with the proactive task based on a user profile, [0076] The assistant system 140 may accordingly suggest auto-completions 510 including “send a message to Raymond” and “send a message to Roger”. The assistant system 140 may present the two suggested auto-completions 510 in the drop-down menu 515, in which the user can select one of them. For example, the user may select “send a message to Roger” , [Generating candidate entities, i.e. suggestions, based on a user profile indicates the partial search query for which candidates are generated and the user profile are an input pair. Further, in view of the user selecting one of options, indicating it to be a successful search.]).
Zhu in view of Jensen, further in view of Long does not disclose:
wherein the set of input pairs comprises a distribution of pairs of partial search queries and user profile information associated with abandoned search sessions and pairs of partial search queries and user profile information associated with bypassed search sessions.
Botros discloses:
wherein the set of input pairs comprises a distribution of pairs of partial search queries and user profile information associated with bypassed search sessions ([0037] The assist system 140 may use natural-language understanding to analyze the user request based on user's profile and other relevant information , [0108] the user may accept or reject a suggestion via a voice dictation. As yet another example and not by way of limitation, the user may tap once the smart glasses to accept a suggestion but twice to reject the suggestion, [Rejecting a suggestion is indicative that the suggestion was bypassed; wherein the suggestion is provided based on a request tied to a user profile, indicating a pairing of these two pieces of data]).
Zhu, Jensen, Long, and Botros are considered analogous art within auto-completing incomplete received input. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen, further in view of Long to incorporate the teachings of Botros, because of the novel way to consider dialog state, context information, location information, and multimodal signals to generate a personalized model for applying auto-completion of received data, improving human-machine interactions in queries (Botros, [0009]).
Zhu in view of Jensen, further in view of Long, further in view of Botros does not disclose:
wherein the set of input pairs comprises a distribution of pairs of partial search queries and user profile information associated with abandoned search sessions .
Podgorny discloses:
wherein the set of input pairs comprises a distribution of pairs of partial search queries and user profile information associated with abandoned search sessions ([0024] The type ahead suggestions help users formulate their questions, help users enter a search query faster, personalizes the search query by basing the type ahead suggestions on the users' profile data and/or tax data, and decreases the likelihood that a user will abandon the tax return preparation system out of frustration from not being able to articulate the user's question, [Decreasing a likelihood of abandonment, but not complete prevention, indicates a situation in which suggestions are still abandoned; wherein the suggestions are based on the user profile, indicating a pairing of these two pieces of information]).
Zhu, Jensen, Long, Botros, and Podgorny are considered analogous art within input auto-completion. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen, further in view of Long, further in view of Botros to incorporate the teachings of Podgorny, because of the novel way to provide domain-specific and dynamic type ahead suggestions for search query terms with a customer self-service system, improving the user’s experience in completing tax return preparation (Podgorny, [0015]).
Claim(s) 9, 12, 16, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhu in view of Jensen, further in view of Long, further in view of Das et al. (US-11017764-B1), hereinafter Das.
Regarding claim 9, Zhu in view of Jensen, further in view of Long discloses: the method of claim 1.
Zhu further discloses:
generating a second typeahead suggestion using a second model ([0072] The assistant system 140 may further generate the one or more candidate hypotheses 415 based on the personalized language model 410 and the selected context-specific language model 420, [Fig. 4, 410, 420, 415], [Generating candidate hypotheses based on input from two different models indicates either can represent a second model. Also, consider the global models 425, 430 to be secondary models used for generating typeahead suggestions]), wherein the second typeahead suggestion is responsive to the partial search query ([Fig. 4, 460], [Using input from two models to generate auto-complete suggestions indicates either model can be representative of a second model (the context-specific model is secondary as compared to the personalized language model previously mapped to the first model)]);
creating a second prompt based on the second typeahead suggestion ([0039] the assistant system 140 may analyze the user input using natural-language understanding. The analysis may be based on the user profile for more personalized and context-aware understanding, [0050] the proactive agent 285 may also rank the generated candidate entities based on the user profile and the content associated with the candidate entities. The ranking may be based on the similarities between a user's interests and the candidate entities, [0065] generation of the one or more candidate hypotheses 415 may be further based on one or more context-specific language models 420 [Ranking candidate entities/hypotheses, i.e. typeahead suggestions, wherein the model is user-dependent (see assistant system (containing proactive agent 285) leveraging a “personalized user model”, ([0063])), indicates a required user-specific prompt (representing user interests) for the model to match similarities be able to rank the suggestions based on a specific user; wherein generating and then ranking hypotheses, i.e. user-specific ranking, indicates that the ranking operation is required to be dependent upon the user and typeahead suggestion, i.e. hypotheses, in the form of a ranking prompt. Extending this operation to a second prompt/suggestion does not change the functionality of prompting of Zhu as Zhu discloses generating hypotheses using the context-specific, i.e. second, model]);
causing the LLM to evaluate the second typeahead suggestion based on the second prompt to obtain a second evaluation output ([0050] The assistant system 140 may then calculate similarity scores (e.g., based on cosine similarity) between the feature vector representing the user's interest and the feature vectors representing the candidate entities, [Determining a similarity score, i.e. evaluation, between entities, i.e. typeahead suggestions, and user information, i.e. reasonably understood to represent a prompt, indicates an evaluation of the typeahead suggestion based on the prompt. Extending this operation to a second suggestion does not change the functionality of Zhu, i.e. either suggestions generated through the personalized model or context-specific model could be used as candidate entities for evaluation]); and,
comparing the second evaluation output with the evaluation output ([0070] generate a list of ranked candidate hypotheses 415 may be effective solutions for addressing the technical challenge of presenting the most relevant candidate hypotheses 415 to a user which more correctly reflect a user's intention in a current dialog session, [Generating a ranked list, wherein it is previously disclosed that there are two models which could generate hypotheses to be ranked, indicating the ranking list is a comparison of evaluation outputs between first and second suggestions]).
Zhu in view of Jensen, further in view of Long does not disclose:
flagging the first model or the second model based on the comparison of the second evaluation output with the evaluation output.
Das discloses:
flagging the first model or the second model based on the comparison of the second evaluation output with the evaluation output ([Col. 109, Lines 25-45] In various embodiments, the NL application 3340 scores the proposed complete third NL request and the predictions 3540(3) based on the request prediction model 3414(3) and the data dependency model 3416(3) (or the most recent version of the sequence prediction model), and compares the scores… selects the best-scoring prediction 3540(3) as the complete third NL request 3415(3), [Selecting the best scoring prediction based on individual scores from two models indicates that the lower scoring model suggestion of the two is “flagged”, i.e. model output determined to not be selected. The two models of Das could be used as the two models defined in Zhu without a change in functionality to Zhu as a request prediction tracks to a typeahead and data dependency, which can be replaced with sequence prediction, also tracks to typeahead functionality]).
Zhu are considered analogous art within input auto-completion suggestions. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen, further in view of Long to incorporate the teachings of Das, because of the novel way to improve the function of natural language application implementations by generating models based on historical request data, historical intent data, and dependency data, reducing the amount of time/effort associated with processing natural language requests (Das, [Col. 119, Lines 5-25]).
Regarding claim 12, Zhu in view of Jensen, further in view of Long discloses: the method of claim 1.
Zhu further discloses:
iteratively training the LLM using a training evaluation output comprising an evaluation score and a reason for the evaluation score ([0050] The ranking may be alternatively based on a ranking model that is trained based on user feedback data, [User feedback based reasonably tracks to an evaluation score with an associated reason, i.e. the user will understand why an output is not appropriate and adjust the model through feedback]), and a training output comprising the training evaluation output ([Fig. 4, 460 “auto-complete suggestion”], [0065] the dialog engine 235 may train the personalized language model 410 based on training data 435 accessed from the user context engine 225, [0069] this disclosure contemplates training any suitable language model based on any suitable training data in any suitable manner [Training the personalized language model, wherein the personalized language model is depended upon for output, see connection between personalized language model 410 and dialog engine 235 resulting in auto-completed suggestion 460 output to client device (see Fig. 5A/B), indicates that the training of the language model based on the dialog engine also includes a required training output step]).
Zhu in view of Jensen, further in view of Long does not disclose:
iteratively training the LLM using a training prompt comprising a partial search query training input, training user profile information associated with a training user profile, and a training typeahead suggestion.
Das discloses:
iteratively training the LLM using a training prompt comprising a partial search query training input, training user profile information associated with a training user profile, and a training typeahead suggestion ([Col. 105, Lines 59-61] The ML algorithm 3510 trains the request prediction model 3414 based on information included in the knowledge database 3370, [Col. 97, Lines 25-40] the knowledge database 3370 includes, without limitation, a search model 3470, a knowledge graph 3482, any number of user profiles 3484, an interaction history database 3486, an intent database 3488, any number of DSL templates 3498, a disambiguation model 3491, an interaction model 3492, a follow-up model 3493, an expansion model 3495, a presentation model 3494, a request completion model 3412, a request prediction model 3414, and a data dependency model 3416, [Col. 96, Lines 22-48] the knowledge database 3370 indicates that John has repeatedly followed-up the request “daily sales last week” with the request “daily sales last week in London,” then a next request prediction ML model (not shown in FIG. 33) may automatically recognize the partial request “in London” from John, following a request “daily sales last week” from John [Training a request prediction model, reasonably understood to generate typeahead suggestions, therefore an LLM (consider the “large” amounts of data defined in Das, [Col. 6, Lines 20-25], [Col. 45, Lines 60-65]); wherein the training data consists of partial search queries, i.e. “daily sales last week”, training user profile, i.e. “John”, training typeahead suggestion, i.e. “in London”]).
Zhu are considered analogous art within input auto-completion suggestions. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen, further in view of Long to incorporate the teachings of Das, because of the novel way to improve the function of natural language application implementations by generating models based on historical request data, historical intent data, and dependency data, reducing the amount of time/effort associated with processing natural language requests (Das, [Col. 119, Lines 5-25]).
Regarding claim 16, Zhu in view of Jensen, further in view of Long discloses: the system of claim 14.
Zhu further discloses:
wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform at least one operation further comprising:
iteratively training the LLM using a training evaluation output comprising an evaluation score and a reason for the evaluation score ([0050] The ranking may be alternatively based on a ranking model that is trained based on user feedback data, [User feedback based reasonably tracks to an evaluation score with an associated reason, i.e. the user will understand why an output is not appropriate and adjust the model through feedback]), and a training output comprising the training evaluation output ([Fig. 4, 460 “auto-complete suggestion”], [0065] the dialog engine 235 may train the personalized language model 410 based on training data 435 accessed from the user context engine 225, [0069] this disclosure contemplates training any suitable language model based on any suitable training data in any suitable manner [Training the personalized language model, wherein the personalized language model is depended upon for output, see connection between personalized language model 410 and dialog engine 235 resulting in auto-completed suggestion 460 output to client device (see Fig. 5A/B), indicates that the training of the language model based on the dialog engine also includes a required training output step]).
Zhu in view of Jensen, further in view of Long does not disclose:
wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform at least one operation further comprising:
iteratively training the LLM using a training prompt comprising a partial search query training input, training user profile information associated with a training user profile, and a training typeahead suggestion.
Das discloses:
wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform at least one operation further comprising:
iteratively training the LLM using a training prompt comprising a partial search query training input, training user profile information associated with a training user profile, and a training typeahead suggestion ([Col. 105, Lines 59-61] The ML algorithm 3510 trains the request prediction model 3414 based on information included in the knowledge database 3370, [Col. 97, Lines 25-40] the knowledge database 3370 includes, without limitation, a search model 3470, a knowledge graph 3482, any number of user profiles 3484, an interaction history database 3486, an intent database 3488, any number of DSL templates 3498, a disambiguation model 3491, an interaction model 3492, a follow-up model 3493, an expansion model 3495, a presentation model 3494, a request completion model 3412, a request prediction model 3414, and a data dependency model 3416, [Col. 96, Lines 22-48] the knowledge database 3370 indicates that John has repeatedly followed-up the request “daily sales last week” with the request “daily sales last week in London,” then a next request prediction ML model (not shown in FIG. 33) may automatically recognize the partial request “in London” from John, following a request “daily sales last week” from John [Training a request prediction model, reasonably understood to generate typeahead suggestions, therefore an LLM (consider the “large” amounts of data defined in Das, [Col. 6, Lines 20-25], [Col. 45, Lines 60-65]); wherein the training data consists of partial search queries, i.e. “daily sales last week”, training user profile, i.e. “John”, training typeahead suggestion, i.e. “in London”]).
Zhu are considered analogous art within input auto-completion suggestions. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen, further in view of Long to incorporate the teachings of Das, because of the novel way to improve the function of natural language application implementations by generating models based on historical request data, historical intent data, and dependency data, reducing the amount of time/effort associated with processing natural language requests (Das, [Col. 119, Lines 5-25]).
Regarding claim 19, Zhu in view of Jensen, further in view of Long discloses: the non-transitory machine-readable storage medium of claim 17.
Zhu further discloses:
wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform at least one operation further comprising:
iteratively training the LLM using a training evaluation output comprising an evaluation score and a reason for the evaluation score ([0050] The ranking may be alternatively based on a ranking model that is trained based on user feedback data, [User feedback based reasonably tracks to an evaluation score with an associated reason, i.e. the user will understand why an output is not appropriate and adjust the model through feedback]), and a training output comprising the training evaluation output ([Fig. 4, 460 “auto-complete suggestion”], [0065] the dialog engine 235 may train the personalized language model 410 based on training data 435 accessed from the user context engine 225, [0069] this disclosure contemplates training any suitable language model based on any suitable training data in any suitable manner [Training the personalized language model, wherein the personalized language model is depended upon for output, see connection between personalized language model 410 and dialog engine 235 resulting in auto-completed suggestion 460 output to client device (see Fig. 5A/B), indicates that the training of the language model based on the dialog engine also includes a required training output step]).
Zhu in view of Jensen, further in view of Long does not disclose:
wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform at least one operation further comprising:
iteratively training the LLM using a training prompt comprising a partial search query training input, training user profile information associated with a training user profile, and a training typeahead suggestion.
Das discloses:
wherein the instructions, when executed by the at least one processor, cause the at least one processor to perform at least one operation further comprising:
iteratively training the LLM using a training prompt comprising a partial search query training input, training user profile information associated with a training user profile, and a training typeahead suggestion ([Col. 105, Lines 59-61] The ML algorithm 3510 trains the request prediction model 3414 based on information included in the knowledge database 3370, [Col. 97, Lines 25-40] the knowledge database 3370 includes, without limitation, a search model 3470, a knowledge graph 3482, any number of user profiles 3484, an interaction history database 3486, an intent database 3488, any number of DSL templates 3498, a disambiguation model 3491, an interaction model 3492, a follow-up model 3493, an expansion model 3495, a presentation model 3494, a request completion model 3412, a request prediction model 3414, and a data dependency model 3416, [Col. 96, Lines 22-48] the knowledge database 3370 indicates that John has repeatedly followed-up the request “daily sales last week” with the request “daily sales last week in London,” then a next request prediction ML model (not shown in FIG. 33) may automatically recognize the partial request “in London” from John, following a request “daily sales last week” from John [Training a request prediction model, reasonably understood to generate typeahead suggestions, therefore an LLM (consider the “large” amounts of data defined in Das, [Col. 6, Lines 20-25], [Col. 45, Lines 60-65]); wherein the training data consists of partial search queries, i.e. “daily sales last week”, training user profile, i.e. “John”, training typeahead suggestion, i.e. “in London”]).
Zhu are considered analogous art within input auto-completion suggestions. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen, further in view of Long to incorporate the teachings of Das, because of the novel way to improve the function of natural language application implementations by generating models based on historical request data, historical intent data, and dependency data, reducing the amount of time/effort associated with processing natural language requests (Das, [Col. 119, Lines 5-25]).
Regarding claim 20, Zhu in view of Jensen, further in view of Long discloses: the non-transitory machine-readable storage medium of claim 17.
Zhu further discloses:
generating a second typeahead suggestion using a second model ([0072] The assistant system 140 may further generate the one or more candidate hypotheses 415 based on the personalized language model 410 and the selected context-specific language model 420, [Fig. 4, 410, 420, 415], [Generating candidate hypotheses based on input from two different models indicates either can represent a second model. Also, consider the global models 425, 430 to be secondary models used for generating typeahead suggestions]), wherein the second typeahead suggestion is responsive to the partial search query ([Fig. 4, 460], [Using input from two models to generate auto-complete suggestions indicates either model can be representative of a second model (the context-specific model is secondary as compared to the personalized language model previously mapped to the first model)]);
creating a second prompt based on the second typeahead suggestion ([0039] the assistant system 140 may analyze the user input using natural-language understanding. The analysis may be based on the user profile for more personalized and context-aware understanding, [0050] the proactive agent 285 may also rank the generated candidate entities based on the user profile and the content associated with the candidate entities. The ranking may be based on the similarities between a user's interests and the candidate entities, [0065] generation of the one or more candidate hypotheses 415 may be further based on one or more context-specific language models 420 [Ranking candidate entities/hypotheses, i.e. typeahead suggestions, wherein the model is user-dependent (see assistant system (containing proactive agent 285) leveraging a “personalized user model”, ([0063])), indicates a required user-specific prompt (representing user interests) for the model to match similarities be able to rank the suggestions based on a specific user; wherein generating and then ranking hypotheses, i.e. user-specific ranking, indicates that the ranking operation is required to be dependent upon the user and typeahead suggestion, i.e. hypotheses, in the form of a ranking prompt. Extending this operation to a second prompt/suggestion does not change the functionality of prompting of Zhu as Zhu discloses generating hypotheses using the context-specific, i.e. second, model]);
causing the LLM to evaluate the second typeahead suggestion based on the second prompt to obtain a second evaluation output ([0050] The assistant system 140 may then calculate similarity scores (e.g., based on cosine similarity) between the feature vector representing the user's interest and the feature vectors representing the candidate entities, [Determining a similarity score, i.e. evaluation, between entities, i.e. typeahead suggestions, and user information, i.e. reasonably understood to represent a prompt, indicates an evaluation of the typeahead suggestion based on the prompt. Extending this operation to a second suggestion does not change the functionality of Zhu, i.e. either suggestions generated through the personalized model or context-specific model could be used as candidate entities for evaluation]); and,
comparing the second evaluation output with the evaluation output ([0070] generate a list of ranked candidate hypotheses 415 may be effective solutions for addressing the technical challenge of presenting the most relevant candidate hypotheses 415 to a user which more correctly reflect a user's intention in a current dialog session, [Generating a ranked list, wherein it is previously disclosed that there are two models which could generate hypotheses to be ranked, indicating the ranking list is a comparison of evaluation outputs between first and second suggestions]).
Zhu in view of Jensen, further in view of Long does not disclose:
flagging the first model or the second model based on the comparison of the second evaluation output with the evaluation output.
Das discloses:
flagging the first model or the second model based on the comparison of the second evaluation output with the evaluation output ([Col. 109, Lines 25-45] In various embodiments, the NL application 3340 scores the proposed complete third NL request and the predictions 3540(3) based on the request prediction model 3414(3) and the data dependency model 3416(3) (or the most recent version of the sequence prediction model), and compares the scores… selects the best-scoring prediction 3540(3) as the complete third NL request 3415(3), [Selecting the best scoring prediction based on individual scores from two models indicates that the lower scoring model suggestion of the two is “flagged”, i.e. model output determined to not be selected. The two models of Das could be used as the two models defined in Zhu without a change in functionality to Zhu as a request prediction tracks to a typeahead and data dependency, which can be replaced with sequence prediction, also tracks to typeahead functionality]).
Zhu are considered analogous art within input auto-completion suggestions. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Zhu in view of Jensen, further in view of Long to incorporate the teachings of Das, because of the novel way to improve the function of natural language application implementations by generating models based on historical request data, historical intent data, and dependency data, reducing the amount of time/effort associated with processing natural language requests (Das, [Col. 119, Lines 5-25]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Anushiravani et al. (US-20250110997-A1) discloses “Various embodiments of the present disclosure provide model-based domain-aware autocomplete techniques for generating autocomplete suggestions in a complex search domain. Example embodiments are configured to generate, using a domain-aware autocomplete model, a label for an autocomplete suggestion based on a set of keywords within an autocomplete suggestion training dataset associated with a target domain source. Example embodiments are also configured to generate, using a weak-labeling model, an updated label for the autocomplete suggestion by decorrelating the set of keywords from the label. Example embodiments are also configured to generate, using a sentence classification model, a category for the autocomplete suggestion based on the updated label. Example embodiments are also configured to, using the domain-aware autocomplete model, generate a suggestion-category pair (SCP) based on the autocomplete suggestion and the category for the autocomplete suggestion. Example embodiments are also configured for initiating performance of a search query resolution based on the SCP.” (abstract). See entire document.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to THEODORE JOHN WITHEY whose telephone number is (703)756-1754. The examiner can normally be reached Monday - Friday, 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Flanders can be reached at (571) 272-7516. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/THEODORE WITHEY/Examiner, Art Unit 2655
/ANDREW C FLANDERS/Supervisory Patent Examiner, Art Unit 2655
Prosecution Insights