DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 04/30/2026 have been fully considered but they are not persuasive.
Regarding applicant’s remarks directed to the rejection of claims under 35 USC § 103,
Alleged No teaching of generate the first output that includes no corresponding token for calling any functional component different from the deep learning model
In Remarks p. 11-12, Applicant contends:
“The cited references do not disclose or suggest the recited automatic determining
step performed as part of obtaining the first output”
Applicant argues that the modules of Shuster each “has its own input construction rule, its own control tokens, and a separate model call. By contrast, amended claim I requires that the determination be performed "by the deep learning model" as part of "obtaining the first output," that is, in the same prediction by the deep learning model that produces the first output.”
Further, Applicant argues that “Second, the binary decision of each Shuster module is whether to invoke a particular tool (e.g., Internet search or long-term memory access), and not whether to call any external functional component versus to reply directly. As Shuster, Section 3 .1.1, states, "[i]f neither search nor long-term memory access is required, an entity is extracted from the history instead, and that is appended to the context (prefixed with control tokens)," after which the Generate dialogue response module is invoked.”
The relevant claim limitations appear to be “in response to that it is determined that a reply to the initial input is to be made without an external functional component being called, generate the first output that includes no corresponding token for calling any functional component different from the deep learning model, and using the first output as the reply to the initial input” in amended claim 1.
As Applicant notes, Shuster discloses “"[i]f neither search nor long-term memory access is required [in response to that it is determined that a reply to the initial input is to be made without an external functional component being called; ie neither external functional components should be called, thus Examiner notes that the control tokens indicating accessing the respective components (do search/access memory) are not included; See table 1], an entity is extracted from the history instead, and that is appended to the context (prefixed with control tokens)," after which the Generate dialogue response module is invoked [generate the first output ie generated dialogue response that includes no corresponding token for calling any functional component different from the deep learning model, and using the first output as the reply to the initial input].; Examiner further provides cropped figure 2 wherein figure 2 shows generating the dialogue response without calling any functional components”
PNG
media_image1.png
641
1058
media_image1.png
Greyscale
Further, Examiner respectfully points to Table 1 wherein Examiner notes different tokens are outputted for the different decisions:
PNG
media_image2.png
55
1041
media_image2.png
Greyscale
PNG
media_image3.png
52
1038
media_image3.png
Greyscale
)
After careful consideration, the arguments are not considered unpersuasive as Shuster augments the model of Shazeer with the modules that provides the decision on whether to call the external functional component tools or not to improve interactions in a more responsible and useful manner. Shuster further considers the modules as part of the model (Shuster, Section 3, “At its core, BlenderBot 3 (BB3) is a transformer model (Vaswani et al., 2017) which produces dialogue responses using a series of modules, each of which is a sequence-to-sequence task.”) Examiner emphasizes the distinction between the modules and external functional components in the annotated figure 2 above and provides cropped table 2 to provide the distinct tokens of not accessing the external functional components (ie do not search/do not access memory).
Alleged No teaching of the recited external memory bank, historical reply storage, and similarity-based retrieval
In Remarks p. 12-14, Applicant contends:
“Shazeer does not disclose any external memory bank that stores the user's prior
dialogue with the deep learning model, nor does Shazeer disclose returning a historical reply item generated by the deep learning model based on a similarity comparison with the input data.
Shuster does not cure this deficiency. The long-term memory store of Shuster, Section
3.1, holds persona facts about the user (for example, "I am a hair stylist," as illustrated in Shuster FIG. 1 ), which are static attributes and not historical replies generated by the deep learning model. Shuster, Section 3 .2.2, selects, from a set of persona lines, the line with "highest word overlap with the next utterance" as a target during construction of the training data; this is a target-selection step in training-data construction, and not a runtime, similarity-based retrieval of a historical reply from the user's stored dialogue with the deep learning model.
Harijan also fails to cure the deficiency. The conversation data structure of Harijan,
shown in FIG. 3B and described at paragraphs [0043]-[0044], records messages exchanged between two human users (for example, "User" and "John W."). The Office Action takes the position that "John W. can be substituted by the ML Language model of Shazeer." The Applicant respectfully submits that Harijan itself does not describe John W. as a deep learning model, and that the proposed substitution finds support, on this record, only in the present application.”
The relevant claim limitations appear to be “wherein the first functional component is an external memory bank, and the external memory bank stores a first data group set related to the user, and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item, wherein the first intermediate inquiry is based on the input data, and wherein the first intermediate result is a historical reply item corresponding to a historical input data item that is in the first data group set and whose similarity with the input data is higher than a first threshold.” in amended claim 1.
As noted in the previous Office Action, Shazeer in view of Shuster and Harijan teaches (emphasis added):
Shazeer teaches wherein the first functional component is an external memory bank,
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database;”)
However, Shazeer does not explicitly teach and the external memory bank stores a first data group set related to the user, and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item
Shuster teaches and the external memory bank stores a first data group set related to the user,
(Shuster, Section 3.1, “Access long-term memory Given the full input context, and a store of (text-based) memories [the external memory bank stores a first data group set related to the user (see Figure 1 Memory Store Person 1’s Persona: I am a hair stylist and Person 2’s Persona: I love dogs wherein Examiner notes that the Memory Store is comprised of the user’s (Person 1) ‘memory’ and the system’s (Person 2) ‘memory’)], output a memory from the memory store [wherein the first functional component is an external memory bank], referred to as a recalled memory. Note: if the memory store is too large to fit in the context, we adopt some simple strategies. For the 3B parameter model, we use the Fusion-in-Decoder method (Izacard and Grave, 2021). For the OPT-based models for simplicity of implementation, we sample the memories to fit in the 2048 token context. We keep those with overlapping keywords to prior turns.”)
However, Shuster does not explicitly teach and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item [generated by the deep learning model for the historical input data item].
Harijan teaches wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item [generated by the deep learning model for the historical input data item]
(Harijan, “[0043] An embodiment that may be used to track conversations and store conversation data incorporates a conversation data structure and message data structure as depicted in FIG. 3 [wherein each data group in the first data group set ie a conversation data structure comprises at least a historical input data item ie user input (ex Fig. 3B 362) and a historical reply item ie another user’s input (ex Fig. 3B 364 wherein John W. can be substituted by the DL model of Shuster ie John is the chatbot)]. A messaging service or messaging application may generate a conversation data structure to capture message data and time data within a conversation. There are many data structures suitable for tracking and storing conversations such as an array, matrix, stack, linked list, tree, queue or string. Hierarchical data structures such as a tree may be beneficial for organization of priorities and/or topics and keywords. Data structures may be stored in several ways, including in one or more databases. Data structures can be stored both locally on the device and remotely in a network. A messaging service application or system may utilize a context engine, stored and executed by one or more of the processors and memory depicted in FIGS. 7 and 8, to store and track conversation data in the effort to prevent sending a message to an unintended recipient.
[0044] One embodiment may include a context engine storing a conversation in a conversation data structure such as conversation data structure 310 depicted in FIG. 3A. Conversation data structure 310 includes fields such as date/time field 312, message field 314, from field 316, keywords field 318, and topic field 320. Conversation data structure 310 includes one or more incoming and/or outgoing messages or entries such as entry 322, entry 324, entry 326, and entry 328. Each entry, e.g., entry 322, may include data in each of the respective data fields such as date/time field 312, message field 314, from field 316, keywords field 318, and topic field 320. To generate the data structure such as conversation data structure 310, the context engine would parse a sent or received message for data for each field. The messaging service or application would analyze each message in a conversation and record the date and time of the message in the date/time field 312, record message data to the message field 314, and record sender data to the from field 316. Sender data may be a name, telephone number, username, email address or other unique identifier.”)
PNG
media_image4.png
375
602
media_image4.png
Greyscale
Shuster teaches and wherein the first intermediate result is a historical reply item corresponding to a historical input data item that is in the first data group set and whose similarity with the input data is higher than a first threshold.
(Shuster, Section 3.2.2, “Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context, which is calculated as the one with the highest word overlap with the next utterance [whose similarity with the input data is higher than a first threshold; wherein a historical item in the long-term memory is used to compare word overlap with the next utterance (input data)].”)
After careful consideration, the argument is considered unpersuasive as the Shuster discloses the external memory bank (memory store) to hold memories of the replies of the system (DL model ie Person 2) and the user (ie Person 1) (See figure 1 of Shuster: Memory Store Person 1’s Persona: I am a hair stylist and Person 2’s Persona: I love dogs wherein Examiner notes that the Memory Store is comprised of the user’s (Person 1) ‘memory’ and the system’s (Person 2) ‘memory’). Secondly, Examiner respectfully notes that the first intermediate result is obtained from the long-term memory store; which as described in Section 3.2.2 to be populated with the next utterance (ie input data). In other words, Shuster reads on the claim as recited as Shuster determines the intermediate result in part of similarity to the next utterance (input data) wherein training-data construction is interpreted as populating the long-term memory (to which the intermediate result is called from). Lastly, Harijan is replied upon to teach a particular format to hold the memories because Harijan is in the same field of facilitating conversations between users and is reasonably pertinent to the problem the inventor faced of storing of conversations and determining historical data relevant to the current conversation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer and Shuster to incorporate the teachings of Harijan in order provide a method of conversation tracking and storing in order to keep the next utterance on topic. (Harijan, “[0042] A conversation may be a message exchange between two or more recipients. A conversation may also be a message exchange discussing a single conversation topic or multiple conversation topics, where topics change quickly. A conversation could be a single message, for instance, in the case of a user or recipient changing topics and/or initiating a new conversation. A user of a messaging application may be involved in multiple conversations with multiple recipients, but also may be involved in multiple conversations with a single recipient. An embodiment of the present invention facilitates tracking conversations and storing conversation data to prevent a message from being sent in the wrong conversation.”)
Alleged No teaching of no proper motivation to combine Shazeer with Shuster
In Remarks p. 14, Applicant contends:
“As described in Shazeer, a single machine-learned language model 414, in token-by token generation, outputs tool tokens (such as <tool internet provider API>, <tool meter>, and
<tool cust-acct>) for tool invocation. Shuster, by contrast, employs a pipeline of separately
invoked modules ("Internet search decision," "Long-term memory access decision," "Extract
relevant entity," and "Generate dialogue response"), each with its own input construction and
control tokens. To accommodate Shuster' s binary classifiers within Shazeer would call for either
(i) the addition of a separate classification module outside Shazeer' s language model, which is
inconsistent with the architecture described in Shazeer, or (ii) restructuring of Shazeer into a
multi-module pipeline, which would change the very mechanism that Shazeer relies upon. Per
MPEP § 2143.0l(VI), where a proposed modification would change the principle of operation of
the prior art invention being modified, the teachings of the cited references are not sufficient to
render the claims prima facie obvious.”
After careful consideration, the argument is considered unpersuasive as Examiner notes that the rest of MPEP § 2143.0l(VI) recites “In re Ratti, 270 F.2d 810, 813, 123 USPQ 349, 352 (CCPA 1959) (Claims were directed to an oil seal comprising a bore engaging portion with outwardly biased resilient spring fingers inserted in a resilient sealing member. The primary reference relied upon in a rejection based on a combination of references disclosed an oil seal wherein the bore engaging portion was reinforced by a cylindrical sheet metal casing. The seal construction taught in the primary reference required rigidity for operation, whereas the seal in the claimed invention required resiliency. The court reversed the rejection holding the "suggested combination of references would require a substantial reconstruction and redesign of the elements shown in [the primary reference] as well as a change in the basic principle under which the [primary reference] construction was designed to operate.").” As noted, the primary reference required an opposing element of the same operation to the claimed invention.
Examiner respectfully points out that is not the case for Shazeer in view of Shuster. Both Shazeer and Shuster are in the same field of dialogue generation and invoking external tools to generate the dialogue output. Shuster providing the modules would be merely determining whether to call the external tools in determining the dialogue generation (to which Examiner notes that Shazeer discloses in para. [0024], “Approaches of the disclosure may achieve improved or optimized integration with external tools since a machine-learning process may be applied to minimize computational overhead in calling such services; for example, tool tokens and the order in which they are generated may be adapted to minimize computational overheads such as latency and/or network usage.” One of ordinary skills would thus, be motivated to combine Shazeer and Shuster to provide the decision classifier of Shuster to Shazeer as doing so would further improve interactions in a more responsible and useful manner (Shuster, Section 1, “The goal of this research program is then to explore how to construct models that continue to improve from such interactions both in terms of becoming more responsible and more useful.”)
Alleged No teaching of concatenation of the initial input and the first intermediate result
In Remarks p. 14-15, Applicant contends:
“Amended claim 1 further recites that the second input is "a concatenation of the initial
input and the first intermediate result." Shazeer describes that the additional information
generated by the structural tool 15 is "appended to a current version of the one or more
intermediate text strings 16" (Shazeer, paragraph [0044], emphasis added). The object that is
appended to in Shazeer is therefore the intermediate text string 16, and not the initial input (i.e.,
the contextual text token 12). The Applicant respectfully submits that Shazeer therefore does not
disclose or suggest the recited concatenation.”
The relevant claim limitations appear to be “the second input being a concatenation of the initial input and the first intermediate result;” in claim 1.
As noted in the previous Office Action, Shazeer teaches (emphasis added):
(Shazeer, “[0102] <tool cust-acct> [0103] customer id777 [determining a second input ie call to tool cust-acct using customer id777 for the deep learning model based at least on the initial input and the first intermediate result] [0104] <tool-output> [0105] John Doe [0106] Internet Provider Ultimate Plan [0107] expected speed: 1000 Mbps [0108] [the second input being a concatenation of the initial input and the first intermediate result… obtaining a second output (interpreted to be the same as the second input) of the deep learning model based on the second input; wherein a concatenation is the customer id (first intermediate result) and the call request for the information pertaining expected speed; the expected speed information follows the customer id] </tool>”; see annotated figure 1)
PNG
media_image5.png
425
781
media_image5.png
Greyscale
After careful consideration, the argument is considered unpersuasive as Examiner provides a definition of concatenation as obtained from “concatenation.” Dictionary.com, 2026. Web. 12 May 2026:
“a series of interconnected or interdependent things or events.
Human history is a concatenation of power struggles and people trying to survive.”
As shown from annotated figure 1, the second input can be interpreted as a concatenation of the initial input and the first intermediate results as each were a necessary event that together made a series that generates the second input.
The examiner refers to the rejection under 35 USC § 103 in the current office action for more details.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4, 6-7, 9-11, and 13-16 are rejected under 35 U.S.C. 103 as being unpatentable over US Pub No. US20220374608A1 Shazeer et al. (“Shazeer”) in view of Shuster, Kurt, et al. "Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage." arXiv preprint arXiv:2208.03188 (2022) (“Shuster”) in further view of US Pub No. US20200356587A1 Harijan et al. (“Harijan”).
In regards to claim 1,
Shazeer teaches A computer-implemented data generation method based on a deep learning model, wherein the deep learning model is able to generate reply data based on input data of a user,
(Shazeer, Abstract, “The present disclosure is directed to systems and methods that include and/or leverage one or more machine-learned language models that generate intermediate textual analysis (e.g., including usage of structural tools such as APIs) in service of contextual text generation. For example, a computing system can obtain a contextual text string that includes one or more contextual text tokens. The computing system can process the contextual text string with the machine-learned language model to generate one or more intermediate text strings that include one or more intermediate text tokens. The computing system can process the one or more intermediate text strings with the machine-learned language model to generate an output text string comprising one or more output text tokens. The one or more intermediate text strings can include textual analysis of the contextual text string that supports the output text string.”)
Shazeer teaches and the data generation method comprises: determining an initial input for the deep learning model based on input data from a user;
(Shazeer, “[0089] Context 2: [0090] Cust: ah ok . . . so i recovered it, but my internet is slow”)
Shazeer teaches obtaining a first output of the deep learning model, generating the first output that includes a first token for automatically calling a first functional component different from the deep learning model and a first intermediate inquiry determined based on the initial input and recognizable by the first functional component; obtaining a first intermediate result determined by the first functional component based on the first intermediate inquiry;
(Shazeer, “[0091] Intermediate analysis 2: [0092] <tool internet provider API> [0093] what to do when internet is slow [first token ie accessing the tool internet provider API for automatically calling a first functional component different from the deep learning model (see figure 4 wherein structural tools 415 is a separate module from ML language model 414) and a first intermediate inquiry ie “what to do when internet is slow” determined based on the initial input see context 2 (Shazeer, [0089]) and recognizable by the first functional component]
[0094] <tool-output> [0095] you just need to measure the speed with the custom Internet Provider meter [0096] </tool> [0097] <tool meter> [0098] customer id 777 [0099] <tool-output> [0100] 1234 Mbps [obtaining/generating a first output ie tool-output where in this example, it is 1234 Mbps of the deep learning model;] [0101] </tool>
[obtaining a first intermediate result ie tool-output where in this example, it is customer id 777 determined by the first functional component based on the first intermediate inquiry]”)
PNG
media_image6.png
384
706
media_image6.png
Greyscale
Shazeer teaches determining a second input for the deep learning model based at least on the initial input and the first intermediate result, the second input being a concatenation of the initial input and the first intermediate result; obtaining a second output of the deep learning model based on the second input;
(Shazeer, “[0102] <tool cust-acct> [0103] customer id777 [determining a second input ie call to tool cust-acct using customer id777 for the deep learning model based at least on the initial input and the first intermediate result] [0104] <tool-output> [0105] John Doe [0106] Internet Provider Ultimate Plan [0107] expected speed: 1000 Mbps [0108] [the second input being a concatenation of the initial input and the first intermediate result… obtaining a second output (interpreted to be the same as the second input) of the deep learning model based on the second input; wherein a concatenation is the customer id (first intermediate result) and the call request for the information pertaining expected speed; the expected speed information follows the customer id] </tool>”)
Shazeer teaches and generating a reply to the initial input, comprising: in response to the second output including a second token for automatically calling a second functional component (see figure 4 wherein structural tools 415 is a separate module from ML language model 414) and a second intermediate inquiry that is obtained based on the second input and recognizable by the second functional component, performing a corresponding function call operation for the second output, comprising: obtaining a second intermediate result determined by the second functional component based on the second intermediate inquiry;
(Shazeer, “[0102] <tool cust-acct> [0103] customer id777 [in response to the second output including a second token ie requested access to tool cust-acct for automatically calling a second functional component and a second intermediate inquiry ie request for the tool-output information with the provided customer id777 that is obtained based on the second input ie customer id777 and request for expected speed of the customer and recognizable by the second functional component] [0104] <tool-output> [0105] John Doe [0106] Internet Provider Ultimate Plan [0107] expected speed: 1000 Mbps [0108] [performing a corresponding function call operation for the second output ie 1000 Mbps, comprising: obtaining a second intermediate result ie expected speed: 1000 Mbps determined by the second functional component based on the second intermediate inquiry] </tool>”)
Shazeer teaches determining a third input for the deep learning model based at least on the second input and the second intermediate result, the third input being a concatenation of the second input and the second intermediate result;
obtaining a third output of the deep learning model based on the third input;
(Shazeer, “[0057] As shown in FIG. 4, one or more contextual tokens 412 can be input into a machine-learned language model 414 such as those described in FIGS. 1 and 2. The model 414 can generate intermediate text strings (e.g., which may include accessing or leveraging structural tools 415) [determining a third input for the deep learning model based at least on the second input and the second intermediate result, the third input being a concatenation of the second input and the second intermediate result;
obtaining a third output of the deep learning model based on the third input; wherein this third input and output would be another call to any one of deemed relevant structural tools 415]. Ultimately, the model 414 can generate one output token(s) 416. A reward function 418 can determine a reward based on the output tokens 416. The reward function 418 can determine how well the output tokens 416 satisfied or led to satisfaction of some objective (e.g., user satisfaction). Optionally, the output tokens 416 can be supplied to a user or other interactive agent to result in additional, new contextual tokens, which can restart the illustrated process.”)
Shazeer teaches and in response to the third output including no corresponding token for calling any functional component different from the deep learning model, using the third output as the reply to the initial input.
(Shazeer, “[0109] Output 2: [0110] Agent: ok. I'm seeing that you have bought the 1000 Mbps package, and you're getting 1234 Mbps, so everything is working as intended. Would you like to upgrade?”; wherein Shazeer teaches combined queried items as the reply and there are no subsequent tokens/further calling for structural tools)
Shazeer teaches wherein the first functional component is an external memory bank,
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database;”)
Shazeer teaches wherein the first intermediate inquiry is based on the input data,
(Shazeer, “[0091] Intermediate analysis 2: [0092] <tool internet provider API> [0093] what to do when internet is slow [wherein the first intermediate inquiry ie “what to do when internet is slow” is based on the input data see context 2 (Shazeer, [0089])]”)
Shazeer further teaches accessing a database for additional information
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database;”)
However, Shazeer does not explicitly teach the obtaining the first output including automatically determining by the deep learning model, whether an external functional component is to be called or whether a reply to the initial input is to be made without an external functional component being called, and in response to that it is determined that an external functional component is to be called, in response to that it is determined that a reply to the initial input is to be made without an external functional component being called, generate the first output that includes no corresponding token for calling any functional component different from the deep learning model, and using the first output as the reply to the initial input; and the external memory bank stores a first data group set related to the user, and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item [generated by the deep learning model for the historical input data item] and wherein the first intermediate result is a historical reply item corresponding to a historical input data item that is in the first data group set and whose similarity with the input data is higher than a first threshold
Shuster teaches the obtaining the first output including automatically determining by the deep learning model, whether an external functional component is to be called or whether a reply to the initial input is to be made without an external functional component being called, and in response to that it is determined that an external functional component is to be called,
(Shuster, Section 3.1, “Internet search decision Given the last turn of context, this module outputs whether internet search should be conducted or not…
Long-term memory access decision Given the last turn of context, and a store of (text-based) memories, output whether long-term memory access should be conducted or not.”)
Shuster teaches in response to that it is determined that a reply to the initial input is to be made without an external functional component being called, generate the first output that includes no corresponding token for calling any functional component different from the deep learning model, and using the first output as the reply to the initial input;
(Shuster, Section 3.1.1, “"[i]f neither search nor long-term memory access is required [in response to that it is determined that a reply to the initial input is to be made without an external functional component being called; ie neither external functional components should be called, thus Examiner notes that the control tokens indicating accessing the respective components (do search/access memory) are not included; See table 1], an entity is extracted from the history instead, and that is appended to the context (prefixed with control tokens)," after which the Generate dialogue response module is invoked [generate the first output ie generated dialogue response that includes no corresponding token for calling any functional component different from the deep learning model, and using the first output as the reply to the initial input].; Examiner further provides cropped figure 2 wherein figure 2 shows generating the dialogue response without calling any functional components”
PNG
media_image1.png
641
1058
media_image1.png
Greyscale
Further, Examiner respectfully points to Table 1 wherein Examiner notes different tokens are outputted for the different decisions:
PNG
media_image2.png
55
1041
media_image2.png
Greyscale
PNG
media_image3.png
52
1038
media_image3.png
Greyscale
)
Shuster teaches and the external memory bank stores a first data group set related to the user,
(Shuster, Section 3.1, “Access long-term memory Given the full input context, and a store of (text-based) memories [the external memory bank stores a first data group set related to the user (see Figure 1 Memory Store Person 1’s Persona: I am a hair stylist and Person 2’s Persona: I love dogs wherein Examiner notes that the Memory Store is comprised of the user’s (Person 1) ‘memory’ and the system’s (Person 2) ‘memory’)], output a memory from the memory store [wherein the first functional component is an external memory bank; wherein the memory store of Shuster is analogous to the database of Shazeer], referred to as a recalled memory. Note: if the memory store is too large to fit in the context, we adopt some simple strategies. For the 3B parameter model, we use the Fusion-in-Decoder method (Izacard and Grave, 2021). For the OPT-based models for simplicity of implementation, we sample the memories to fit in the 2048 token context. We keep those with overlapping keywords to prior turns.”)
Shuster teaches and wherein the first intermediate result is a historical reply item [corresponding to a historical input data item that is in the first data group set] and whose similarity with the input data is higher than a first threshold.
(Shuster, Section 3.2.2, “Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context, which is calculated as the one with the highest word overlap with the next utterance [whose similarity with the input data is higher than a first threshold; wherein a historical item in the long-term memory is used to compare word overlap with the next utterance (input data)].”)
However, Shuster does not explicitly teach and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item [generated by the deep learning model for the historical input data item]… a historical reply item corresponding to a historical input data item that is in the first data group set.
Harijan teaches and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item [generated by the deep learning model for the historical input data item],
(Harijan, “[0043] An embodiment that may be used to track conversations and store conversation data incorporates a conversation data structure and message data structure as depicted in FIG. 3 [wherein each data group in the first data group set ie a conversation data structure comprises at least a historical input data item ie user input (ex Fig. 3B 362) and a historical reply item ie another user’s input (ex Fig. 3B 364 wherein John W. can be substituted by the DL model of Shuster ie John is the chatbot)]. A messaging service or messaging application may generate a conversation data structure to capture message data and time data within a conversation. There are many data structures suitable for tracking and storing conversations such as an array, matrix, stack, linked list, tree, queue or string. Hierarchical data structures such as a tree may be beneficial for organization of priorities and/or topics and keywords. Data structures may be stored in several ways, including in one or more databases. Data structures can be stored both locally on the device and remotely in a network. A messaging service application or system may utilize a context engine, stored and executed by one or more of the processors and memory depicted in FIGS. 7 and 8, to store and track conversation data in the effort to prevent sending a message to an unintended recipient.
[0044] One embodiment may include a context engine storing a conversation in a conversation data structure such as conversation data structure 310 depicted in FIG. 3A. Conversation data structure 310 includes fields such as date/time field 312, message field 314, from field 316, keywords field 318, and topic field 320. Conversation data structure 310 includes one or more incoming and/or outgoing messages or entries such as entry 322, entry 324, entry 326, and entry 328. Each entry, e.g., entry 322, may include data in each of the respective data fields such as date/time field 312, message field 314, from field 316, keywords field 318, and topic field 320. To generate the data structure such as conversation data structure 310, the context engine would parse a sent or received message for data for each field. The messaging service or application would analyze each message in a conversation and record the date and time of the message in the date/time field 312, record message data to the message field 314, and record sender data to the from field 316. Sender data may be a name, telephone number, username, email address or other unique identifier.”)
PNG
media_image4.png
375
602
media_image4.png
Greyscale
Harijan teaches a historical reply item corresponding to a historical input data item that is in the first data group set
(Harijan, “[0043] An embodiment that may be used to track conversations and store conversation data incorporates a conversation data structure and message data structure as depicted in FIG. 3 [a historical reply item ie another user’s input (ex Fig. 3B 364 wherein John W. can be substituted by the ML Language model of Shazeer ie John is the chatbot) corresponding to a historical input data item ie user input (ex Fig. 3B 362) that is in the first data group set]. A messaging service or messaging application may generate a conversation data structure to capture message data and time data within a conversation. There are many data structures suitable for tracking and storing conversations such as an array, matrix, stack, linked list, tree, queue or string. Hierarchical data structures such as a tree may be beneficial for organization of priorities and/or topics and keywords. Data structures may be stored in several ways, including in one or more databases. Data structures can be stored both locally on the device and remotely in a network. A messaging service application or system may utilize a context engine, stored and executed by one or more of the processors and memory depicted in FIGS. 7 and 8, to store and track conversation data in the effort to prevent sending a message to an unintended recipient.”)
Shazeer and Shuster are both considered to be analogous to the claimed invention because they are in the same field of using ML models for NLP and utilizing API calls. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer to incorporate the teachings of Shuster in order to provide a decision classifier to determine whether to search or not based (Shuster, Section 3.2.2, “We can hence build a decision classifier based on whether humans used knowledge or not (per-turn) as the basis of whether we should search or not.”) to further improve interactions in a more responsible and useful manner (Shuster, Section 1, “The goal of this research program is then to explore how to construct models that continue to improve from such interactions both in terms of becoming more responsible and more useful.”)
Harijan is considered to be analogous to the claimed invention because they are in the same field of facilitating conversations between users and is reasonably pertinent to the problem the inventor faced of storing of conversations and determining historical data relevant to the current conversation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer in view of Shuster and Harijan to incorporate the teachings of Harijan in order provide a method of conversation tracking and storing in order to keep the next utterance on topic. (Harijan, “[0042] A conversation may be a message exchange between two or more recipients. A conversation may also be a message exchange discussing a single conversation topic or multiple conversation topics, where topics change quickly. A conversation could be a single message, for instance, in the case of a user or recipient changing topics and/or initiating a new conversation. A user of a messaging application may be involved in multiple conversations with multiple recipients, but also may be involved in multiple conversations with a single recipient. An embodiment of the present invention facilitates tracking conversations and storing conversation data to prevent a message from being sent in the wrong conversation.”)
In regards to claim 4,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 2,
Harijan teaches further comprising: in response to determining that a similarity between any data group in the first data group set and a first data group that is based on the input data and the reply is lower than a second threshold, entering the first data group into the first data group set.
(Harijan, “[0060] In step 602, the context engine analyzes the first conversation to generate a first conversation data structure. In step 604, the context engine receives a message intended to be sent to the recipient from the first conversation. In step 606 the context engine correlates the message with the first conversation data structure to determine a first relevance score. In step 608, the context engine compares the first relevance score to a predetermined threshold. If, in step 610, the relevance score is greater than or equal to the predetermined threshold then the message is sent in the first conversation in step 612. If, in step 614, the relevance score for the first conversation is less than the predetermined threshold [in response to determining that a similarity between any data group in the first data group set and a first data group that is based on the input data and the reply is lower than a second threshold] then the context engine must analyze a second conversation to generate a second conversation data structure. Then, in step 618, the context engine correlates the message with the second conversation data structure to determine a second relevance score. In step 620, the context engine compares the relevance score for the first conversation with the relevance score for the second conversation. If the first relevance score is greater than the second relevance score it may indicate a likelihood that the message is a better fit with the first conversation [entering the first data group into the first data group set; ie the message should go into the first conversation data structure] than the second conversation and the message should be sent, even if the first relevance did not meet the predetermined threshold.”)
In regards to claim 6,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 2,
Harijan teaches wherein each data group in the first data group set further comprises an entry time item corresponding to a historical input data item and a historical reply item that are in the group.
(Harijan, “[0044] One embodiment may include a context engine storing a conversation in a conversation data structure such as conversation data structure 310 depicted in FIG. 3A. Conversation data structure 310 includes fields such as date/time field 312 [wherein each data group in the first data group set further comprises an entry time item corresponding to a historical input data item and a historical reply item that are in the group], message field 314, from field 316, keywords field 318, and topic field 320. Conversation data structure 310 includes one or more incoming and/or outgoing messages or entries such as entry 322, entry 324, entry 326, and entry 328. Each entry, e.g., entry 322, may include data in each of the respective data fields such as date/time field 312, message field 314, from field 316, keywords field 318, and topic field 320. To generate the data structure such as conversation data structure 310, the context engine would parse a sent or received message for data for each field. The messaging service or application would analyze each message in a conversation and record the date and time of the message in the date/time field 312, record message data to the message field 314, and record sender data to the from field 316. Sender data may be a name, telephone number, username, email address or other unique identifier.”)
In regards to claim 7,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 6,
Shazeer teaches wherein the first intermediate inquiry is based on the input data,
(Shazeer, “[0091] Intermediate analysis 2: [0092] <tool internet provider API> [0093] what to do when internet is slow [wherein the first intermediate inquiry ie “what to do when internet is slow” is based on the input data see context 2 (Shazeer, [0089])]”)
Shazeer further teaches accessing a database for additional information
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database;”)
However, Shazeer does not explicitly teach and wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set and whose similarity with the input data is higher than a first threshold.
Shuster teaches and wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set and whose similarity with the input data is higher than a first threshold.
(Shuster, Section 3.2.2, “Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context, which is calculated as the one with the highest word overlap with the next utterance [wherein the first intermediate result is a historical reply item…. whose similarity with the input data is higher than a first threshold; wherein a historical item in the long-term memory is used to compare word overlap with the next utterance (input data)].”)
However, Shuster does not explicitly teach wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set
Harijan teaches wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set
(Harijan, “[0050] An embodiment may include the context engine correlating a message data structure and a conversation data structure by way of incorporating data from the date/time field 312 and data from the date/time field 332. For instance, a comparison of data from the date/time field 332 for message data structure 330 to data from the date/time field 312 from conversation data structure 310 may identify similar times or a smaller time gap, which may indicate a higher likelihood that the message is relevant to the corresponding conversation. It may be beneficial to weight correlation data or weight a relevance score based on the time gap between the message and the latest entry in the conversation data structure [wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set], because it is common for users to send messages to unintended recipients in older conversations and conversations where they had previously replied and the conversation was left open. A weighting inversely proportional to the time gap would help ensure timely replies had higher correlation values. In an embodiment, correlating a message data structure and a conversation data structure may simply be identifying the time gap between the message and the latest entry in the conversation data structure to calculate a relevance score based on how quick the response would be.”)
In regards to claim 9,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 1,
Shazeer teaches wherein the determining an initial input for the deep learning model comprises: obtaining, from an external memory bank based on the input data,
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database;”)
(Shazeer, “[0057]… Optionally, the output tokens 416 can be supplied to a user or other interactive agent to result in additional, new contextual tokens, which can restart the illustrated process.”; wherein the initial input can be determined on the previous structural tool calls)
However, Shazeer does not explicitly teach obtaining, from an external memory bank based on the input data, a historical reply item corresponding to a historical input data item whose similarity with the input data is higher than a first threshold; and determining the initial input based on the input data and the historical reply item, wherein the external memory bank stores a first data group set related to the user, and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
Shuster teaches a historical reply item corresponding to a historical input data item whose similarity with the input data is higher than a first threshold;
(Shuster, Section 3.2.2, “Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context, which is calculated as the one with the highest word overlap with the next utterance [obtaining, from an external memory bank based on the input data, whose similarity with the input data is higher than a first threshold; wherein a historical item in the long-term memory is used to compare word overlap with the next utterance (input data)].”)
Shuster teaches and determining the initial input based on the input data and the historical reply item,
(Shuster, Section 3.1, “BB3 is a modular system but the modules are not independent components – this is achieved by training a single transformer model to execute the modules, with special control codes in the input context telling the model which module it is executing [determining the initial input based on the input data and the historical reply item; wherein the input context contains the dialogue history]. The input context otherwise typically contains the dialogue history (sometimes truncated, depending on the module), with each speaker prefixed with their ID, either “Person 1:” or “Person 2:” in order to differentiate them.”)
Shuster teaches wherein the external memory bank stores a first data group set related to the user,
(Shuser, Section 3.1, “Access long-term memory Given the full input context, and a store of (text-based) memories [the external memory bank stores a first data group set related to the user (see Figure 1 Memory Store Person 1’s Persona: I am a hair stylist and Person 2’s Persona: I love dogs wherein Examiner notes that the Memory Store is comprised of the user’s (Person 1) ‘memory’ and the system’s (Person 2) ‘memory’)], output a memory from the memory store [wherein the first functional component is an external memory bank], referred to as a recalled memory. Note: if the memory store is too large to fit in the context, we adopt some simple strategies. For the 3B parameter model, we use the Fusion-in-Decoder method (Izacard and Grave, 2021). For the OPT-based models for simplicity of implementation, we sample the memories to fit in the 2048 token context. We keep those with overlapping keywords to prior turns.”)
However, Shuster does not explicitly teach a historical reply item corresponding to a historical input data item…
and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
Harijan teaches a historical reply item corresponding to a historical input data item…
(Harijan, “[0050] An embodiment may include the context engine correlating a message data structure and a conversation data structure by way of incorporating data from the date/time field 312 and data from the date/time field 332. For instance, a comparison of data from the date/time field 332 for message data structure 330 to data from the date/time field 312 from conversation data structure 310 may identify similar times or a smaller time gap, which may indicate a higher likelihood that the message is relevant to the corresponding conversation. It may be beneficial to weight correlation data or weight a relevance score based on the time gap between the message and the latest entry in the conversation data structure [a historical reply item corresponding to a historical input data item; ie an identified relevance between a two entries from different users], because it is common for users to send messages to unintended recipients in older conversations and conversations where they had previously replied and the conversation was left open. A weighting inversely proportional to the time gap would help ensure timely replies had higher correlation values. In an embodiment, correlating a message data structure and a conversation data structure may simply be identifying the time gap between the message and the latest entry in the conversation data structure to calculate a relevance score based on how quick the response would be.”)
Harijan teaches and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
(Harijan, “[0043] An embodiment that may be used to track conversations and store conversation data incorporates a conversation data structure and message data structure as depicted in FIG. 3 [wherein each data group in the first data group set ie a conversation data structure comprises at least a historical input data item ie user input (ex Fig. 3B 362) and a historical reply item ie another user’s input (ex Fig. 3B 364 wherein John W. can be substituted by the ML Language model of Shazeer ie John is the chatbot)]. A messaging service or messaging application may generate a conversation data structure to capture message data and time data within a conversation. There are many data structures suitable for tracking and storing conversations such as an array, matrix, stack, linked list, tree, queue or string. Hierarchical data structures such as a tree may be beneficial for organization of priorities and/or topics and keywords. Data structures may be stored in several ways, including in one or more databases. Data structures can be stored both locally on the device and remotely in a network. A messaging service application or system may utilize a context engine, stored and executed by one or more of the processors and memory depicted in FIGS. 7 and 8, to store and track conversation data in the effort to prevent sending a message to an unintended recipient.”)
In regards to claim 10,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 1,
Shazeer teaches wherein the initial input comprises context information of the input data.
(Shazeer, “[0022] Generally, the present disclosure is directed to systems and methods that include and/or leverage one or more machine-learned language models that generate intermediate textual analysis (e.g., including usage of structural tools such as APIs) in service of contextual text generation. For example, a computing system can obtain a contextual text string that includes one or more contextual text tokens.”; see figure 1)
PNG
media_image7.png
302
702
media_image7.png
Greyscale
In regards to claim 11,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 10,
Shuster teaches wherein the determining an initial input for the deep learning model comprises: obtaining, from an external memory bank, at least one pair of historical input data item and historical reply item whose similarity with the input data and the context information meets a fourth threshold;
(Shuster, Section 3.2.2, “Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context, which is calculated as the one with the highest word overlap [whose similarity with the input data and the context information ie a given context meets a fourth threshold] with the next utterance.”)
Shuster teaches and determining the initial input for the deep learning model based on the input data, the context information, and the at least one pair of historical input data item and historical reply item,
(Shuster, Section 3.1, “BB3 is a modular system but the modules are not independent components – this is achieved by training a single transformer model to execute the modules, with special control codes in the input context telling the model which module it is executing. The input context otherwise typically contains the dialogue history (sometimes truncated, depending on the module), with each speaker prefixed with their ID, either “Person 1:” or “Person 2:” in order to differentiate them [determining the initial input for the deep learning model based on the input data, the context information, and the at least one pair of historical input data item and historical reply item; see Figure 2 Memory Store wherein the historical reply item is generated by the DL model and stored in the Memory Store].”)
Shuster teaches wherein the external memory bank stores a first data group set related to the user,
(Shuser, Section 3.1, “Access long-term memory Given the full input context, and a store of (text-based) memories [the external memory bank stores a first data group set related to the user (see Figure 1 Memory Store Person 1’s Persona: I am a hair stylist and Person 2’s Persona: I love dogs wherein Examiner notes that the Memory Store is comprised of the user’s (Person 1) ‘memory’ and the system’s (Person 2) ‘memory’)], output a memory from the memory store [wherein the first functional component is an external memory bank], referred to as a recalled memory. Note: if the memory store is too large to fit in the context, we adopt some simple strategies. For the 3B parameter model, we use the Fusion-in-Decoder method (Izacard and Grave, 2021). For the OPT-based models for simplicity of implementation, we sample the memories to fit in the 2048 token context. We keep those with overlapping keywords to prior turns.”)
However, Shuster does not explicitly teach at least one pair of historical input data item and historical reply item…
and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
Harijan teaches at least one pair of historical input data item and historical reply item…
(Harijan, “[0050] An embodiment may include the context engine correlating a message data structure and a conversation data structure by way of incorporating data from the date/time field 312 and data from the date/time field 332. For instance, a comparison of data from the date/time field 332 for message data structure 330 to data from the date/time field 312 from conversation data structure 310 may identify similar times or a smaller time gap, which may indicate a higher likelihood that the message is relevant to the corresponding conversation. It may be beneficial to weight correlation data or weight a relevance score based on the time gap between the message and the latest entry in the conversation data structure [at least one pair of historical input data item and historical reply item; ie an identified relevance between a two entries from different users], because it is common for users to send messages to unintended recipients in older conversations and conversations where they had previously replied and the conversation was left open. A weighting inversely proportional to the time gap would help ensure timely replies had higher correlation values. In an embodiment, correlating a message data structure and a conversation data structure may simply be identifying the time gap between the message and the latest entry in the conversation data structure to calculate a relevance score based on how quick the response would be.”)
Harijan teaches and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
(Harijan, “[0043] An embodiment that may be used to track conversations and store conversation data incorporates a conversation data structure and message data structure as depicted in FIG. 3 [wherein each data group in the first data group set ie a conversation data structure comprises at least a historical input data item ie user input (ex Fig. 3B 362) and a historical reply item ie another user’s input (ex Fig. 3B 364 wherein John W. can be substituted by the DL model of Shuster ie John is the chatbot)]. A messaging service or messaging application may generate a conversation data structure to capture message data and time data within a conversation. There are many data structures suitable for tracking and storing conversations such as an array, matrix, stack, linked list, tree, queue or string. Hierarchical data structures such as a tree may be beneficial for organization of priorities and/or topics and keywords. Data structures may be stored in several ways, including in one or more databases. Data structures can be stored both locally on the device and remotely in a network. A messaging service application or system may utilize a context engine, stored and executed by one or more of the processors and memory depicted in FIGS. 7 and 8, to store and track conversation data in the effort to prevent sending a message to an unintended recipient.”)
In regards to claim 13,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 1,
Shazeer teaches wherein the determining a second input for the deep learning model based at least on the initial input and the first intermediate result comprises: determining the second input for the deep learning model based on the initial input, the first intermediate result, and the first intermediate inquiry.
(Shazeer, “[0102] <tool cust-acct> [0103] customer id777 [determining the second input for the deep learning model ie call to tool cust-acct using customer id777 based on the initial input, the first intermediate result, and the first intermediate inquiry] [0104] <tool-output> [0105] John Doe [0106] Internet Provider Ultimate Plan [0107] expected speed: 1000 Mbps [0108] </tool>”)
In regards to claim 14,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 1,
Shazeer teaches wherein generating the reply to the initial input comprises: in response to the second output including no token for calling any functional component different from the deep learning model, using the second output as the reply to the initial input.
(Shazeer, “[0057] As shown in FIG. 4, one or more contextual tokens 412 can be input into a machine-learned language model 414 such as those described in FIGS. 1 and 2. The model 414 can generate intermediate text strings (e.g., which may include accessing or leveraging structural tools 415). Ultimately, the model 414 can generate one output token(s) 416 [in response to the second output including no token for calling any functional component different from the deep learning model, using the second output as the reply to the initial input]. A reward function 418 can determine a reward based on the output tokens 416. The reward function 418 can determine how well the output tokens 416 satisfied or led to satisfaction of some objective (e.g., user satisfaction). Optionally, the output tokens 416 can be supplied to a user or other interactive agent to result in additional, new contextual tokens, which can restart the illustrated process.”)
In regards to claim 15,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 1,
Shazeer teaches wherein the generating the reply to the initial input comprises: in response to that an Nth output of the deep learning model comprises an Nth token for calling an Nth functional component and an Nth intermediate inquiry obtained based on an Nth input and recognizable by the Nth functional component, performing a function call operation corresponding to the Nth output until it is determined that an (N+1)th output comprises no corresponding token for calling any functional component different from the deep learning model, and using the (N+1)th output as the reply to the initial input, wherein N is an integer greater than 2.
(Shazeer, “[0057] As shown in FIG. 4, one or more contextual tokens 412 can be input into a machine-learned language model 414 such as those described in FIGS. 1 and 2. The model 414 can generate intermediate text strings (e.g., which may include accessing or leveraging structural tools 415) [in response to that an Nth output of the deep learning model comprises an Nth token for calling an Nth functional component and an Nth intermediate inquiry obtained based on an Nth input and recognizable by the Nth functional component, performing a function call operation corresponding to the Nth output… wherein N is an integer greater than 2; Shazeer places no limits on accessing the structural tools and explicitly recites the process can be restarted with the obtained context from the previous structural tool calls]. Ultimately, the model 414 can generate one output token(s) 416 [until it is determined that an (N+1)th output comprises no corresponding token for calling any functional component different from the deep learning model]. A reward function 418 can determine a reward based on the output tokens 416. The reward function 418 can determine how well the output tokens 416 satisfied or led to satisfaction of some objective (e.g., user satisfaction). Optionally, the output tokens 416 can be supplied to a user [using the (N+1)th output as the reply to the initial input] or other interactive agent to result in additional, new contextual tokens, which can restart the illustrated process.”)
In regards to claim 16,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 15,
Shazeer teaches wherein each of the second functional component and the Nth functional component is one in a functional component group comprising: an external search engine; a retrieval model obtained by joint training with the deep learning model; at least one application programming interface callable by the deep learning model; and an external memory bank, wherein the external memory bank stores a first data group set related to the user, and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include… a query service that queries results from a search engine, knowledge graph, or digital assistant”)
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Shazeer in view of Shuster in view of Harijan in further view of Xu, Yanwei, et al. "Scalable continual top-k keyword search in relational databases." Data & Knowledge Engineering 86 (2013): 206-223. (“Xu”)
In regards to claim 5,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 2,
Xu teaches further comprising: in response to determining that a similarity between a second data group in the first data group set and a first data group that is based on the input data and the reply is higher than a third threshold and determining that the first data group conflicts with the second data group, entering the first data group into the first data group set, and deleting the second data group from the first data group set.
(Xu, Section 1, “For continual keyword query evaluation, when the database is updated, two situations must be considered:
1. Database updates may change the existing top-k results: some top-k results may be replaced by new ones that are related to the new tuples, and some top-k results may be invalid due to deletions.
2. Database updates may change the relevance scores of existing results because the underlying statistics (e.g., word frequencies) are changed.
In this paper, we describe a system which can efficiently report the top-k results of every monitoring query while the database is being updated continually.
The outline of the system is as follows: When a continual query is issued, it is evaluated in a pipelined way to find the set of results whose upper bounds of relevance scores are higher than a threshold θ [in response to determining that a similarity between a second data group ie deleted tuples (invalid tuples that was in the existing top-k results) in the first data group set and a first data group ie new tuples that is based on the input data ie the query and the reply ie the result is higher than a third threshold; wherein a similarity of the two data groups are above a threshold] by calculating the upper bound of the future relevance score for every query result.
When the database is updated, we first update the relevance scores of the computed results, then find the new results whose upper bounds of relevance scores are larger than θ and delete the results containing the deleted tuples [determining that the first data group ie new tuples conflicts with the second data group ie deleted tuples, entering the first data group into the first data group set ie enter new tuples into the top-k results, and deleting the second data group ie delete invalid tuples from the top-k results from the first data group set].”)
Xu is considered to be analogous to the claimed invention because they are in the same field of search retrieval and data management. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer in view of Shuster and Harijan to incorporate the teachings of Xu in order to augment the external memory bank with an efficient method for information retrieval without requiring users (the chatbot of Shuster) to know the complex underlying database schemas (Xu, Abstract, “Keyword search in relational databases has been widely studied in recent years because it does not require users neither to master a certain structured query language nor to know the complex underlying database schemas. Most of existing methods focus on answering snapshot keyword queries in static databases. In practice, however, databases are updated frequently, and users may have long-term interests on specific topics. To deal with such a situation, it is necessary to build effective and efficient facility in a database system to support continual keyword queries . In this paper, we propose an efficient method for answering continual topk keyword queries over relational databases. The proposed method is built on an existing scheme of keyword search on relational data streams, but incorporates the ranking mechanisms into the query processing methods and makes two improvements to support efficient top-k keyword search in relational databases. Compared to the existing methods, our method is more efficient both in computing the topk results in a static database and in maintaining the topk results when the database continually being updated. Experimental results validate the effectiveness and efficiency of the proposed method.”)
Claim(s) 8 is rejected under 35 U.S.C. 103 as being unpatentable over Shazeer in view of Shuster in view of Harijan in further view of US Pat No. US5765179A Sumita et al. (“Sumita”)
In regards to claim 8,
Shazeer in view of Shuster and Harijan teaches The data generation method according to claim 6,
Sumita teaches further comprising: deleting a data group whose timelines expires from the external memory bank based on the entry time item.
PNG
media_image8.png
308
299
media_image8.png
Greyscale
(Sumita, Col. 5 line 60-Col. 6 line 3, “According to FIG. 6, when a new status data is received, the status data management unit 102 checks whether there is any room in the status data memory unit 103 or not (501), and deletes the oldest status data when there is no room (502) [deleting a data group whose timeliness expires (wherein a timeline expires when the memory capacity is reached) from the external memory bank (wherein the memory bank is provided by Shuster) based on the entry time item (and the entry time item is as identified in the conversation data structure provided by Harijan)]. Then, the status data management unit 102 writes this received new status data into the status data memory unit 103 (503), and notifies the writing success to the language processing server which executed this status data writing (504), while broadcasting the status data arrival notice indicating the arrival of this new status data to all the language processing servers (505).”)
Sumita is considered to be analogous to the claimed invention because they are in the same field of connecting multiple systems using natural language processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer in view of Shuster and Harijan to incorporate the teachings of Sumita in order to provide a method of sharing status data in order to improve the overall system performance (Sumita, Col. 2 lines 21-28, “It is therefore an object of the present invention to provide a language processing application system connecting and utilizing a plurality of language processing functions, capable of operating these language processing functions on different computer systems, and Improving a performance level of each language processing function by sharing status data such as user customization data, dictionary data, field data, etc. among these language processing functions.”)
Claim(s) 17 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Shuster in view of Shazeer and Harijan.
In regards to claim 17,
Shuster teaches A training method, wherein the training method comprises: obtaining first sample data, the first sample data comprising a first sample initial input and a first sample output, wherein the first sample initial input comprises an expression of intention of calling a first preset functional component different from a deep learning model to be trained, and wherein the first sample output comprises a first preset token for calling the first preset functional component and a first sample intermediate input recognizable by the first preset functional component; obtaining second sample data, the second sample data comprising a second sample initial input and a second sample output, wherein the second sample initial input comprises no expression of intention of calling any preset functional component different from the deep learning model to be trained, and wherein the second sample output comprises no corresponding token for calling any preset functional component;
(Shuster, Section 3.2.2, “Internet search decision We use several datasets as input context for the “do search” or “do not search” decision. We use the QA datsets SQuAD (Rajpurkar et al., 2016), TriviaQA (Joshi et al., 2017) and Natural Questions (NQ) (Kwiatkowski et al., 2019) as examples of “do search ”. We also use data from the Wizard of Wikipedia (WoW) (Dinan et al., 2019b) and Wizard of Internet (WizInt) tasks (Komeili et al., 2022). These datasets consist of training dialogues where some turns contain human-authored relevant knowledge responses given retrieved documents. We can hence build a decision classifier based on whether humans used knowledge or not (per-turn) as the basis of whether we should search or not.
We also use PersonaChat (PC) (Zhang et al., 2018), empathetic dialogues (ED) (Rashkin et al., 2019) and Multi-Session Chat (MSC) (Xu et al., 2022a) to derive training data. We employ the heuristic where, if there is an entity in the context, we use that instance as a training example for “do search” [obtaining first sample data, the first sample data comprising a first sample initial input and a first sample output, wherein the first sample initial input comprises an expression of intention of calling a first preset functional component different from the deep learning model to be trained, and wherein the first sample output comprises a preset first token for calling the first preset functional component and a first sample intermediate input recognizable by the first preset functional component; ie training example directed to do search], otherwise we use it as an example of “do not search” [obtaining second sample data, the second sample data comprising a second sample initial input and a second sample output, wherein the second sample initial input comprises no expression of intention of calling any preset functional component different from the deep learning model to be trained, and wherein the second sample output comprises no corresponding token for calling any preset functional component; ie training example directed to do not search].”; see also figures 16 and 18)
PNG
media_image9.png
495
772
media_image9.png
Greyscale
PNG
media_image10.png
660
879
media_image10.png
Greyscale
(Shuster, Appendix D, “Table 18 provides the prompts used for the OPT-175B baseline model when generating for each of the BB3 modules. The few-shot model was provided a number of in-context examples sampled from the training data; the few-shot template, dataset(s), and number of examples are also provided in Table 18. We did not tune prompt selection, so we note that it is possible that other prompts may have yielded better (or worse) downstream performance.”)
Shuster teaches processing the first sample initial input by using the deep learning model to be trained, to obtain a first predicted output; adjusting a parameter of the deep learning model to be trained based on a comparison between the first sample output and the first predicted output; processing the second sample initial input by using the deep learning model to be trained, to obtain a second predicted output; and adjusting a parameter of the deep learning model to be trained based on a comparison between the second sample output and the second predicted output, to obtain a trained deep learning model,
(Shuster, Section 3.2.2, “Generate internet search query We use the WizInt dataset which contains human-authored search queries during crowdsourced dialogue turns to directly train the internet search query generation module in a supervised fashion [processing the first/second sample initial input ie WizInt (also from the Internet Search decision wherein it was previous taught that it comprises of both sample types (search/do not search)) by using the deep learning model to be trained, to obtain a first/second predicted output ie model generated search query; adjusting a parameter of the deep learning model based on a comparison between the first/second sample output and the first/second predicted output; wherein supervised training is performed and thus, the parameters of the models are adjusted based on comparing the model generated output to the ground truth]. We also use the newly collected Feedback on Interactive Talk & Search (FITS) dataset2 (Xu et al., 2022b) of internet-augmented conversational tasks in a similar manner.”)
Shazeer teaches wherein the trained deep learning model is configured to generate a reply by implementing the method according to claim 1.
This limitation is taught by Shazeer in view of Shuster and Harijan in the entirety of claim 1 under 35 USC § 103.
Shazeer in view of Shuster are both considered to be analogous to the claimed invention because they are in the same field of using ML models for NLP and utilizing API calls to assist in response generation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shuster to incorporate the teachings of Shazeer in order to provide a method of contextual text generation as to improve interpretability (Shazeer, “[0009] One example aspect of the present disclosure is directed to a computing system for contextual text generation with improved interpretability.”)
Harijan is considered to be analogous to the claimed invention because they are in the same field of facilitating conversations between users and is reasonably pertinent to the problem the inventor faced of storing of conversations and determining historical data relevant to the current conversation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer in view of Shuster and Harijan to incorporate the teachings of Harijan in order provide a method of conversation tracking and storing in order to keep the next utterance on topic. (Harijan, “[0042] A conversation may be a message exchange between two or more recipients. A conversation may also be a message exchange discussing a single conversation topic or multiple conversation topics, where topics change quickly. A conversation could be a single message, for instance, in the case of a user or recipient changing topics and/or initiating a new conversation. A user of a messaging application may be involved in multiple conversations with multiple recipients, but also may be involved in multiple conversations with a single recipient. An embodiment of the present invention facilitates tracking conversations and storing conversation data to prevent a message from being sent in the wrong conversation.”)
In regards to claim 21,
Shuster and Shazeer and Harijan teaches The training method according to claim 17,
Shuster teaches further comprising: obtaining fourth sample data, the fourth sample data comprising a fourth sample initial input, a fourth sample intermediate input recognizable by an external memory bank, a sample memory result, and a fourth sample reply, wherein the fourth sample intermediate input is determined based on the fourth sample initial input; obtaining a predicted memory result determined by the external memory bank based on the fourth sample intermediate input; adjusting a parameter of the external memory bank based on a comparison between the predicted memory result and the sample memory result; determining a fourth sample target input for the deep learning model based at least on the fourth sample initial input and the sample memory result;
(Shuster, Section 3.2.2, “Generate a long-term memory The MSC dataset is exclusively used for this task as it contains crowdsourced examples of summarized facts derived from the last utterance of dialogue contexts in natural conversations. We use these summarized facts as the targets for training this module [adjusting a parameter of the external memory bank based on a comparison between the predicted memory result and the sample memory result; ie training the module based on the derived data as constructed below]. Long-term memory access decision MSC, ED, PC and BST are used to construct this task, in a similar way to the extract relevant entity task: if there is an entity present this is used as a positive example of memory access, otherwise it is not, in order to construct a binary prediction task [obtaining fourth sample data, the fourth sample data comprising a fourth sample initial input ie input of utterances with dialogue contexts, a fourth sample intermediate input ie binary prediction of whether to access the memory or not recognizable by an external memory bank, a sample memory result ie particular persona line, and a fourth sample reply ie given context, wherein the fourth sample intermediate input is determined based on the fourth sample initial input]. Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context [determining a fourth sample target input for the deep learning model based at least on the fourth sample initial input and the sample memory result], which is calculated as the one with the highest word overlap with the next utterance.”)
Shuster teaches processing the fourth sample target input by using the deep learning model, to obtain a fourth predicted reply;
and adjusting a parameter of the deep learning model based on a comparison between the fourth sample reply and the fourth predicted reply.
(Shuster, Section 3.2.2, “Generate dialogue response Final dialogue responses are trained [adjusting a parameter of the deep learning model based on a comparison between the fourth sample reply and the fourth predicted reply; wherein training is adjusting parameters of the deep learning model based on comparisons of given data] with a number of datasets. PC, ED, MSC, BST, WizInt and WoW are used for capturing personality, empathy, long-term memory, blending and knowledge as in BlenderBot 1 and 2. The new FITS dataset is also used for opendomain internet-driven tasks. In each case, the input context contains the usual dialogue of those tasks, concatenated to extra memory or knowledge sentences, when available. In WoW, WizInt and FITS each dialogue response is annotated with the relevant knowledge used to construct it in the original dataset, so we can make use of those gold knowledge responses. For PC, ED and BST we use the gold knowledge entity and/or memory that was calculated for the extract relevant entity and long-term memory access decision module tasks [processing the fourth sample target input by using the deep learning model, to obtain a fourth predicted reply]. We additionally add a number of task-oriented dialogue tasks: GoogleSGD (Rastogi et al., 2020) and Taskmaster 1, 2 & 3 (Byrne et al., 2019). Finally, we add the Funpedia task (Dinan et al., 2020b) – which involves learning to produce an engaging dialogue utterance given a wikipedia sentence – and the LIGHT (Urbanek et al., 2019) and LIGHT WILD (Shuster et al., 2021b) tasks – which are open-domain dialogue tasks grounded in a medieval fantasy setting – where the former was collected from crowdworkers, and the latter from real players of the LIGHT text-adventure game in an online deployment setting.”)
Claim(s) 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Shuster in view of Shazeer and Harijan in further view of CN Pub. No. CN113988157A Qu et al. (“Qu”) (Pub date: 01-28-2022)
In regards to claim 18,
Shuster and Shazeer and Harijan teaches The training method according to claim 17,
Shuster teaches further comprising: obtaining third sample data, the third sample data comprising a third sample initial input, a sample search inquiry, a plurality of sample search results, and a third sample reply of the deep learning model to be trained for the third sample initial input, wherein the sample search inquiry is a third sample intermediate input generated by the deep learning model to be trained based on the third sample initial input,
(Shuster, Section 3.2.2, “Internet search decision We use several datasets as input context for the “do search” or “do not search” decision [obtaining third sample data, the third sample data comprising a third sample initial input ie search/do not search]. We use the QA datsets SQuAD (Rajpurkar et al., 2016), TriviaQA (Joshi et al., 2017) and Natural Questions (NQ) (Kwiatkowski et al., 2019) as examples of “do search ”. We also use data from the Wizard of Wikipedia (WoW) (Dinan et al., 2019b) and Wizard of Internet (WizInt) tasks (Komeili et al., 2022). These datasets consist of training dialogues where some turns contain human-authored relevant knowledge responses given retrieved documents. We can hence build a decision classifier based on whether humans used knowledge or not (per-turn) as the basis of whether we should search or not. We also use PersonaChat (PC) (Zhang et al., 2018), empathetic dialogues (ED) (Rashkin et al., 2019) and Multi-Session Chat (MSC) (Xu et al., 2022a) to derive training data. We employ the heuristic where, if there is an entity in the context, we use that instance as a training example for “do search”, otherwise we use it as an example of “do not search”.
Generate internet search query We use the WizInt dataset which contains human-authored search queries during crowdsourced dialogue turns to directly train the internet search query generation module in a supervised fashion [a sample search inquiry; wherein the sample search inquiry is a third sample intermediate input generated by the deep learning model to be trained based on the third sample initial input]. We also use the newly collected Feedback on Interactive Talk & Search (FITS) dataset2 (Xu et al., 2022b) of internet-augmented conversational tasks in a similar manner.
Generate knowledge response We can again make use of the WoW, WizInt and FITS datasets, but in this case to learn to generate a knowledge response given a dialogue context and input document(s) [a plurality of sample search results], as those datasets contain crowdsourced human demonstrations of this task. We note in each case the knowledge response is a direct copy of some of the tokens in the source documents, and does not involve generating new tokens, sentences, phrases or summaries. Hence, this task aims to avoid model hallucination (made-up facts). We also use a set of QA tasks as well, where the answer is viewed as a knowledge response output (even if it is a short phrase). We use MS Marco (Nguyen et al., 2016), NQ, SQuAD and TriviaQA in this way, following Shuster et al. (2022). We use the “Natural Language Generation” competition track (NLGen v2.1) of MS MARCO, in which the annotator is told “provide your answer in a way in which it could be read from a smart speaker and make sense without any additional context”3 . As such, the original targets do not have direct overlap with one of the input documents in this task, so we modify the task to satisfy this constraint by finding the highest overlapping input sentence with the answer [a third sample reply of the deep learning model to be trained for the third sample initial input], and make that the target instead. If the F1 overlap is less than 0.5 we drop the example, leaving 281,658 examples out of the original 808,731. For NQ, three different settings are used: with all documents as input, with only the gold document, and with a sampled dialogue history context, following Adolphs et al. (2021).”)
Shuster teaches and the third sample intermediate input is recognizable by a retrieval model different from the deep learning model to be trained, and wherein the plurality of sample search results are results outputted by the retrieval model based on the sample search inquiry;
(Shuster, Section 3.1, “Internet search This module is not executed by the transformer but a call to the actual internet search engine [the third sample intermediate input ie internet search query is recognizable by a retrieval model ie an internet search engine different from the deep learning model]. It returns N documents/snippets [wherein the plurality of sample search results are results outputted by the retrieval model based on the sample search inquiry]. In our deployment we use Mojeek (https:// www.mojeek.com/).”)
However Shuster and Shazeer and Harijan does not explicitly teach performing a ranking operation on the plurality of sample search results based on a matching degree between each of the plurality of sample search results and the third sample reply, to obtain a ranked plurality of sample search results
Qu teaches performing a ranking operation on the plurality of sample search results based on a matching degree between each of the plurality of sample search results and the third sample reply, to obtain a ranked plurality of sample search results; and training the retrieval model based on the ranked plurality of sample search results.
(Qu, Detailed Description, “As another example, the obtained search term may be input into an initial semantic search model, a document related to the search term and a relevance score are searched in a document library through the initial semantic search model, the searched related documents are ranked according to the relevance score from high to low, the first document or the first few documents in the ranking are taken as positive example candidate documents, and the rest documents in the top N documents in the ranking are taken as negative example candidate documents [performing a ranking operation on the plurality of sample search results based on a matching degree between each of the plurality of sample search results and the third sample reply, to obtain a ranked plurality of sample search results]. It should be noted that the initial semantic search model refers to a semantic search model trained by an existing training method [training the retrieval model based on the ranked plurality of sample search results].”)
Qu is considered to be analogous to the claimed invention because they are in the same field of information retrieval. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shuster to incorporate the teachings of Qu in order to provide a training method for a retrieval model with a scheme to improve training efficiency (Qu, Abstract, “The disclosure provides a semantic retrieval network training method and device, electronic equipment and a storage medium, and relates to the field of data processing, in particular to the field of artificial intelligence such as natural language processing and deep learning. The specific implementation scheme is as follows: obtaining a training sample; the training sample comprises a search word and n candidate documents corresponding to the search word; wherein n is an integer greater than 1; inputting the training samples into the refined model to obtain n first relevancy output by the refined model, wherein each first relevancy is used for representing the relevancy between a corresponding candidate document and a search word; inputting the training sample into a semantic retrieval model, and obtaining n second relevance degrees output by the semantic retrieval model, wherein each second relevance degree is used for representing the relevance between a corresponding candidate document and a search word; and performing joint training on the semantic retrieval model and the fine ranking model according to the n first correlation degrees and the n second correlation degrees. This scheme can improve training efficiency, promotes the training effect.”)
In regards to claim 19,
Shuster, Shazeer, Harijan and Qu teaches The training method according to claim 18,
Qu teaches wherein the performing a ranking operation on the plurality of sample search results based on a matching degree between each of the plurality of sample search results and the third sample reply comprises: selecting a first sample search result having a highest current matching degree from the plurality of sample search results;
(Qu, “As an example, to reduce the computation cost, a portion of the relevant documents may be fetched and input to the initial fine-ranking model to compute the third degree of relevance. The implementation mode can be as follows: sequencing related documents according to the relevance between the documents and the search terms, wherein the relevance refers to the relevance output by an initial semantic retrieval model, and the relevance between the documents sequenced in the front and the search terms is higher; according to the ranking, the first 50 documents are taken out and input into the initial fine ranking model, and the third relevance between each document in the 50 documents and the search terms is obtained.
In step 403, a target third correlation with the highest score is determined in the third correlations.
And step 404, in response to that the target third relevance is greater than the first threshold, taking the document corresponding to the target third relevance as a positive example candidate document corresponding to the search term [selecting a first sample search result having a highest current matching degree from the plurality of sample search results].”)
Qu teaches and repeating the ranking operation on remaining parts of the plurality of sample search results based on a matching degree between each of the remaining parts and the updated third sample reply until completion of ranking all of the plurality of sample search results.
(Qu, Detailed Description, “As an example, the related documents obtained in step 401 are ranked according to the relevance to the search term, where the relevance refers to the relevance output by the initial semantic search model, and the higher the relevance between the top ranked documents and the search term is; according to the ranking, the relevance of each document and the search terms is respectively compared with a second threshold from front to back until the relevance of a certain document and the search terms is smaller than the second threshold, and the document and all documents after the document in the ranking are taken as target selection documents [repeating the ranking operation on remaining parts of the plurality of sample search results based on a matching degree between each of the remaining parts and the updated third sample reply until completion of ranking all of the plurality of sample search results]; in the target selection documents, n-1 documents are selected from the front to the back according to the ranking as negative example candidate documents of the search terms.”)
However, Qu does not explicitly teach deleting overlapping content between the third sample reply and the first sample search result to update the third sample reply;
Shuster teaches deleting overlapping content between the third sample reply and the first sample search result to update the third sample reply;
(Shuster, Section , “We use the “Natural Language Generation” competition track (NLGen v2.1) of MS MARCO, in which the annotator is told “provide your answer in a way in which it could be read from a smart speaker and make sense without any additional context”3 . As such, the original targets do not have direct overlap with one of the input documents in this task, so we modify the task to satisfy this constraint by finding the highest overlapping input sentence with the answer, and make that the target instead. If the F1 overlap is less than 0.5 we drop the example, leaving 281,658 examples out of the original 808,731 [deleting overlapping content between the third sample reply and the first sample search result ie input document provided by Qu to update the third sample reply]. For NQ, three different settings are used: with all documents as input, with only the gold document, and with a sampled dialogue history context, following Adolphs et al. (2021).”)”)
In regards to claim 20,
Shuster, Shazeer, Harijan and Qu teaches The training method according to claim 18,
Qu teaches wherein the retrieval model comprises a ranking sub-model and a recall sub-model, wherein the training the retrieval model based on the ranked plurality of sample search results comprises: training the ranking sub-model of the retrieval model based on the ranked plurality of sample search results;
(Qu, Detailed Description, “According to the semantic retrieval network training method disclosed by the embodiment of the disclosure, the training samples are respectively input into the fine ranking model [ranking sub-model] and the semantic retrieval model [recall sub-model] to obtain the first correlation degree output by the fine ranking model and the second correlation degree output by the semantic retrieval model, and the semantic retrieval model and the fine ranking model are jointly trained [training the ranking sub-model of the retrieval model based on the ranked plurality of sample search results] according to the first correlation degree and the second correlation degree, so that not only can the model training efficiency be improved, but also the trained effect can be improved, and the training cost is saved.”)
Qu teaches and using the trained ranking sub-model as a teacher model to train the recall sub-model.
(Qu, Detailed Description, “As another example, a penalty value for the refined model may be calculated based on a difference between the positive example candidate document and the first relevance to the search term, the negative example candidate document and the first relevance to the search term; calculating a loss value of the semantic retrieval model according to the difference value between the positive example candidate document and the second degree of correlation with the search word and the negative example candidate document and the second degree of correlation with the search word; carrying out weighted calculation on the loss value of the fine ranking model and the loss value of the semantic retrieval model to obtain a joint loss value; and performing joint training on the semantic retrieval model and the fine ranking model according to the joint loss value [using the trained ranking sub-model as a teacher model to train the recall sub-model].”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US Pub No. US20220004930A1 Baidu teaches functional components (Baidu, [0003])
US Pub No. US20210234814A1 Baidu teaches a conversation control system based on information relevant to the user input
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASMINE THAI whose telephone number is (703)756-5904. The examiner can normally be reached M-F 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.T./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights