DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 09/09/2025 have been fully considered but they are partially persuasive.
Regarding applicant’s remarks directed to the rejection of claims under 35 USC § 101, the applicant argues that the amended claims do not recite any abstract ideas and are directed to a practical application. Examiner respectfully disagrees that the amended claims do not recite any abstract ideas (see previous office action dated 07/18/2025), however, Examiner respectfully agrees that the amended claims are directed towards a practical application wherein the improvement is realized in the accuracy of the model through performing external calls as needed. Examiner withdraws the prior rejection of claims under 35 USC § 101.
Regarding applicant’s remarks directed to the rejection of claims under 35 USC § 102 and 103, the arguments are directed to newly amended limitations that were not previously examined by the examiner. Therefore, applicants arguments are rendered moot. Further in view of the amendments and further search, a new ground(s) of rejection is made. The examiner refers to the rejection under 35 USC § 102 and 103 in the current office action for more details.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 10, and 13-16 is rejected under 35 U.S.C. 102(a)(1) as being anticipated by US Pub No. US20220374608A1 Shazeer et al. (“Shazeer”)
In regards to claim 1,
Shazeer teaches A computer-implemented data generation method based on a deep learning model, wherein the deep learning model is able to generate reply data based on input data of a user,
(Shazeer, Abstract, “The present disclosure is directed to systems and methods that include and/or leverage one or more machine-learned language models that generate intermediate textual analysis (e.g., including usage of structural tools such as APIs) in service of contextual text generation. For example, a computing system can obtain a contextual text string that includes one or more contextual text tokens. The computing system can process the contextual text string with the machine-learned language model to generate one or more intermediate text strings that include one or more intermediate text tokens. The computing system can process the one or more intermediate text strings with the machine-learned language model to generate an output text string comprising one or more output text tokens. The one or more intermediate text strings can include textual analysis of the contextual text string that supports the output text string.”)
Shazeer teaches and the data generation method comprises: determining an initial input for the deep learning model based on input data from a user;
(Shazeer, “[0089] Context 2: [0090] Cust: ah ok . . . so i recovered it, but my internet is slow”)
Shazeer teaches obtaining a first output of the deep learning model, the first output including a first token for automatically calling a first functional component different from the deep learning model and a first intermediate inquiry determined based on the initial input and recognizable by the first functional component; obtaining a first intermediate result determined by the first functional component based on the first intermediate inquiry;
(Shazeer, “[0091] Intermediate analysis 2: [0092] <tool internet provider API> [0093] what to do when internet is slow [first token ie accessing the tool internet provider API for automatically calling a first functional component different from the deep learning model (see figure 4 wherein structural tools 415 is a separate module from ML language model 414) and a first intermediate inquiry ie “what to do when internet is slow” determined based on the initial input see context 2 (Shazeer, [0089]) and recognizable by the first functional component]
[0094] <tool-output> [0095] you just need to measure the speed with the custom Internet Provider meter [0096] </tool> [0097] <tool meter> [0098] customer id 777 [0099] <tool-output> [0100] 1234 Mbps [obtaining a first output ie tool-output where in this example, it is 1234 Mbps of the deep learning model;] [0101] </tool>
[obtaining a first intermediate result ie tool-output where in this example, it is customer id 777 determined by the first functional component based on the first intermediate inquiry]”)
PNG
media_image1.png
384
706
media_image1.png
Greyscale
Shazeer teaches determining a second input for the deep learning model based at least on the initial input and the first intermediate result, the second input being a concatenation of the initial input and the first intermediate result; obtaining a second output of the deep learning model based on the second input;
(Shazeer, “[0102] <tool cust-acct> [0103] customer id777 [determining a second input ie call to tool cust-acct using customer id777 for the deep learning model based at least on the initial input and the first intermediate result] [0104] <tool-output> [0105] John Doe [0106] Internet Provider Ultimate Plan [0107] expected speed: 1000 Mbps [0108] [the second input being a concatenation of the initial input and the first intermediate result… obtaining a second output (interpreted to be the same as the second input) of the deep learning model based on the second input; wherein a concatenation is the customer id (first intermediate result) and the call request for the information pertaining expected speed; the expected speed information follows the customer id] </tool>”)
Shazeer teaches and generating a reply to the initial input, comprising: in response to the second output including a second token for automatically calling a second functional component (see figure 4 wherein structural tools 415 is a separate module from ML language model 414) and a second intermediate inquiry that is obtained based on the second input and recognizable by the second functional component, performing a corresponding function call operation for the second output, comprising: obtaining a second intermediate result determined by the second functional component based on the second intermediate inquiry;
(Shazeer, “[0102] <tool cust-acct> [0103] customer id777 [in response to the second output including a second token ie requested access to tool cust-acct for automatically calling a second functional component and a second intermediate inquiry ie request for the tool-output information with the provided customer id777 that is obtained based on the second input ie customer id777 and request for expected speed of the customer and recognizable by the second functional component] [0104] <tool-output> [0105] John Doe [0106] Internet Provider Ultimate Plan [0107] expected speed: 1000 Mbps [0108] [performing a corresponding function call operation for the second output ie 1000 Mbps, comprising: obtaining a second intermediate result ie expected speed: 1000 Mbps determined by the second functional component based on the second intermediate inquiry] </tool>”)
Shazeer teaches determining a third input for the deep learning model based at least on the second input and the second intermediate result, the third input being a concatenation of the second input and the second intermediate result;
obtaining a third output of the deep learning model based on the third input;
(Shazeer, “[0057] As shown in FIG. 4, one or more contextual tokens 412 can be input into a machine-learned language model 414 such as those described in FIGS. 1 and 2. The model 414 can generate intermediate text strings (e.g., which may include accessing or leveraging structural tools 415) [determining a third input for the deep learning model based at least on the second input and the second intermediate result, the third input being a concatenation of the second input and the second intermediate result;
obtaining a third output of the deep learning model based on the third input; wherein this third input and output would be another call to any one of deemed relevant structural tools 415]. Ultimately, the model 414 can generate one output token(s) 416. A reward function 418 can determine a reward based on the output tokens 416. The reward function 418 can determine how well the output tokens 416 satisfied or led to satisfaction of some objective (e.g., user satisfaction). Optionally, the output tokens 416 can be supplied to a user or other interactive agent to result in additional, new contextual tokens, which can restart the illustrated process.”)
Shazeer teaches and in response to the third output including no corresponding token for calling any functional component different from the deep learning model, using the third output as the reply to the initial input.
(Shazeer, “[0109] Output 2: [0110] Agent: ok. I'm seeing that you have bought the 1000 Mbps package, and you're getting 1234 Mbps, so everything is working as intended. Would you like to upgrade?”; wherein Shazeer teaches combined queried items as the reply and there are no subsequent tokens/further calling for structural tools)
In regards to claim 10,
Shazeer teaches The data generation method according to claim 1,
Shazeer teaches wherein the initial input comprises context information of the input data.
(Shazeer, “[0022] Generally, the present disclosure is directed to systems and methods that include and/or leverage one or more machine-learned language models that generate intermediate textual analysis (e.g., including usage of structural tools such as APIs) in service of contextual text generation. For example, a computing system can obtain a contextual text string that includes one or more contextual text tokens.”; see figure 1)
PNG
media_image2.png
302
702
media_image2.png
Greyscale
In regards to claim 13,
Shazeer teaches The data generation method according to claim 1,
Shazeer teaches wherein the determining a second input for the deep learning model based at least on the initial input and the first intermediate result comprises: determining the second input for the deep learning model based on the initial input, the first intermediate result, and the first intermediate inquiry.
(Shazeer, “[0102] <tool cust-acct> [0103] customer id777 [determining the second input for the deep learning model ie call to tool cust-acct using customer id777 based on the initial input, the first intermediate result, and the first intermediate inquiry] [0104] <tool-output> [0105] John Doe [0106] Internet Provider Ultimate Plan [0107] expected speed: 1000 Mbps [0108] </tool>”)
In regards to claim 14,
Shazeer teaches The data generation method according to claim 1,
Shazeer teaches wherein generating the reply to the initial input comprises: in response to the second output including no token for calling any functional component different from the deep learning model, using the second output as the reply to the initial input.
(Shazeer, “[0057] As shown in FIG. 4, one or more contextual tokens 412 can be input into a machine-learned language model 414 such as those described in FIGS. 1 and 2. The model 414 can generate intermediate text strings (e.g., which may include accessing or leveraging structural tools 415). Ultimately, the model 414 can generate one output token(s) 416 [in response to the second output including no token for calling any functional component different from the deep learning model, using the second output as the reply to the initial input]. A reward function 418 can determine a reward based on the output tokens 416. The reward function 418 can determine how well the output tokens 416 satisfied or led to satisfaction of some objective (e.g., user satisfaction). Optionally, the output tokens 416 can be supplied to a user or other interactive agent to result in additional, new contextual tokens, which can restart the illustrated process.”)
In regards to claim 15,
Shazeer teaches The data generation method according to claim 1,
Shazeer teaches wherein the generating the reply to the initial input comprises: in response to that an Nth output of the deep learning model comprises an Nth token for calling an Nth functional component and an Nth intermediate inquiry obtained based on an Nth input and recognizable by the Nth functional component, performing a function call operation corresponding to the Nth output until it is determined that an (N+1)th output comprises no corresponding token for calling any functional component different from the deep learning model, and using the (N+1)th output as the reply to the initial input, wherein N is an integer greater than 2.
(Shazeer, “[0057] As shown in FIG. 4, one or more contextual tokens 412 can be input into a machine-learned language model 414 such as those described in FIGS. 1 and 2. The model 414 can generate intermediate text strings (e.g., which may include accessing or leveraging structural tools 415) [in response to that an Nth output of the deep learning model comprises an Nth token for calling an Nth functional component and an Nth intermediate inquiry obtained based on an Nth input and recognizable by the Nth functional component, performing a function call operation corresponding to the Nth output… wherein N is an integer greater than 2; Shazeer places no limits on accessing the structural tools and explicitly recites the process can be restarted with the obtained context from the previous structural tool calls]. Ultimately, the model 414 can generate one output token(s) 416 [until it is determined that an (N+1)th output comprises no corresponding token for calling any functional component different from the deep learning model]. A reward function 418 can determine a reward based on the output tokens 416. The reward function 418 can determine how well the output tokens 416 satisfied or led to satisfaction of some objective (e.g., user satisfaction). Optionally, the output tokens 416 can be supplied to a user [using the (N+1)th output as the reply to the initial input] or other interactive agent to result in additional, new contextual tokens, which can restart the illustrated process.”)
In regards to claim 16,
Shazeer teaches The data generation method according to claim 15,
Shazeer teaches wherein each of the second functional component and the Nth functional component is one in a functional component group comprising: an external search engine; a retrieval model obtained by joint training with the deep learning model; at least one application programming interface callable by the deep learning model; and an external memory bank, wherein the external memory bank stores a first data group set related to the user, and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include… a query service that queries results from a search engine, knowledge graph, or digital assistant”)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 2-4, 6, 7, 9, and 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Shazeer in view of Shuster, Kurt, et al. "Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage." arXiv preprint arXiv:2208.03188 (2022) (“Shuster”) in further view of US Pub No. US20200356587A1 Harijan et al. (“Harijan”).
In regards to claim 2,
Shazeer teaches The data generation method according to claim 1,
Shazeer teaches wherein the first functional component is an external memory bank,
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database;”)
However, Shazeer does not explicitly teach and the external memory bank stores a first data group set related to the user, and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item
Shuster teaches and the external memory bank stores a first data group set related to the user,
(Shuser, Section 3.1, “Access long-term memory Given the full input context, and a store of (text-based) memories [the external memory bank stores a first data group set related to the user (see Figure 1 Memory Store Person 1’s Persona: I am a hair stylist and Person 2’s Persona: I love dogs wherein Examiner notes that the Memory Store is comprised of the user’s (Person 1) ‘memory’ and the system’s (Person 2) ‘memory’)], output a memory from the memory store [wherein the first functional component is an external memory bank], referred to as a recalled memory. Note: if the memory store is too large to fit in the context, we adopt some simple strategies. For the 3B parameter model, we use the Fusion-in-Decoder method (Izacard and Grave, 2021). For the OPT-based models for simplicity of implementation, we sample the memories to fit in the 2048 token context. We keep those with overlapping keywords to prior turns.”)
However, Shuster does not explicitly teach and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item [generated by the deep learning model for the historical input data item].
Harijan teaches wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item [generated by the deep learning model for the historical input data item]
(Harijan, “[0043] An embodiment that may be used to track conversations and store conversation data incorporates a conversation data structure and message data structure as depicted in FIG. 3 [wherein each data group in the first data group set ie a conversation data structure comprises at least a historical input data item ie user input (ex Fig. 3B 362) and a historical reply item ie another user’s input (ex Fig. 3B 364 wherein John W. can be substituted by the DL model of Shuster ie John is the chatbot)]. A messaging service or messaging application may generate a conversation data structure to capture message data and time data within a conversation. There are many data structures suitable for tracking and storing conversations such as an array, matrix, stack, linked list, tree, queue or string. Hierarchical data structures such as a tree may be beneficial for organization of priorities and/or topics and keywords. Data structures may be stored in several ways, including in one or more databases. Data structures can be stored both locally on the device and remotely in a network. A messaging service application or system may utilize a context engine, stored and executed by one or more of the processors and memory depicted in FIGS. 7 and 8, to store and track conversation data in the effort to prevent sending a message to an unintended recipient.
[0044] One embodiment may include a context engine storing a conversation in a conversation data structure such as conversation data structure 310 depicted in FIG. 3A. Conversation data structure 310 includes fields such as date/time field 312, message field 314, from field 316, keywords field 318, and topic field 320. Conversation data structure 310 includes one or more incoming and/or outgoing messages or entries such as entry 322, entry 324, entry 326, and entry 328. Each entry, e.g., entry 322, may include data in each of the respective data fields such as date/time field 312, message field 314, from field 316, keywords field 318, and topic field 320. To generate the data structure such as conversation data structure 310, the context engine would parse a sent or received message for data for each field. The messaging service or application would analyze each message in a conversation and record the date and time of the message in the date/time field 312, record message data to the message field 314, and record sender data to the from field 316. Sender data may be a name, telephone number, username, email address or other unique identifier.”)
PNG
media_image3.png
375
602
media_image3.png
Greyscale
Shazeer and Shuster are both considered to be analogous to the claimed invention because they are in the same field of using ML models for NLP and utilizing API calls. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer to incorporate the teachings of Shuster in order to construct models that improves interactions by providing and storing personas (Shuster, Section 1, “The goal of this research program is then to explore how to construct models that continue to improve from such interactions both in terms of becoming more responsible and more useful.”)
Harijan is considered to be analogous to the claimed invention because they are in the same field of facilitating conversations between users and is reasonably pertinent to the problem the inventor faced of storing of conversations and determining historical data relevant to the current conversation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer and Shuster to incorporate the teachings of Harijan in order provide a method of conversation tracking and storing in order to keep the next utterance on topic. (Harijan, “[0042] A conversation may be a message exchange between two or more recipients. A conversation may also be a message exchange discussing a single conversation topic or multiple conversation topics, where topics change quickly. A conversation could be a single message, for instance, in the case of a user or recipient changing topics and/or initiating a new conversation. A user of a messaging application may be involved in multiple conversations with multiple recipients, but also may be involved in multiple conversations with a single recipient. An embodiment of the present invention facilitates tracking conversations and storing conversation data to prevent a message from being sent in the wrong conversation.”)
In regards to claim 3,
Shazeer and Shuster and Harijan teaches The data generation method according to claim 2,
Shazeer teaches wherein the first intermediate inquiry is based on the input data,
(Shazeer, “[0091] Intermediate analysis 2: [0092] <tool internet provider API> [0093] what to do when internet is slow [wherein the first intermediate inquiry ie “what to do when internet is slow” is based on the input data see context 2 (Shazeer, [0089])]”)
Shazeer further teaches accessing a database for additional information
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database;”)
However, Shazeer does not explicitly teach and wherein the first intermediate result is a historical reply item corresponding to a historical input data item that is in the first data group set and whose similarity with the input data is higher than a first threshold
Shuster teaches and wherein the first intermediate result is a historical reply item corresponding to a historical input data item that is in the first data group set and whose similarity with the input data is higher than a first threshold.
(Shuster, Section 3.2.2, “Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context, which is calculated as the one with the highest word overlap with the next utterance [whose similarity with the input data is higher than a first threshold; wherein a historical item in the long-term memory is used to compare word overlap with the next utterance (input data)].”)
However, Shuster does not explicitly teach a historical reply item corresponding to a historical input data item that is in the first data group set
Harijan teaches a historical reply item corresponding to a historical input data item that is in the first data group set
(Harijan, “[0043] An embodiment that may be used to track conversations and store conversation data incorporates a conversation data structure and message data structure as depicted in FIG. 3 [a historical reply item ie another user’s input (ex Fig. 3B 364 wherein John W. can be substituted by the ML Language model of Shazeer ie John is the chatbot) corresponding to a historical input data item ie user input (ex Fig. 3B 362) that is in the first data group set]. A messaging service or messaging application may generate a conversation data structure to capture message data and time data within a conversation. There are many data structures suitable for tracking and storing conversations such as an array, matrix, stack, linked list, tree, queue or string. Hierarchical data structures such as a tree may be beneficial for organization of priorities and/or topics and keywords. Data structures may be stored in several ways, including in one or more databases. Data structures can be stored both locally on the device and remotely in a network. A messaging service application or system may utilize a context engine, stored and executed by one or more of the processors and memory depicted in FIGS. 7 and 8, to store and track conversation data in the effort to prevent sending a message to an unintended recipient.”)
In regards to claim 4,
Shazeer and Shuster and Harijan teaches The data generation method according to claim 2,
Harijan teaches further comprising: in response to determining that a similarity between any data group in the first data group set and a first data group that is based on the input data and the reply is lower than a second threshold, entering the first data group into the first data group set.
(Harijan, “[0060] In step 602, the context engine analyzes the first conversation to generate a first conversation data structure. In step 604, the context engine receives a message intended to be sent to the recipient from the first conversation. In step 606 the context engine correlates the message with the first conversation data structure to determine a first relevance score. In step 608, the context engine compares the first relevance score to a predetermined threshold. If, in step 610, the relevance score is greater than or equal to the predetermined threshold then the message is sent in the first conversation in step 612. If, in step 614, the relevance score for the first conversation is less than the predetermined threshold [in response to determining that a similarity between any data group in the first data group set and a first data group that is based on the input data and the reply is lower than a second threshold] then the context engine must analyze a second conversation to generate a second conversation data structure. Then, in step 618, the context engine correlates the message with the second conversation data structure to determine a second relevance score. In step 620, the context engine compares the relevance score for the first conversation with the relevance score for the second conversation. If the first relevance score is greater than the second relevance score it may indicate a likelihood that the message is a better fit with the first conversation [entering the first data group into the first data group set; ie the message should go into the first conversation data structure] than the second conversation and the message should be sent, even if the first relevance did not meet the predetermined threshold.”)
In regards to claim 6,
Shazeer and Shuster and Harijan teaches The data generation method according to claim 2,
Harijan teaches wherein each data group in the first data group set further comprises an entry time item corresponding to a historical input data item and a historical reply item that are in the group.
(Harijan, “[0044] One embodiment may include a context engine storing a conversation in a conversation data structure such as conversation data structure 310 depicted in FIG. 3A. Conversation data structure 310 includes fields such as date/time field 312 [wherein each data group in the first data group set further comprises an entry time item corresponding to a historical input data item and a historical reply item that are in the group], message field 314, from field 316, keywords field 318, and topic field 320. Conversation data structure 310 includes one or more incoming and/or outgoing messages or entries such as entry 322, entry 324, entry 326, and entry 328. Each entry, e.g., entry 322, may include data in each of the respective data fields such as date/time field 312, message field 314, from field 316, keywords field 318, and topic field 320. To generate the data structure such as conversation data structure 310, the context engine would parse a sent or received message for data for each field. The messaging service or application would analyze each message in a conversation and record the date and time of the message in the date/time field 312, record message data to the message field 314, and record sender data to the from field 316. Sender data may be a name, telephone number, username, email address or other unique identifier.”)
In regards to claim 7,
Shazeer and Shuster and Harijan teaches The data generation method according to claim 6,
Shazeer teaches wherein the first intermediate inquiry is based on the input data,
(Shazeer, “[0091] Intermediate analysis 2: [0092] <tool internet provider API> [0093] what to do when internet is slow [wherein the first intermediate inquiry ie “what to do when internet is slow” is based on the input data see context 2 (Shazeer, [0089])]”)
Shazeer further teaches accessing a database for additional information
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database;”)
However, Shazeer does not explicitly teach and wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set and whose similarity with the input data is higher than a first threshold.
Shuster teaches and wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set and whose similarity with the input data is higher than a first threshold.
(Shuster, Section 3.2.2, “Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context, which is calculated as the one with the highest word overlap with the next utterance [wherein the first intermediate result is a historical reply item…. whose similarity with the input data is higher than a first threshold; wherein a historical item in the long-term memory is used to compare word overlap with the next utterance (input data)].”)
However, Shuster does not explicitly teach wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set
Harijan teaches wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set
(Harijan, “[0050] An embodiment may include the context engine correlating a message data structure and a conversation data structure by way of incorporating data from the date/time field 312 and data from the date/time field 332. For instance, a comparison of data from the date/time field 332 for message data structure 330 to data from the date/time field 312 from conversation data structure 310 may identify similar times or a smaller time gap, which may indicate a higher likelihood that the message is relevant to the corresponding conversation. It may be beneficial to weight correlation data or weight a relevance score based on the time gap between the message and the latest entry in the conversation data structure [wherein the first intermediate result is a historical reply item corresponding to a historical input data item that has a latest time stamp in the first data group set], because it is common for users to send messages to unintended recipients in older conversations and conversations where they had previously replied and the conversation was left open. A weighting inversely proportional to the time gap would help ensure timely replies had higher correlation values. In an embodiment, correlating a message data structure and a conversation data structure may simply be identifying the time gap between the message and the latest entry in the conversation data structure to calculate a relevance score based on how quick the response would be.”)
In regards to claim 9,
Shazeer teaches The data generation method according to claim 1,
Shazeer teaches wherein the determining an initial input for the deep learning model comprises: obtaining, from an external memory bank based on the input data,
(Shazeer, “[0025] As examples, structural tools that the machine-learned language model may have access to include: a database lookup to access additional information from a database;”)
(Shazeer, “[0057]… Optionally, the output tokens 416 can be supplied to a user or other interactive agent to result in additional, new contextual tokens, which can restart the illustrated process.”; wherein the initial input can be determined on the previous structural tool calls)
However, Shazeer does not explicitly teach obtaining, from an external memory bank based on the input data, a historical reply item corresponding to a historical input data item whose similarity with the input data is higher than a first threshold; and determining the initial input based on the input data and the historical reply item, wherein the external memory bank stores a first data group set related to the user, and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
Shuster teaches a historical reply item corresponding to a historical input data item whose similarity with the input data is higher than a first threshold;
(Shuster, Section 3.2.2, “Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context, which is calculated as the one with the highest word overlap with the next utterance [obtaining, from an external memory bank based on the input data, whose similarity with the input data is higher than a first threshold; wherein a historical item in the long-term memory is used to compare word overlap with the next utterance (input data)].”)
Shuster teaches and determining the initial input based on the input data and the historical reply item,
(Shuster, Section 3.1, “BB3 is a modular system but the modules are not independent components – this is achieved by training a single transformer model to execute the modules, with special control codes in the input context telling the model which module it is executing [determining the initial input based on the input data and the historical reply item; wherein the input context contains the dialogue history]. The input context otherwise typically contains the dialogue history (sometimes truncated, depending on the module), with each speaker prefixed with their ID, either “Person 1:” or “Person 2:” in order to differentiate them.”)
Shuster teaches wherein the external memory bank stores a first data group set related to the user,
(Shuser, Section 3.1, “Access long-term memory Given the full input context, and a store of (text-based) memories [the external memory bank stores a first data group set related to the user (see Figure 1 Memory Store Person 1’s Persona: I am a hair stylist and Person 2’s Persona: I love dogs wherein Examiner notes that the Memory Store is comprised of the user’s (Person 1) ‘memory’ and the system’s (Person 2) ‘memory’)], output a memory from the memory store [wherein the first functional component is an external memory bank], referred to as a recalled memory. Note: if the memory store is too large to fit in the context, we adopt some simple strategies. For the 3B parameter model, we use the Fusion-in-Decoder method (Izacard and Grave, 2021). For the OPT-based models for simplicity of implementation, we sample the memories to fit in the 2048 token context. We keep those with overlapping keywords to prior turns.”)
However, Shuster does not explicitly teach a historical reply item corresponding to a historical input data item…
and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
Harijan teaches a historical reply item corresponding to a historical input data item…
(Harijan, “[0050] An embodiment may include the context engine correlating a message data structure and a conversation data structure by way of incorporating data from the date/time field 312 and data from the date/time field 332. For instance, a comparison of data from the date/time field 332 for message data structure 330 to data from the date/time field 312 from conversation data structure 310 may identify similar times or a smaller time gap, which may indicate a higher likelihood that the message is relevant to the corresponding conversation. It may be beneficial to weight correlation data or weight a relevance score based on the time gap between the message and the latest entry in the conversation data structure [a historical reply item corresponding to a historical input data item; ie an identified relevance between a two entries from different users], because it is common for users to send messages to unintended recipients in older conversations and conversations where they had previously replied and the conversation was left open. A weighting inversely proportional to the time gap would help ensure timely replies had higher correlation values. In an embodiment, correlating a message data structure and a conversation data structure may simply be identifying the time gap between the message and the latest entry in the conversation data structure to calculate a relevance score based on how quick the response would be.”)
Harijan teaches and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
(Harijan, “[0043] An embodiment that may be used to track conversations and store conversation data incorporates a conversation data structure and message data structure as depicted in FIG. 3 [wherein each data group in the first data group set ie a conversation data structure comprises at least a historical input data item ie user input (ex Fig. 3B 362) and a historical reply item ie another user’s input (ex Fig. 3B 364 wherein John W. can be substituted by the ML Language model of Shazeer ie John is the chatbot)]. A messaging service or messaging application may generate a conversation data structure to capture message data and time data within a conversation. There are many data structures suitable for tracking and storing conversations such as an array, matrix, stack, linked list, tree, queue or string. Hierarchical data structures such as a tree may be beneficial for organization of priorities and/or topics and keywords. Data structures may be stored in several ways, including in one or more databases. Data structures can be stored both locally on the device and remotely in a network. A messaging service application or system may utilize a context engine, stored and executed by one or more of the processors and memory depicted in FIGS. 7 and 8, to store and track conversation data in the effort to prevent sending a message to an unintended recipient.”)
Shazeer and Shuster are both considered to be analogous to the claimed invention because they are in the same field of using ML models for NLP and utilizing API calls to assist in response generation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer to incorporate the teachings of Shuster in order to construct models that improves interactions by providing and storing personas (Shuster, Section 1, “The goal of this research program is then to explore how to construct models that continue to improve from such interactions both in terms of becoming more responsible and more useful.”)
Harijan is considered to be analogous to the claimed invention because they are in the same field of facilitating conversations between users and is reasonably pertinent to the problem the inventor faced of storing of conversations and determining historical data relevant to the current conversation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Shazeer and Shuster to incorporate the teachings of Harijan in order provide a method of conversation tracking and storing in order to keep the next utterance on topic. (Harijan, “[0042] A conversation may be a message exchange between two or more recipients. A conversation may also be a message exchange discussing a single conversation topic or multiple conversation topics, where topics change quickly. A conversation could be a single message, for instance, in the case of a user or recipient changing topics and/or initiating a new conversation. A user of a messaging application may be involved in multiple conversations with multiple recipients, but also may be involved in multiple conversations with a single recipient. An embodiment of the present invention facilitates tracking conversations and storing conversation data to prevent a message from being sent in the wrong conversation.”)
In regards to claim 11,
Shazeer teaches The data generation method according to claim 10,
Shuster teaches wherein the determining an initial input for the deep learning model comprises: obtaining, from an external memory bank, at least one pair of historical input data item and historical reply item whose similarity with the input data and the context information meets a fourth threshold;
(Shuster, Section 3.2.2, “Access long-term memory Again, MSC, ED, PC and BST are used to construct training data. In this case the target is the particular persona line used for a given context, which is calculated as the one with the highest word overlap [whose similarity with the input data and the context information ie a given context meets a fourth threshold] with the next utterance.”)
Shuster teaches and determining the initial input for the deep learning model based on the input data, the context information, and the at least one pair of historical input data item and historical reply item,
(Shuster, Section 3.1, “BB3 is a modular system but the modules are not independent components – this is achieved by training a single transformer model to execute the modules, with special control codes in the input context telling the model which module it is executing. The input context otherwise typically contains the dialogue history (sometimes truncated, depending on the module), with each speaker prefixed with their ID, either “Person 1:” or “Person 2:” in order to differentiate them [determining the initial input for the deep learning model based on the input data, the context information, and the at least one pair of historical input data item and historical reply item; see Figure 2 Memory Store wherein the historical reply item is generated by the DL model and stored in the Memory Store].”)
Shuster teaches wherein the external memory bank stores a first data group set related to the user,
(Shuser, Section 3.1, “Access long-term memory Given the full input context, and a store of (text-based) memories [the external memory bank stores a first data group set related to the user (see Figure 1 Memory Store Person 1’s Persona: I am a hair stylist and Person 2’s Persona: I love dogs wherein Examiner notes that the Memory Store is comprised of the user’s (Person 1) ‘memory’ and the system’s (Person 2) ‘memory’)], output a memory from the memory store [wherein the first functional component is an external memory bank], referred to as a recalled memory. Note: if the memory store is too large to fit in the context, we adopt some simple strategies. For the 3B parameter model, we use the Fusion-in-Decoder method (Izacard and Grave, 2021). For the OPT-based models for simplicity of implementation, we sample the memories to fit in the 2048 token context. We keep those with overlapping keywords to prior turns.”)
However, Shuster does not explicitly teach at least one pair of historical input data item and historical reply item…
and wherein each data group in the first data group set comprises at least a historical input data item and a historical reply item generated by the deep learning model for the historical input data item.
Harijan teaches at least one pair of historical input data item and historical reply item…
(Harijan, “[0050] An embodiment may include the context engine correlating a message data structure and a conversation data structure by way of incorporating data from the date/time field 312 and data from the date/time field 332. For instance, a comparison of data from the date/time field 332 for message data structure 330 to data from the date/time field 312 from conversation data structure 310 may identify similar times or a smaller time gap, which may indicate a higher likelihood that the message is relevant to the corresponding conversation. It may be beneficial to weight correlation data or weight a relevance score based on the time gap between the message and the latest entry in the conversation data structure [at least one pair of historical input data item and historical reply item; ie an identified relevance between a two entries from different users], because it is common for users to send messages to unintended recipients in older conversations and conversations where they had previously replied and the conversation was left open. A weighting inversely proportional to the time gap would help ensure timely replies had higher correlation values. In an embodiment, correlating a message data structure and a conversation data structure may simply be identifying the time gap between the message and the latest entry in the conversation data structure to calculate a relevance score based on how quick the response would be.”)
Harijan teaches a