DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 03/16/2026 have been fully considered but they are not persuasive.
Applicant argues, see Applicant’s Remarks pg. 9, that cited references are silent with respect to obtaining a state vector. Examiner would like to point out that previously-cited reference Bartl teaches of a HRED model. This HRED model is a vector encoding model that takes the conversation which is the natural language input and encodes it into the vector, which is interpreted as the state vector.
Specifically:
(Bartl, pg. 1120, Col. 1, paragraph 3, “In our approach, a context embedding, a vector encoding the meaning of a conversation [related to the natural language input] up to a certain time step t [obtaining the natural language input directed to the entity], encoded by the HRED model [neural network to obtain a state vector], serves as input to the decoder [stored in a database prior to obtaining the natural language input] component to generate a textual answer.” And pg. 1121, Col. 1, III., paragraph 1, “The second component, a retrieval-based approach using an Approximate Nearest Neighbor (ANN) model, is responsible for retrieving similar conversations from a database of embedding- and rawtext- tuples. Given the context of an unfinished conversation, suitable responses are considered to be contained in similar conversations, retrieved by the ANN model.”)
Therefore, the 35 USC 103 rejection is maintained.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 4-9, 11-16 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Bartl et al (A retrieval-based dialogue system utilizing utterance and context embeddings, "Bartl"), in view of Demiralp et al (US Published Patent Application No. 20150286928, "Demiralp") and Yan et al (Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System, "Yan").
In regard to claim 1, Bartl teaches obtaining, via a user interface with which a user is interacting, a natural language input directed to an entity; (Bartl, pg. 1120, Col. 1, Introductions “Text-only based Dialogue systems [a user interface], also called Conversational Agents, Chatbots or Chatterbots, have become very popular in the research community and for large companies.”, pg. 1121, Col. 1, B., paragraph 2, “Formally, a dialogue D, the input to such a model, can be represented as a sequence of utterances D = (U1, ...,UM), with Um being a sequence of word indices Um = (wm,1, ...,wm,Nm), each of them usually pointing to a vocabulary reference or directly to a word embedding.”)
updating the state vector using the conversation response of the entity. (Bartl, pg. 1121, Col. 1, A., paragraph 1, “Two gates, the reset and update gates rt and zt, operate directly on the hidden state, i.e., the hidden layer. Parametrized by W, U and b, while conditioned on the current input xt and previous result yt−1 [conversation response],…The update gate zt combines the function of the input and forget gate by controlling how much the new hidden state (ht) is defined by either the current input or the last hidden state,”)
in connection with obtaining the natural language input directed to the entity, obtaining, based on a first neural network and similarity distances between a pre-stored embedding of the user and other pre-stored embeddings of other entities other than the user and the entity, a state vector representing stored data that is (i) related to the natural language input and (ii) stored in a database prior to obtaining the natural language input; (Bartl, pg. 1120, Col. 1, paragraph 3, “In our approach, a context embedding, a vector encoding the meaning of a conversation [related to the natural language input] up to a certain time step t [obtaining the natural language input directed to the entity], encoded by the HRED model [neural network to obtain a state vector], serves as input to the decoder [stored in a database prior to obtaining the natural language input] component to generate a textual answer.” And pg. 1121, Col. 1, III., paragraph 1, “The second component, a retrieval-based approach using an Approximate Nearest Neighbor (ANN) model, is responsible for retrieving similar conversations from a database of embedding- and rawtext- tuples. Given the context of an unfinished conversation, suitable responses are considered to be contained in similar conversations, retrieved by the ANN model.”)
However, Bartl does not specifically teach one or more processors programmed with computer program instructions that, when executed, cause operations comprising:
inputting the natural language input and the state vector into a second neural network associated with the entity to generate a conversation response of the entity to the natural language input for presentation via the user interface;
Demirapl teaches one or more processors programmed with computer program instructions that, when executed, cause operations comprising: (Demirapl, paragraph 0017, “The causal modeling application 104 can be implemented as a software application, such as executable software instructions ( e.g., computer-executable instructions) that are executable by a processing system of the computing device 102 and stored on a computer-readable storage memory of the device.”)
Bartl and Demirapl are related to the same field of endeavor (i.e. analyzing natural language input). In view of the teachings of Demirapl, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Demirapl to Bartl before the effective filing date of the claimed invention in order to allow for a more accurate model. (Demirapl, paragraph 0019, “Further, dynamical causal modeling does not assume that random fluctuations are seri ally uncorrelated, thus allowing for more accurate and simultaneous modeling of influence variables, such as endogenous and exogenous variables.”)
However, Bartl and Demirapl do not explicitly teach inputting the natural language input and the state vector into a second neural network associated with the entity to generate a conversation response of the entity to the natural language input for presentation via the user interface;
Yan teaches inputting the natural language input and the state vector into a second neural network associated with the entity to generate a conversation response of the entity to the natural language input for presentation via the user interface; (Yan, pg. 59, Col. 2, paragraph 8, “Concretely, we build a CNN upon the output of Bi-LSTM [a conversation response of the entity]. For every window with the size of m in Bi-LSTM output vectors, i.e., (Ht)m = [ht, ht+1, · · · , ht+m1], where t is a certain position, the convolutional filter F = [F(0), . . . , F(m − 1)] will generate a vector sequence using the convolution operation “∗” between the two vectors.” And paragraph 9, “In practice, we also add a scalar bias b to the result of convolution. In this way, we obtain the vector oF is a vector, each dimension corresponding to each word in the sentence. [the natural language input]”)
Bartl, Demirapl and Yan are related to the same field of endeavor (i.e. analyzing natural language input). In view of the teachings of Yan, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Yan to Bartl and Demirapl before the effective filing date of the claimed invention in order to allow for more efficiency. (Yan, pg. 57, Col. 1, 2.2, paragraph 4, “reformulations in our scenario. For efficiency consideration, we leverage vector concatenation, which is simple yet”)
In regard to claim 2, Bartl, Demirapl and Yan teach the system of claim 1.
Bartl further teaches wherein obtaining the state vector comprises: in connection with obtaining the natural language input, querying the database based on the natural language input to obtain the stored data related to the natural language input; and (Bartl, pg. 1122, Col. 2, D., paragraph 1, “ Given a query context of an unfinished conversation, using the previously discussed LSH-Forest algorithm, one can retrieve a candidate set from a database of encoded conversations. Candidate answers will be scored based on the matching degree between the retrieved and the original context in terms of question-to-question similarity or answer relevance or other text-based features. However, the scoring functions introduced in this section will solely be based on vector comparison metrics, such as the cosine similarity, as text-based comparison is less rewarding and more difficult and tedious to implement. For the sake of clarity, the query context embedding is defined as cq, the textual candidates as r1, r2, ..., rk, the context embeddings of candidates as cr1, cr2 , ..., crk , and the utterance embeddings of candidates as hr1, hr2 , ..., hrk .”)
inputting the stored data into the first neural network to obtain the state vector. (Bartl, pg. 1120, Col. 1, paragraph 3, “In our approach, a context embedding, a vector encoding the meaning of a conversation up to a certain time step t [inputting the stored data], encoded by the HRED model [first neural network to obtain the state vector], serves as input to the decoder component to generate a textual answer.”)
In regard to claim 4, Bartl, Demirapl and Yan teach the system of claim 1.
Bartl further teaches based a response template being identified for the conversation response related to another entity other than the user and the entity, querying the database using the similarity distances between the pre-stored embedding of the user and other pre-stored embeddings of the other entities to obtain the stored data; and (Bartl, pg. 1122, Col. 2, D., paragraph 2, “The similarity of two conversations or the distance between a query context cq and a candidate context crk has, intuitively, a big impact on the retrieved answer, i.e, the more two questions are similar, the higher the probability that the answers are similar as well. If the cosine similarity has been chosen as the distance function D, the labels returned by the nearest neighbor search are already sorted by context-to-context distance.”)
inputting the stored data into the first neural network to obtain the state vector. (Bartl, pg. 1120, Col. 1, paragraph 3, “In our approach, a context embedding, a vector encoding the meaning of a conversation up to a certain time step t [inputting the stored data], encoded by the HRED model [first neural network to obtain the state vector], serves as input to the decoder component to generate a textual answer.”)
In regard to claim 5, Bartl, Demirapl and Yan teach the system of claim 1.
Demirapl further teaches storing, in the database, the updated state vector as part of a conversation memory node connected to one or more database nodes that include the stored data. (Demirapl, paragraph 0039, “However, the described techniques can be implemented with any vector space representation of textual content [updated state vector], making the approach applicable to multiple domains and businesses. In implementations, longitudinal bodies of text (e.g., chat, text stream, etc.) are converted to vector space representations using a text analytics engine. This format converts the longitudinal textual communications into multiple time series, where each time series represents the degree to which a particular topic, sentiment, or any other linguistic feature is present in the text. These time series constitute the values for the nodes xi described above, and the parameters of the model are estimated as described above [as part of a conversation memory node connected to one or more database nodes that include the stored data.].” and paragraph 0066, “The cloud-based data service 702 includes data servers 714 that may be implemented as any suitable memory, memory device, or electronic data storage for network-based data storage, and the data servers communicate data to computing devices via the network 712. The data servers 714 maintain a database 716 [in the database] of the input data 106, the text data 110, as well as the causal relationships model 306 that is generated by the causal modeling application 104. The cloudbased data service 702 can also include the sentiment analysis application 112 that generates the input data 106, and the database 716 may include the sentiment category vocabulary database 114 that is utilized by the sentiment analysis application 112 to generate the input data.”)
Bartrl and Demirapl are combinable for the same rationale as set forth above with respect to claim 1.
In regard to claim 6, Bartl, Demirapl and Yan teach the system of claim 1.
Bartl further teaches wherein the stored data stored in the database comprises inferred data, and the database comprises a probability weight associated with the inferred data, and wherein obtaining the state vector comprises: (Bartl, pg. 1122, Col. 1, C., paragraph 1, “Using the encoded corpus as a database [stored data stored in the database] of vectors, a Nearest Neighbor Search (NNS) algorithm can be used to find close embeddings in the whole set.” And Col. 2, D., paragraph 2, “The similarity of two conversations or the distance between a query context cq and a candidate context crk has, intuitively, a big impact on the retrieved answer, i.e, the more two questions are similar, the higher the probability that the answers are similar as well. If the cosine similarity has been chosen as the distance function D, the labels returned by the nearest neighbor search are already sorted by context-to-context distance.”)
based on the probability weight associated with the inferred data, obtaining the inferred data stored in the database for obtaining the state vector; and (Bartl, pg. 1122, Col. 2, D., paragraph 2, “The similarity of two conversations or the distance between a query context cq and a candidate context crk has, intuitively, a big impact on the retrieved answer, i.e, the more two questions are similar, the higher the probability that the answers are similar as well. If the cosine similarity has been chosen as the distance function D, the labels returned by the nearest neighbor search are already sorted by context-to-context distance.”)
inputting the inferred data into the first neural network to obtain the state vector. (Bartl, pg. pg. 1120, Col. 1, paragraph 3, “In our approach, a context embedding, a vector encoding the meaning of a conversation up to a certain time step t [inputting the stored data], encoded by the HRED model [first neural network to obtain the state vector], serves as input to the decoder component to generate a textual answer.”)
In regard to claim 7, Bartl, Demirapl and Yan teach the system of claim 1.
Bartl further teaches based on the probability weight associated with the inferred data failing to satisfy a probability threshold, querying an interlocutor for additional data related to the inferred data, (Bartl, pg. 1122, Col. 1, C., paragraph 3, “One of the main issues with the basic LSH algorithm [12] is that choosing the optimal number of hashing functions k and number of tables l requires one to know the most suitable value for r, the threshold separating similar and dissimilar points [failing to satisfy a probability threshold].” And Col. 2, D., paragraph 2, “The similarity of two conversations or the distance between a query context cq and a candidate context crk has, intuitively, a big impact on the retrieved answer, i.e, the more two questions are similar, the higher the probability that the answers are similar as well. If the cosine similarity has been chosen as the distance function D, the labels returned by the nearest neighbor search are already sorted by context-to-context distance.”)
wherein generating the conversation response comprises generating the conversation response based on the natural language input, the state vector, and the additional data obtained from the interlocutor. (Bartl, pg. 1122, Col. 2, D., paragraph 1, ”However, the scoring functions introduced in this section will solely be based on vector [the state vector] comparison metrics, such as the cosine similarity, as text-based comparison is less rewarding and more difficult and tedious to implement. For the sake of clarity, the query context embedding is defined as cq [the additional data obtained], the textual candidates as r1, r2, ..., rk, the context embeddings of candidates as cr1, cr2 , ..., crk , and the utterance embeddings of candidates [the natural language input] as hr1, hr2 , ..., hrk .”)pg. 1123, Col. 1, paragraph 1, “Therefore, in a pre-step, according to the previously described context relevance metric, the top n candidates are accumulated to represent the best general answer topic. In the next step, candidates are ranked based on their similarity to these n responses.”)
In regard to claim 8, Bartl teaches obtaining, via a user interface with which the user is interacting, a natural language input directed to an entity; (Bartl, pg. 1120, Col. 1, Introduction, paragraph 1, “Text-only based Dialogue systems [a user interface], also called Conversational Agents, Chatbots or Chatterbots, have become very popular in the research community and for large companies.”, pg. 1121, Col. 1B., paragraph 2, “Formally, a dialogue D, the input to such a model, can be represented as a sequence of utterances D = (U1, ...,UM), with Um being a sequence of word indices Um = (wm,1, ...,wm,Nm), each of them usually pointing to a vocabulary reference or directly to a word embedding.”)
in connection with obtaining the natural language input, performing a vector similarity search of a database based on prestored embeddings representing other entities, other than the user and the entity, to obtain one or more vectors corresponding to stored data related to the natural language input; (Bartl, pg. 1120, Col. 1, paragraph 3, “In our approach, a context embedding, a vector encoding the meaning of a conversation [related to the natural language input] up to a certain time step t [obtaining the natural language input directed to the entity], encoded by the HRED model [neural network to obtain a state vector], serves as input to the decoder [stored in a database prior to obtaining the natural language input] component to generate a textual answer.” And pg. 1121, Col. 1, III., paragraph 1, “The second component, a retrieval-based approach using an Approximate Nearest Neighbor (ANN) model, is responsible for retrieving similar conversations from a database of embedding- and rawtext- tuples. Given the context of an unfinished conversation, suitable responses are considered to be contained in similar conversations, retrieved by the ANN model.”)
causing, via the user interface, presentation of the response of the entity. (Bartl, pg. 1120, Col. 1, Introduction, paragraph 1, “Text-only based Dialogue systems [a user interface], also called Conversational Agents, Chatbots or Chatterbots, have become very popular in the research community and for large companies.”, pg. 1123, Col. 2, B., paragraph 3, “Each of the models, generative- or retrieval-based, receives the context of a conversation from an evaluation sample and has to generate or retrieve a suitable answer.”)
However, Bartl does not specifically teach executing, via one or more processors, operations comprising:
inputting the natural language input and the one or more vectors derived from the vector similarity search into a machine learning model to generate a response of the entity to the natural language input for presentation via the user interface; and
Demirapl teaches executing, via one or more processors, operations comprising: (Demirapl, paragraph 0017, “The causal modeling application 104 can be implemented as a software application, such as executable software instructions ( e.g., computer-executable instructions) that are executable by a processing system of the computing device 102 and stored on a computer-readable storage memory of the device.”)
Bartl and Demirapl are related to the same field of endeavor (i.e. analyzing natural language input). In view of the teachings of Demirapl, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Demirapl to Bartl before the effective filing date of the claimed invention in order to allow for a more accurate model. (Demirapl, paragraph 0019, “Further, dynamical causal modeling does not assume that random fluctuations are seri ally uncorrelated, thus allowing for more accurate and simultaneous modeling of influence variables, such as endogenous and exogenous variables.”)
However, Bartl and Demirapl do not explicitly teach based on the natural language input and the one or more vectors, generating, via a machine learning model, a response of the entity to the natural language input for presentation via the user interface; and
Yan teaches inputting the natural language input and the one or more vectors derived from the vector similarity search into a machine learning model to generate a response of the entity to the natural language input for presentation via the user interface; and (Yan, pg. 59, Col. 2, paragraph 8, “Concretely, we build a CNN upon the output of Bi-LSTM [a conversation response of the entity]. For every window with the size of m in Bi-LSTM output vectors, i.e., (Ht)m = [ht, ht+1, · · · , ht+m1], where t is a certain position, the convolutional filter F = [F(0), . . . , F(m − 1)] will generate a vector sequence using the convolution operation “∗” between the two vectors.” And paragraph 9, “In practice, we also add a scalar bias b to the result of convolution. In this way, we obtain the vector oF is a vector, each dimension corresponding to each word in the sentence. [the natural language input]”)
Bartl, Demirapl and Yan are related to the same field of endeavor (i.e. analyzing natural language input). In view of the teachings of Yan, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Yan to Bartl and Demirapl before the effective filing date of the claimed invention in order to allow for more efficiency. (Yan, pg. 57, Col. 1, 2.2, paragraph 4, “reformulations in our scenario. For efficiency consideration, we leverage vector concatenation, which is simple yet”)
In regard to claim 9, Bartl, Demirapl and Yan teach the method of claim 8.
Bartl further teaches obtaining the stored data based on the vector similarity search of the database; and (Bartl, pg. 1122, Col. 2, D., paragraph 2, “The similarity of two conversations or the distance between a query context cq and a candidate context crk has, intuitively, a big impact on the retrieved answer, i.e, the more two questions are similar, the higher the probability that the answers are similar as well.”)
based on the stored data, generating, via a neural network, the one or more vectors corresponding to the stored data. (Bartl, pg. 1122, Col. 2, D., paragraph 1, “Candidate answers will be scored based on the matching degree between the retrieved and the original context in terms of question-to-question similarity or answer relevance or other text-based features. However, the scoring functions introduced in this section will solely be based on vector comparison metrics, such as the cosine similarity, as text-based comparison is less rewarding and more difficult and tedious to implement.”)
In regard to claim 11, Bartl, Demirapl and Yan teach the method of claim 8.
Bartl further teaches based a response template being identified for the response and related to another entity other than the user and the entity, querying the database using similar distances between a pre-stored embedding of the user and the prestored embedding representing the other entities to obtain the stored data; and (Bartl, pg. 1122, Col. 2, D., paragraph 2, “The similarity of two conversations or the distance between a query context cq and a candidate context crk has, intuitively, a big impact on the retrieved answer, i.e, the more two questions are similar, the higher the probability that the answers are similar as well. If the cosine similarity has been chosen as the distance function D, the labels returned by the nearest neighbor search are already sorted by context-to-context distance.”)
based on the stored data, generating, via a neural network, the one or more vectors corresponding to the stored data. (Bartl, pg. 1120, Col. 1, paragraph 3, “In our approach, a context embedding, a vector encoding the meaning of a conversation up to a certain time step t [inputting the stored data], encoded by the HRED model [first neural network to obtain the state vector], serves as input to the decoder component to generate a textual answer.”)
In regard to claim 12, Bartl, Demirapl and Yan teach the method of claim 8.
Bartl further teaches wherein the stored data stored in the database comprises inferred data and a probability weight associated with the inferred data. (Bartl, pg. 1122, Col. 2, D., paragraph 2, “The similarity of two conversations or the distance between a query context cq and a candidate context crk has, intuitively, a big impact on the retrieved answer, i.e, the more two questions are similar, the higher the probability that the answers are similar [a probability weight associated with the inferred data] as well. If the cosine similarity has been chosen as the distance function D, the labels returned by the nearest neighbor search are already sorted by context-to-context distance.”)
In regard to claim 13, Bartl, Demirapl and Yan teach the method of claim 12.
Bartl further teaches based on the probability weight associated with the inferred data failing to satisfy a probability threshold, querying an interlocutor for additional data related to the inferred data, (Bartl, pg. 1122, pg. 1122, Col. 1, C., paragraph 3, “One of the main issues with the basic LSH algorithm [12] is that choosing the optimal number of hashing functions k and number of tables l requires one to know the most suitable value for r, the threshold separating similar and dissimilar points [failing to satisfy a probability threshold].” Col. 2, D., paragraph 2, “The similarity of two conversations or the distance between a query context cq and a candidate context crk has, intuitively, a big impact on the retrieved answer, i.e, the more two questions are similar, the higher the probability that the answers are similar as well. If the cosine similarity has been chosen as the distance function D, the labels returned by the nearest neighbor search are already sorted by context-to-context distance.”)
wherein generating the response comprises generating the response based on the natural language input, the one or more vectors corresponding to the stored data, and the additional data obtained from the interlocutor. (Bartl, pg. 1123, Col. 1, paragraph 1, “Therefore, in a pre-step, according to the previously described context relevance metric, the top n candidates are accumulated to represent the best general answer topic. In the next step, candidates are ranked based on their similarity to these n responses.”)
In regard to claim 14, Bartl, Demirapl and Yan teach the method of claim 8.
Bartl further teaches updating the one or more vectors based on the response of the entity. (Bartl, pg. 1121, Col. 1, A., paragraph 1, “Two gates, the reset and update gates rt and zt, operate directly on the hidden state, i.e., the hidden layer. Parametrized by W, U and b, while conditioned on the current input xt and previous result yt−1 [conversation response],…The update gate zt combines the function of the input and forget gate by controlling how much the new hidden state (ht) is defined by either the current input or the last hidden state,”)
In regard to claim 15, Bartl, Demirapl and Yan teach the method of claim 14.
Bartl further teaches storing, in the database, the one or more updated vector as part of one or more conversation memory nodes. (Bartl, pg. 1121, Col. 1, III., paragraph 1, “The second component,
a retrieval-based approach using an Approximate Nearest Neighbor (ANN) model, is responsible for retrieving similar conversations from a database of embedding- and rawtext- tuples.” And pg. 1124, Col. 2, V., paragraph 3, “by representing context embeddings as states (tree nodes) and utterance embeddings as actions (connections between nodes).”)
In regard to claim 16, Bartl teaches obtaining, via a user interface, a natural language input directed to an entity; (Bartl, pg. 1120, Col. 1, Introduction, paragraph 1, “Text-only based Dialogue systems [a user interface], also called Conversational Agents, Chatbots or Chatterbots, have become very popular in the research community and for large companies.”, pg. 1121, Col. 1, B., paragraph 2, “Formally, a dialogue D, the input to such a model, can be represented as a sequence of utterances D = (U1, ...,UM), with Um being a sequence of word indices Um = (wm,1, ...,wm,Nm), each of them usually pointing to a vocabulary reference or directly to a word embedding.”)
in connection with obtaining the natural language input, querying a database based on prestored probabilistic metadata respectively associated with the inferred data in the database to obtain inferred data related to the natural language input; (Bartl, pg. 1120, Col. 1, paragraph 3, “In our approach, a context embedding, a vector encoding the meaning of a conversation [related to the natural language input] up to a certain time step t [obtaining the natural language input directed to the entity], encoded by the HRED model [neural network to obtain a state vector], serves as input to the decoder [stored in a database prior to obtaining the natural language input] component to generate a textual answer.” And pg. 1121, Col. 1, III., paragraph 1, “The second component, a retrieval-based approach using an Approximate Nearest Neighbor (ANN) model, is responsible for retrieving similar conversations from a database of embedding- and rawtext- tuples. Given the context of an unfinished conversation, suitable responses are considered to be contained in similar conversations [querying a database based on prestored probabilistic metadata], retrieved by the ANN model.”)
causing, via the user interface, presentation of the response of the entity. (Bartl, pg. 1123, Col. 2, B., paragraph 3, “Each of the models, generative- or retrieval-based, receives the context of a conversation from an evaluation sample and has to generate or retrieve a suitable answer.”) and (Bartl, pg. 1120, Col. 1, Introduction, paragraph 1, “Text-only based Dialogue systems [a user interface], also called Conversational Agents, Chatbots or Chatterbots, have become very popular in the research community and for large companies.”, pg. 1123, Col. 2, B., paragraph 3, “Each of the models, generative- or retrieval-based, receives the context of a conversation from an evaluation sample and has to generate or retrieve a suitable answer.”)
However, Bartl does not specifically teach One or more non-transitory computer-readable media comprising computer program instructions that, when executed by one or more processors, cause operations comprising:
inputting the natural language input and the stored inferred data associated with the respective probabilistic metadata in the database, into a machine learning model to generate a response of the entity to the natural language input for presentation via the user interface; and
Demirapl teaches One or more non-transitory computer-readable media comprising computer program instructions that, when executed by one or more processors, cause operations comprising: (Demirapl, paragraph 0017, “The causal modeling application 104 can be implemented as a software application, such as executable software instructions ( e.g., computer-executable instructions) that are executable by a processing system of the computing device 102 and stored on a computer-readable storage memory of the device.”)
Bartl and Demirapl are related to the same field of endeavor (i.e. analyzing natural language input). In view of the teachings of Demirapl, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Demirapl to Bartl before the effective filing date of the claimed invention in order to allow for a more accurate model. (Demirapl, paragraph 0019, “Further, dynamical causal modeling does not assume that random fluctuations are seri ally uncorrelated, thus allowing for more accurate and simultaneous modeling of influence variables, such as endogenous and exogenous variables.”)
However, Bartl and Demirapl do not explicitly teach inputting the natural language input and the stored inferred data associated with the respective probabilistic metadata in the database, into a machine learning model to generate a response of the entity to the natural language input for presentation via the user interface; and
Yan teaches inputting the natural language input and the stored inferred data associated with the respective probabilistic metadata in the database, into a machine learning model to generate a response of the entity to the natural language input for presentation via the user interface; and (Yan, pg. 59, Col. 2, paragraph 8, “Concretely, we build a CNN upon the output of Bi-LSTM [a conversation response of the entity]. For every window with the size of m in Bi-LSTM output vectors, i.e., (Ht)m = [ht, ht+1, · · · , ht+m1], where t is a certain position, the convolutional filter F = [F(0), . . . , F(m − 1)] will generate a vector sequence using the convolution operation “∗” between the two vectors.” And paragraph 9, “In practice, we also add a scalar bias b to the result of convolution. In this way, we obtain the vector oF is a vector, each dimension corresponding to each word in the sentence. [the natural language input]” and pg. 61, Col. 1, 6.1, paragraph 1, “We conducted data filtering and cleaning procedures by removing extremely short replies and those of low linguistic quality such as meaningless babblings according to the evaluation framework put forward in [40, 42], so as to maintain a meaningful, high-quality conversation record. In total, the database contains ∼10 million ⟨posting, reply⟩ pairs. Some statistics are summarized in Table 4. [the stored inferred data associated with the respective probabilistic metadata in the database]”)
Bartl, Demirapl and Yan are related to the same field of endeavor (i.e. analyzing natural language input). In view of the teachings of Yan, it would have been obvious for a person with ordinary skill in the art to apply the teachings of Yan to Bartl and Demirapl before the effective filing date of the claimed invention in order to allow for more efficiency. (Yan, pg. 57, Col. 1, 2.2, paragraph 4, “reformulations in our scenario. For efficiency consideration, we leverage vector concatenation, which is simple yet”)
In regard to claim 18, Bartl, Demirapl and Yan teach the non-transitory computer-readable media of claim 16.
Bartl further teaches wherein obtaining the stored inferred data comprises, based a response template being identified for the response related to another entity other than the user and the entity, querying the database using an embedding of the user to obtain the stored inferred data. (Bartl, pg. 1122, Col. 2, D., paragraph 1, “Given a query context of an unfinished conversation, using the previously discussed LSH-Forest algorithm, one can retrieve a candidate set from a database of encoded conversations. Candidate answers will be scored based on the matching degree between the retrieved and the original context in terms of question-to-question similarity or answer relevance or other text-based features. However, the scoring functions introduced in this section will solely be based on vector comparison metrics, such as the cosine similarity, as text-based comparison is less rewarding and more difficult and tedious to implement. However, the scoring functions introduced in this section will solely be based on vector comparison metrics, such as the cosine similarity, as text-based comparison is less rewarding and more difficult and tedious to implement. For the sake of clarity, the query context embedding is defined as cq, the textual candidates as r1, r2, ..., rk, the context embeddings of candidates as cr1, cr2 , ..., crk , and the utterance embeddings of candidates as hr1, hr2 , ..., hrk .”)
In regard to claim 19, Bartl, Demirapl and Yan teach the non-transitory computer-readable media of claim 16.
Bartl further teaches based on a probability weight associated with the stored inferred data failing to satisfy a probability threshold, querying an interlocutor for additional data related to the inferred data, (Bartl, pg. 1122, Col. 1, C., paragraph 3, “One of the main issues with the basic LSH algorithm [12] is that choosing the optimal number of hashing functions k and number of tables l requires one to know the most suitable value for r, the threshold separating similar and dissimilar points [failing to satisfy a probability threshold].” And Col. 2, D., paragraph 2, “The similarity of two conversations or the distance between a query context cq and a candidate context crk has, intuitively, a big impact on the retrieved answer, i.e, the more two questions are similar, the higher the probability that the answers are similar as well. If the cosine similarity has been chosen as the distance function D, the labels returned by the nearest neighbor search are already sorted by context-to-context distance.”)
wherein generating the conversation response comprises generating the conversation response based on the natural language input, the stored inferred data, and the additional data obtained from the interlocutor. (Bartl, pg. 1122, Col. 2, D., paragraph 1, ”However, the scoring functions introduced in this section will solely be based on vector [the state vector] comparison metrics, such as the cosine similarity, as text-based comparison is less rewarding and more difficult and tedious to implement. For the sake of clarity, the query context embedding is defined as cq [the additional data obtained], the textual candidates as r1, r2, ..., rk, the context embeddings of candidates as cr1, cr2 , ..., crk , and the utterance embeddings of candidates [the natural language input] as hr1, hr2 , ..., hrk .”)pg. 1123, Col. 1, paragraph 1, “Therefore, in a pre-step, according to the previously described context relevance metric, the top n candidates are accumulated to represent the best general answer topic. In the next step, candidates are ranked based on their similarity to these n responses.”)
In regard to claim 20, Bartl, Demirapl and Yan teach the non-transitory computer-readable media of claim 16.
Bartl further teaches based on the response of the entity, updating one or more vectors corresponding to the inferred stored data; and (Bartl, pg. 1121, Col. 1, A., paragraph 1, “““Two gates, the reset and update gates rt and zt, operate directly on the hidden state, i.e., the hidden layer. Parametrized by W, U and b, while conditioned on the current input xt and previous result yt−1 [conversation response],…The update gate zt combines the function of the input and forget gate by controlling how much the new hidden state (ht) is defined by either the current input or the last hidden state,”)
storing, in the database, the one or more updated vectors as part of one or more conversation memory nodes. (Bartl, pg. 1121, Col. 1, III., paragraph 1, “The second component, a retrieval-based approach using an Approximate Nearest Neighbor (ANN) model, is responsible for retrieving similar conversations from a database of embedding- and rawtext- tuples.” pg. 1124, Col. 2, V., paragraph 3, “by representing context embeddings as states (tree nodes) and utterance embeddings as actions (connections between nodes).”)
Claims 3, 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Bartl, in view of Demiralp and Yan and in further view of d'Avila et al (Abductive reasoning in neural-symbolic systems, "d'Avila").
In regard to claim 3 and analogous claims 10 and 17, Bartl, Demirapl and Yan teach the system of claim 1.
However, Bartl, Demirapl and Yan do not explicitly teach obtaining the state vector comprises, in connection with obtaining the natural language input, using the first neural network, comprising a graph embedding network, to obtain a state graph embedding from symbolic graph data of a graph and sub-symbolic embeddings of the graph, the state graph embedding representing the stored data related to the natural language input; and
inputting the natural language input and the state vector into the second neural network comprises inputting the natural language input and the state graph embedding into the second neural network to generate the conversation response of the entity to the natural language input for presentation via the user interface.
d’avila teaches obtaining the state vector comprises, in connection with obtaining the natural language input, using the first neural network, comprising a graph embedding network, to obtain a state graph embedding from symbolic graph data of a graph and sub-symbolic embeddings of the graph, the state graph embedding representing the stored data related to the natural language input; and (d’avlia, pg. 40, Col. 1, 2., paragraph 1, “In this section, we define neural networks and present the basic concepts of neural-symbolic learning systems used throughout the paper.” And paragraph 2, “An artificial neural network is a directed graph with the following structure: a node (or neuron) i in the graph is characterised, at time t, by its input vector Ii(t) [in connection with obtaining the natural language input, using the first neural network], its input potential Ui(t), its activation state Ai(t), and its output Oi(t). The nodes of the network are interconnected via a set of directed and weighted edges (or connections) such that if there is a connection from node i to node j then Wji 2 R denotes the weight of this connection. The input potential Ui(t) of neuron i at time t is obtained by computing a weighted sum for neuron i such that UiðtÞ ¼ PjWijIiðtÞ [the state graph embedding representing the stored data related to the natural language input]. The activation state Ai(t) of neuron i at time t—a bounded real or integer number—is then given by the neuron’s activation function hi such that AiðtÞ ¼ hiðUiðtÞÞ:”)
inputting the natural language input and the state vector into the second neural network comprises inputting the natural language input and the state graph embedding into the second neural network to generate the conversation response of the entity to the natural language input for presentation via the user interface. (d’avlia, pg. 40, Col. 1, 2., paragraph 1, “In this section, we define neural networks and present the basic concepts of neural-symbolic learning systems used throughout the paper.” And paragraph 2, “An artificial neural network is a directed graph with the following structure: a node (or neuron) i in the graph is characterized [into the second neural network comprises inputting the natural language input], at time t, by its input vector Ii(t), its input potential Ui(t), its activation state Ai(t), and its output Oi(t).” and paragraph 3, “The nodes of a neural network can be organised in layers. A n-layer feedforward network is an acyclic graph. It consists of a sequence of layers and connections between successive layers, containing one input layer, n–2 hidden layers, and one output layer, where n ‡ 2. When n = 3, we say that the network is a single hidden layer network. When each node occurring in the ith layer is connected to each node occurring in the (i + 1)-th layer, we say that the network is fully connected.” And pg. 38, Col. 1, paragraph 3, “In the same setting, deduction is the network computation of output values as a response [to generate the conversation response] to input values (stimuli) given a particular set of weights.”))
Bartl, Demirapl, Yan and d’avila are related to the same field of endeavor (i.e. analyzing natural language input). In view of the teachings of d’avila, it would have been obvious for a person with ordinary skill in the art to apply the teachings of d’avila to Bartl, Demirapl and Yan before the effective filing date of the claimed invention in order provide a faster development of responses in AI. (d’avila, pg. 48, Col. 1, paragraph 2, “we may be getting closer to a unifying theory, or at least promote a faster and principled development of the field.”)
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Bartl, in view of d'avlia.
In regard to claim 21, Bartl teaches obtaining, via a user interface, a natural language input directed to an entity; (Bartl, pg. 1120, Col. 1, Introductions “Text-only based Dialogue systems [a user interface], also called Conversational Agents, Chatbots or Chatterbots, have become very popular in the research community and for large companies.”, pg. 1121, Col. 1, B., paragraph 2, “Formally, a dialogue D, the input to such a model, can be represented as a sequence of utterances D = (U1, ...,UM), with Um being a sequence of word indices Um = (wm,1, ...,wm,Nm), each of them usually pointing to a vocabulary reference or directly to a word embedding.”)
causing, via the user interface, presentation of the response of the entity. (Bartl, pg. 1123, Col. 2, B., paragraph 3, “Each of the models, generative- or retrieval-based, receives the context of a conversation from an evaluation sample and has to generate or retrieve a suitable answer.”) and (Bartl, pg. 1120, Col. 1, Introduction, paragraph 1, “Text-only based Dialogue systems [a user interface], also called Conversational Agents, Chatbots or Chatterbots, have become very popular in the research community and for large companies.”, pg. 1123, Col. 2, B., paragraph 3, “Each of the models, generative- or retrieval-based, receives the context of a conversation from an evaluation sample and has to generate or retrieve a suitable answer.”)
However, Bartl does not explicitly teach in connection with obtaining the natural language input, using a first neural network comprising a graph embedding network to obtain a state graph embedding from non-embedding symbolic graph data of a graph and sub-symbolic embeddings of the graph, the state graph embedding representing stored data related to the natural language input;
inputting the natural language input and the state graph embedding into a second neural network to generate a response of the entity to the natural language input for presentation via the user interface; and
d,avila teaches in connection with obtaining the natural language input, using a first neural network comprising a graph embedding network to obtain a state graph embedding from non-embedding symbolic graph data of a graph and sub-symbolic embeddings of the graph, the state graph embedding representing stored data related to the natural language input; (d’avlia, pg. 40, Col. 1, 2., paragraph 1, “In this section, we define neural networks and present the basic concepts of neural-symbolic learning systems used throughout the paper.” And paragraph 2, “An artificial neural network is a directed graph with the following structure: a node (or neuron) i in the graph is characterised, at time t, by its input vector Ii(t) [in connection with obtaining the natural language input, using the first neural network], its input potential Ui(t), its activation state Ai(t), and its output Oi(t). The nodes of the network are interconnected via a set of directed and weighted edges (or connections) such that if there is a connection from node i to node j then Wji 2 R denotes the weight of this connection. The input potential Ui(t) of neuron i at time t is obtained by computing a weighted sum for neuron i such that UiðtÞ ¼ PjWijIiðtÞ [the state graph embedding representing the stored data related to the natural language input]. The activation state Ai(t) of neuron i at time t—a bounded real or integer number—is then given by the neuron’s activation function hi such that AiðtÞ ¼ hiðUiðtÞÞ:”)
inputting the natural language input and the state graph embedding into a second neural network to generate a response of the entity to the natural language input for presentation via the user interface; and (d’avlia, pg. 40, Col. 2, 2., paragraph 1, “In this section, we define neural networks and present the basic concepts of neural-symbolic learning systems used throughout the paper.” And paragraph 2, “An artificial neural network is a directed graph with the following structure: a node (or neuron) i in the graph is characterised, at time t, by its input vector Ii(t), its input potential Ui(t), its activation state Ai(t), and its output Oi(t).” and paragraph 3, “The nodes of a neural network can be organised in layers. A n-layer feedforward network is an acyclic graph. It consists of a sequence of layers and connections between successive layers, containing one input layer, n–2 hidden layers, and one output layer, where n ‡ 2. When n = 3, we say that the network is a single hidden layer network. When each node occurring in the ith layer is connected to each node occurring in the (i + 1)-th layer, we say that the network is fully connected.” And pg. 38, Col. 1, paragraph 3, “In the same setting, deduction is the network computation of output values as a response [to generate a response] to input values (stimuli) given a particular set of weights.”)
Bartl and d’avila are related to the same field of endeavor (i.e. analyzing natural language input). In view of the teachings of d’avila, it would have been obvious for a person with ordinary skill in the art to apply the teachings of d’avila to Bartl before the effective filing date of the claimed invention in order to respond well to examples not provided in training. (d’avila, pg. 40, Col. 2, paragraph 1, “Typically, a subset of the set of examples available for training is left out of the learning process so that it can be used for checking the network’s generalization ability, i.e. its ability to respond well to examples not seen during training.”)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SKYLAR K VANWORMER whose telephone number is (703)756-1571. The examiner can normally be reached M-F 6:00am to 3:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/S.K.V./Examiner, Art Unit 2146 /USMAAN SAEED/Supervisory Patent Examiner, Art Unit 2146
Prosecution Insights