DETAILED ACTION
Status of Claims
This Office action is responsive to communications filed on 2025-09-19 and 2025-10-14. Claim(s) 4, 13, and 20 was/were cancelled. Claim(s) 1-3, 5-12, and 14-19 is/are pending and are examined herein.
Claim(s) 1-3, 5-12, and 14-19 is/are rejected under 35 USC 103.
Notice of Pre-AIA or AIA Status
The present application, filed on or after 2013-03-16, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2025-09-19 has been entered.
Response to Arguments
Regarding the objections for informalities, the applicant’s amendments resolve the concerns raised in the previous Office action.
Regarding the rejections under 35 USC 103, the applicant’s arguments have been fully considered but they are unpersuasive.
Regarding the previously filed claims, the applicant “traverses these rejections” but provides no rationale for this traversal. This unsubstantiated traversal does not comply with 37 CFR 1.111(c) because it does not clearly point out the patentable novelty which the applicant thinks the previously filed claims present in view of the state of the art disclosed by the references cited or the objections made.
The applicant asserts that the references made of record do not disclose updating the semantic vector during training [remarks, page 11]. However, as best understood by the examiner, the rationale appears to be the mere observation that the reference Agarwal does not use the exact phrase “semantic vector of the candidate intent” [remarks, page 11]. The examiner respectfully reminds the applicant that it is not necessary for prior art references to use exactly the same language as the claim so long as the substance of the claim is disclosed. As noted in the previous rejection, both Galassi and Agarwal disclose assigning vectors to words, where the vectors represent the lexical semantics of word and thus map to the “semantic vector of the candidate intent” of the claim. In view of this, the disclosure in Agarwal that “the corresponding vectors are continually updated during training” [Agarwal, 0010] is in fact a disclosure of the feature recited by the claim. Moreover, as indicated in the conclusion of the previous Office action, the same feature is also disclosed in numerous other references made of record (cf. Hashimoto, Min, Jamali, and Glass).
The applicant asserts that “Cho fails to disclose the claimed training process of the neural network model as defined in Amended Claim 1” [remarks, page 11] but provides no rationale for this assertion: the applicant merely quotes the entire “adjusting” limitation of the claim without indicating what portions, if any, are not disclosed in Cho. This assertion does not comply with 37 CFR 1.111(c) because it does not clearly point out the patentable novelty which the applicant thinks the claim present in view of the state of the art disclosed by the references cited or the objections made.
The examiner notes that, while the amendments appear to involve a slight change in language (namely, from “updating the semantic vector of the candidate intent of the neural network model with the training of the neural network model” to “so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model”), the substance of the amendments is a mere incorporation of the limitations of the cancelled dependent claims (4, 13, and 20) into the respective independent claims. The claims are therefore rejected by the same rationale as described previously, with only minor changes in view of the applicant’s amendments.
Claim Rejections - 35 USC 103
The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 USC 102(b)(2)(C) for any potential 35 USC 102(a)(2) prior art against the later invention.
Claim(s) 1-3, 5-12, and 14-19 is/are rejected under 35 USC 103 as being unpatentable over Rui YAN et al. (US20200311199A1, published 2020-10-01; hereafter “Yan”) in view of Andrea GALASSI et al. (Attention in Natural Language Processing, published 2020-05-28; hereafter “Galassi”), Kwantae CHO et al. (A Performance Comparison of Loss Functions, published 2019; hereafter “Cho”), and Puneet AGARWAL et al. (US20190080225A1, published 2019-03-14; hereafter “Agarwal”).
Claim 1
Yan discloses:
A computer-implemented method for natural language processing, comprising: ([Yan, 0022]: Yan discloses a method on a “server computing device” which receives “natural language input” [Yan, 0022]. The computing device on which the methods disclosed therein are implemented map to the “computer” of the claim.)
acquiring training data comprising a plurality of training texts and first annotation intents of the plurality of training texts; ([Yan, 0065]: Yan discloses a “training dataset 330” which contains “training utterances 332” and “[e]ach training utterance 332 may further include a training utterance intention selected from among the respective plurality of predefined intentions 42” [Yan, 0065]. The training utterances map to the “plurality of training texts” of the claim, and the training utterance intentions of the training utterances are the “first annotation intents” of the claim.)
constructing a neural network model comprising a feature extraction layer and a first recognition layer, ([Yan, 0023, 0029, 0034, and figure 4]: In the system disclosed by Yan, “natural language input 20, in the form of a text input” is sent “into a preprocessor 22. The preprocessor 22 may be configured to generate a tokenized utterance 24 based on the natural language input 20” [Yan, 0023]. Then an “intention detector 40… receive[s] the tokenized utterance 24 from the preprocessor 22” [Yan, 0029] and ultimately “select[s] a predefined intention 42 that estimates an intention of the user who enters the natural language input 20” [Yan, 0034]. The intention detector is a neural network having several components performing computations in a sequence [Yan, figure 4 elements 210, 212, 214, 216, 218]. The intention detector maps to the “neural network” of the claim, its components up through the second bidirectional LSTM network [Yan, figure 4 elements 210, 212, 214] map to the “feature extraction layer” of the claim, and the remainder [Yan, figure 4 elements 216, 218] maps to the “first recognition layer” of the claim.)
the first recognition layer being configured to output, according to [a semantic vector of] a candidate intent and a first semantic vector of each segmented word in a training text outputted by the feature extraction layer, a first intent result of the training text ([Yan, 0033-0034, 0052, and figure 4]: Yan explains that the tokenized utterance generated by the preprocessor “may include one or more words and one or more metadata tokens respectively associated with the one or more words” [Yan, 0052]. The words in the tokenized utterance map to the “segmented words” of the claim, and the “output of the second bidirectional LSTM network 214” [Yan, 0033] maps to the “first semantic vector of each segmented word” of the claim (since the components of the intention detector up through the second bidirectional LSTM network [Yan, figure 4 elements 210, 212, 214] are mapped above to the “feature extraction layer” of the claim). The output of the second bidirectional LSTM network enters the remainder of the network, which, as noted above, “select[s] a predefined intention 42 that estimates an intention of the user who enters the natural language input 20” [Yan, 0034]. Any of the possible predefined intentions which may be selected by the intention detector map to the “candidate intent” of the claim, and the predefined intention that is actually selected by the intention detector maps to the “first intent result” of the claim.)
training the neural network model according to word segmentation results of the plurality of training texts and the first annotation intents of the plurality of training texts to obtain an intent recognition model; ([Yan, 0052, 0065, 0067]: Yan discloses “train[ing] the intention detector 40 using the plurality of training utterances 332” [Yan, 0067] where, as noted above, “[e]ach training utterance 332 may further include a training utterance intention selected from among the respective plurality of predefined intentions” [Yan, 0065]. It also discloses, as noted above, a preprocessing step of “generating a tokenized utterance based on the natural language input. The tokenized utterance may include one or more words and one or more metadata tokens respectively associated with the one or more words” [Yan, 0052]. The training utterances map to the “plurality of training texts” of the claim as noted above, and the tokenized training utterances map to the “word segmentation results of the plurality of training texts” of the claim. The training utterance intentions that are included in the training utterances map to the “first annotation intents” as noted above, and intention detector is the “intent recognition model” of the claim.)
wherein training the neural network model according to the word segmentation results of the plurality of training texts and the first annotation intents of the plurality of training texts to obtain the intent recognition model comprises: inputting the word segmentation results of the plurality of training texts to the neural network model to obtain the first intent result outputted by the neural network model for each training text; ([Yan, 0034, 0052, 0067]: As noted above, Yan discloses training utterances mapping to the “plurality of training texts” of the claim and tokenized training utterances mapping to the “word segmentation results of the plurality of training texts” of the claim. The output of the intention detector on a training utterance maps to the “first intent result” of the claim.)
Yan might not distinctly disclose:
[the first recognition layer being configured to output, according to] a semantic vector of a candidate intent and [a first semantic vector of each segmented word in a training text outputted by the feature extraction layer, a first intent result of the training text] and a score between each segmented word in the training text and the candidate intent, wherein the semantic vector of the candidate intent is configured to represent semantics of the candidate intent;
calculating a loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts;
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated loss function value, until the neural network model converges, to obtain the intent recognition model, so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model, to obtain the intent recognition model.
Galassi is in the field of natural language processing. It discusses the use of attention in a variety of natural processing tasks [Galassi, table 1], including a number of classification tasks (of which the intent detection task of Yan as disclosed above is an example). The input to the attention mechanism consists of keys (i.e., the columns of the matrix K), values (i.e., the columns of the matrix V), and a query text [Galassi, figure 4]. In the combination, the word embedding vectors which are extracted from each word in the tokenized utterance [Yan, 0029] are used as the keys of Galassi, the output of the second bidirectional LSTM network [Yan, 0033] are used as the values of Galassi, and the “candidate intent” as mapped above is used as the query text of Galassi. In other words, an attention mechanism of Galassi is inserted into the intention detector of Yan after the second bidirectional LSTM layer as a part of the “first recognition layer” of the claim. Then Yan in view of Galassi discloses:
[the first recognition layer being configured to output, according to] a semantic vector of [a candidate intent and a first semantic vector of each segmented word in a training text outputted by the feature extraction layer, a first intent result of the training text] and a score between each segmented word in the training text and the candidate intent, wherein the semantic vector of the candidate intent is configured to represent semantics of the candidate intent; ([Galassi, figure 4]: Galassi discloses representing the query text (i.e., the “candidate intent” of the claim) as a vector q in R^{n_q} [Galassi, figure 4], so q in R^{n_q} is the “semantic vector of [the] candidate intent” of the claim. This vector “represent[s] semantics of the candidate intent” as recited by the claim. The attention mechanism computes “[a]ttention weights” [Galassi, figure 4] which “represent the relevance of each element to the given task, with respect to q and K” [Galassi, section II.B paragraph beginning “Such weights”]. In the combination as noted above, the word embedding vectors of the words in the tokenized utterance of Yan [Yan, 0029] are used as key vectors of Galassi, so each attention weight map to a “score between each segmented word in the training text and the candidate intent” of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to incorporate an attention mechanism as in Galassi into the intention detector of Yan because attention “offer[s] a performance gain” and “can also be used as a tool for interpreting the behaviour of neural architectures” [Galassi, section I paragraph beginning “Besides offering a performance gain”] and consequently “has become an increasingly common ingredient of neural architectures for NLP” [Galassi, section I paragraph beginning “For all these reasons”], so the combination would be more effective overall.
While Yan in view of Galassi discloses a training an intention detector implemented as a neural network, and training a neural network involves iteratively adjusting model parameters until model performance converges, model performance being measured by means of a loss function which compares output generated by the model (the “first intent results” of the claim) against true output in labeled training data (the “first annotation intents” of the claim), it may nonetheless be argued that Yan in view of Galassi does not distinctly disclose:
calculating a loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts;
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated loss function value, until the neural network model converges, to obtain the intent recognition model, so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model, to obtain the intent recognition model.
Cho is in the field of machine learning. It discusses loss functions for training neural networks for classification [Cho, abstract]. The intention detector of Yan is a neural network performing a classification task. Moreover, Yan in view of Galassi and Cho discloses:
calculating a loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts; ([Cho, abstract]: Cho discusses loss functions which measure “prediction error” (indicating, for example, that “[i]f the prediction deviates too far from real data, a loss function would generate a very large value”) [Cho, abstract]. The output of the intention detector of Yan corresponds to the prediction of Cho and to the “first intent results” of the claim, and the true training utterance intention of Yan corresponds to the real data of Cho and the “first annotation intents” of the claim. The prediction error, i.e., the value output by a loss function as in Cho, maps to the “loss function value” of the claim.)
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated loss function value, until the neural network model converges, [so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model,] to obtain the intent recognition model. ([Cho, abstract]: Cho discloses that “[p]rogressively, with the help of some optimization function, the loss function lowers the prediction error by providing the network architecture with information that can control the weights of the network architecture” [Cho, abstract]. As noted above, the prediction error is the “calculated loss function value” of the claim. The intention detector of Yan in view of Galassi is the “neural network model” of the claim, and the weights of the intention detector are the “parameters of the neural network model and the semantic vector of the candidate intent” of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to use loss functions as described in Cho to train the intention detector of Yan in view of Galassi because “loss functions [play] an important role in training the network architecture” [Cho, abstract].
Galassi discloses that the query vector (i.e., the “semantic vector of the candidate intent” of the claim) is produced by an annotation function denoted “qaf” [Galassi, table II] and discusses architectures where “the layers in charge of annotation are trained together with the attention model” [Galassi, section IV.A first paragraph]. While this would have rendered obvious to a person of ordinary skill in the art before the effective filing date of the invention the idea of updating the semantic vector “during the training of the neural network model” as recited by the claim, it may be argued that this feature is not distinctly disclosed by Yan in view of Galassi and Cho. In other words, Yan and Galassi and Cho might not distinctly disclose:
so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model,
Agarwal is in the field of natural language processing. It discusses a “a word to vector model, wherein [a] sequence of words is replaced by corresponding vectors” [Agarwal, 0010]. In other words, in the combination, this word to vector model of Agarwal corresponds to the annotation function qaf of Galassi as mentioned above [Galassi, table II]. Moreover, Yan in view of Galassi, Cho, and Agarwal discloses:
so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model, ([Agarwal, 0010]: As noted above, Agarwal discloses a system having “a word to vector model, wherein [a] sequence of words is replaced by corresponding vectors” [Agarwal, 0010]. It further discloses that “the corresponding vectors are continually updated during training” [Agarwal, 0010]. In other words, in the combination, the corresponding vectors produced by the word to vector model of Agarwal correspond to q in Galassi and the “semantic vector of the candidate intent” of the claim. The continual updating during training as described in Agarwal maps to the ”updating” step of the claim. The applicant is invited to consult the conclusion of a previous Office action for several other references which also disclose this limitation.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the intention detector disclosed by Yan in view of Galassi and Cho with the natural language processing techniques of Agarwal because it “present[s] technological improvements as solutions to one or more of the above-mentioned technical problems” (e.g., the problem of outputting “grammatically wrong sentences, while the answers are required to be legally correct”) [Agarwal, 0003-0004], thereby resulting in a more effective system overall.
Claim 2
Yan in view of Galassi, Cho, and Agarwal discloses the elements of the parent claim(s). It also discloses:
[The method according to claim 1, wherein the step of outputting, by the feature extraction layer, a first semantic vector of each segmented word in a training text comprises:] obtaining, for each training text, a word vector of each segmented word in the training text; ([Yan, 0029]: Yan discloses “extract[ing] respective word embedding vector 210 from each word 25 included in the tokenized utterance 24” [Yan, 0029]. The word embedding vector of each word in the tokenized utterance is the “word vector of each segmented word” of the claim.)
obtaining an encoding result and an attention calculation result of each segmented word according to the word vector of each segmented word; ([Galassi, figure 2; Yan, figure 4]: Galassi discloses the use of attention for sequence-to-sequence tasks, where the attention mechanism appears between two recurrent neural networks [Galassi, figure 2; see also, section II.A]. In the combination, this architecture can be used for the “feature extraction layer” disclosed by Yan as mapped above, i.e., by inserting an attention function between the two bidirectional LSTM networks of Yan [Yan, figure 4]. The word embedding vectors of Yan are used as the input sequence x_1, …, x_T of Galassi [Galassi, figure 2 left] and the first bidirectional LSTM network of Yan is used as the first RNN (“BiRNN” [Galassi, figure 2 left]) of Galassi, so that the output of the first bidirectional LSTM network of Yan corresponds to the h_1, …, h_T of Galassi [Galassi, figure 2 left] and maps to the “encoding result” of the claim. The attention weights a_{t1}, …, a_{tT} [Galassi, figure 2 right] map to the “attention calculation result of each segmented word” of the claim.)
and decoding a splicing result between the encoding result and the attention calculation result of each segmented word, and taking a decoding result as the first semantic vector of each segmented word. ([Galassi, figure 2 and section II.A; Yan, figure 4]: The vector c_t produced by the attention mechanism [Galassi, figure 2] is defined to be a linear combinations of the “encoding results” as mapped above with weights given by the “attention calculation results” as mapped above [Galassi, section II.A equation (5)] and maps to the “splicing result between the encoding result and the attention calculation result of each segmented word” of the claim. The second bidirectional LSTM network of Yan [Yan, figure 4] is used as the second RNN (labeled “RNN” [Galassi, figure 2 left]). Galassi explains that the this second RNN is part of a “decoder” [Galassi, section II.A first two paragraphs], so its functionality maps to the “decoding” step of the claim, and the output it produces (which, in the combination, is also the output of the second bidirectional LSTM network of Yan) is the “decoding result” and the “first semantic vector of each segmented word” of the claim.)
The same motivation to combine applies.
Claim 3
Yan in view of Galassi, Cho, and Agarwal discloses:
[The method according to claim 1, wherein outputting, by the first recognition layer according to the semantic vector of the candidate intent and the first semantic vector of each segmented word in each training text outputted by the feature extraction layer, the first intent result of the training text and the score between each segmented word in the training text and the candidate intent comprises:] obtaining, for each training text according to the first semantic vector of each segmented word in the training text and the semantic vector of the candidate intent, a second semantic vector of each segmented word and the score between each segmented word and the candidate intent; ([Yan, 0033]: The “output of the second bidirectional LSTM network 214” [Yan, 0033] maps to the “second semantic vector of each segmented word” of the claim. The remaining elements are already mapped under the parent claim.)
and performing classification according to the second semantic vector of each segmented word, and taking a classification result as the first intent result of the training text. ([Yan, 0034]: As noted above, Yan discloses that the intention detector “select[s] a predefined intention 42 that estimates an intention of the user who enters the natural language input 20” [Yan, 0034]. This is a classification task, with the predefined intention that is selected by the intention detector being the “classification result” and the “first intent result” of the claim.)
The same motivation to combine applies.
Claim 5
Yan in view of Galassi, Cho, and Agarwal discloses the elements of the parent claim(s). It also discloses:
[The method according to claim 1, wherein acquiring the training data comprising the plurality of training texts and the first annotation intents of the plurality of training texts comprises:] acquiring training data comprising the plurality of training texts, the first annotation intents of the plurality of training texts and second annotation intents of the plurality of training texts. ([Yan, 0065]: As noted above, Yan discloses “[e]ach training utterance 332 may further include a training utterance intention selected from among the respective plurality of predefined intentions 42” [Yan, 0065]. The training utterance intentions included in the training utterances are the “second annotation intents of the plurality of training texts” of the claim.)
The same motivation to combine applies.
Claim 6
Yan in view of Galassi, Cho, and Agarwal discloses the elements of the parent claim(s). It also discloses:
[The method according to claim 5, wherein constructing the neural network model comprising the feature extraction layer and the first recognition layer comprises:] constructing the neural network model comprising the feature extraction layer, the first recognition layer ([Yan, figure 4]: As noted above, Yan discloses an intention detector realized as a neural network [Yan, figure 4], the components up through the second bidirectional LSTM layer being the “feature extraction layer” and the components after it being the “first recognition layer” of the claim.)
and a second recognition layer, the second recognition layer being configured to output, according to the first semantic vector of each segmented word in the training text outputted by the feature extraction layer, a second intent result of the training text. ([Yan, figure 4 and 0033]: As noted above, the “output of the second bidirectional LSTM network 214” [Yan, 0033] maps to the “first semantic vector of each segmented word” of the claim. Consequently, any component or combination of components which comes after the second bidirectional LSTM network of Yan [Yan, figure 4] would fall under the broadest reasonable interpretation of the “second recognition layer” of the claim, and its output maps to the “second intent result” of the claim. For concreteness, everything after the second bidirectional LSTM network of Yan is mapped to the “second recognition layer” of the claim, so that the predefined intention that is selected by the intention detector [Yan, 0034] maps to the “second intent result” of the claim.)
The same motivation to combine applies.
Claim 7
Yan in view of Galassi, Cho, and Agarwal discloses the elements of the parent claim(s). It also discloses:
[The method according to claim 6, wherein training the neural network model according to the word segmentation results of the plurality of training texts and the first annotation intents of the plurality of training texts to obtain the intent recognition model comprises:] inputting the word segmentation results of the plurality of training texts to the neural network model to obtain the first intent result and the second intent result outputted by the neural network model for each training text; ([Yan, 0034, 0052, 0067]: As noted above, the training utterances of Yan [Yan, 0065] map to the “plurality of training texts” of the claim, and the words in the tokenized training utterances of Yan [Yan, 0052] map to the “word segmentation results of the plurality of training texts” of the claim. The predefined intention that is selected by the intention detector for a given training utterance [Yan, 0034] maps to the “first intent result” and the “second intent result” of the claim.)
calculating a first loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts, and calculating a second loss function value according to the second intent results of the plurality of training texts and the second annotation intents of the plurality of training texts; ([Cho, abstract]: Cho discusses loss functions which measure “prediction error” (indicating, for example, that “[i]f the prediction deviates too far from real data, a loss function would generate a very large value”) [Cho, abstract]. The output of the intention detector of Yan corresponds to the prediction of Cho and to the “first intent results” of the claim, and the true training utterance intention of Yan corresponds to the real data of Cho and the “first annotation intents” as well as the “second annotation intents” of the claim. The prediction error, i.e., the value output by a loss function as in Cho, maps to the “first loss function value” and the “second loss function value” of the claim.)
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated first loss function value and the calculated second loss function value, until the neural network model converges, to obtain the intent recognition model. ([Cho, abstract]: Cho discloses that “[p]rogressively, with the help of some optimization function, the loss function lowers the prediction error by providing the network architecture with information that can control the weights of the network architecture” [Cho, abstract]. As noted above, the prediction error is the “calculated first calculated loss function value and second loss function value” of the claim. The intention detector of Yan in view of Galassi is the “neural network model” of the claim, and the weights of the intention detector are the “parameters of the neural network model and the semantic vector of the candidate intent” of the claim.)
The same motivation to combine applies.
Claim 8
Yan in view of Galassi, Cho, and Agarwal discloses the elements of the parent claim(s). It also discloses:
A method for intent recognition, comprising: acquiring a to-be-recognized text; and inputting word segmentation results of the to-be-recognized text to an intent recognition model, and obtaining a first intent result and a second intent result of the to-be-recognized text according to an output result of the intent recognition model; wherein the intent recognition model is pre-trained with the method according to claim 1. ([Yan, 0023, 0029, 0034]: As explained under the parent claims, the “natural language input 20” [Yan, 0023] maps to the “to-be-recognized text of the claim”, and the words in the “tokenized utterance 24” generated by the preprocessor [Yan, 0023, 0052] maps to the “word segmentation results of the to-be-recognized text” of the claim. The tokenized utterance serves as input for the intention detector [Yan, 0028], which maps to the “intent recognition model” of the claim. The predefined intention selected by the intention detector [Yan, 0034] maps to the “first intent result”, the “second intent result”, and the “output result” of the claim. See the mappings of the parent claim for the method of training the model.)
The same motivation to combine applies.
Claim 9
Yan in view of Galassi and Agarwal discloses the elements of the parent claim(s). It also discloses:
[The method according to claim 8, wherein obtaining the first intent result and the second intent result of the to-be-recognized text according to the output result of the intent recognition model comprises:] obtaining the second intent result of the to-be-recognized text according to scores between segmented words in the to-be-recognized text and a candidate intent outputted by the intent recognition model. ([Yan, 0034; Galassi, figure 4]: As explained under the parent claims, the words in the tokenized utterance [Yan, 0023, 0052] are the “segmented words in the to-be-recognized text” of the claim. One of the predefined intentions which may be selected by the intention detector is the “candidate intent” of the claim, and the attention weights [Galassi, figure 4] are the “scores between segmented words in the to-be-recognized text and [the] candidate intent” of the claim.)
The same motivation to combine applies.
Claim 10
Yan discloses:
An electronic device, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform ([Yan, 0073 and figure 16]: Yan discloses that “the methods and processes described herein may be tied to a computing system of one or more computing devices” in the sense of being “implemented as a computer-application program or service, an application-programming interface (API), a library, and/or other computer-program product” [Yan, 0073], the computing system being depicted in [Yan, figure 16]. The computing system is the “electronic device” of the claim, its processor and memory map to the “processor” and “memory” of the claim, and the computer implementation of the methods disclosed therein map to the “instructions” of the claim.) a method for natural language processing, wherein the method comprises: ([Yan, 0022]: Yan discloses a method which receives “natural language input” [Yan, 0022].)
acquiring training data comprising a plurality of training texts and first annotation intents of the plurality of training texts; ([Yan, 0065]: Yan discloses a “training dataset 330” which contains “training utterances 332” and “[e]ach training utterance 332 may further include a training utterance intention selected from among the respective plurality of predefined intentions 42” [Yan, 0065]. The training utterances map to the “plurality of training texts” of the claim, and the training utterance intentions of the training utterances are the “first annotation intents” of the claim.)
constructing a neural network model comprising a feature extraction layer and a first recognition layer, ([Yan, 0023, 0029, 0034, and figure 4]: In the system disclosed by Yan, “natural language input 20, in the form of a text input” is sent “into a preprocessor 22. The preprocessor 22 may be configured to generate a tokenized utterance 24 based on the natural language input 20” [Yan, 0023]. Then an “intention detector 40… receive[s] the tokenized utterance 24 from the preprocessor 22” [Yan, 0029] and ultimately “select[s] a predefined intention 42 that estimates an intention of the user who enters the natural language input 20” [Yan, 0034]. The intention detector is a neural network having several components performing computations in a sequence [Yan, figure 4 elements 210, 212, 214, 216, 218]. The intention detector maps to the “neural network” of the claim, its components up through the second bidirectional LSTM network [Yan, figure 4 elements 210, 212, 214] map to the “feature extraction layer” of the claim, and the remainder [Yan, figure 4 elements 216, 218] maps to the “first recognition layer” of the claim.)
the first recognition layer being configured to output, according to [a semantic vector of] a candidate intent and a first semantic vector of each segmented word in each training text outputted by the feature extraction layer, a first intent result of the training text ([Yan, 0033-0034, 0052, and figure 4]: Yan explains that the tokenized utterance generated by the preprocessor “may include one or more words and one or more metadata tokens respectively associated with the one or more words” [Yan, 0052]. The words in the tokenized utterance map to the “segmented words” of the claim, and the “output of the second bidirectional LSTM network 214” [Yan, 0033] maps to the “first semantic vector of each segmented word” of the claim (since the components of the intention detector up through the second bidirectional LSTM network [Yan, figure 4 elements 210, 212, 214] are mapped above to the “feature extraction layer” of the claim). The output of the second bidirectional LSTM network enters the remainder of the network, which, as noted above, “select[s] a predefined intention 42 that estimates an intention of the user who enters the natural language input 20” [Yan, 0034]. Any of the possible predefined intentions which may be selected by the intention detector map to the “candidate intent” of the claim, and the predefined intention that is actually selected by the intention detector maps to the “first intent result” of the claim.)
training the neural network model according to word segmentation results of the plurality of training texts and the first annotation intents of the plurality of training texts to obtain an intent recognition model; ([Yan, 0052, 0065, 0067]: Yan discloses “train[ing] the intention detector 40 using the plurality of training utterances 332” [Yan, 0067] where, as noted above, “[e]ach training utterance 332 may further include a training utterance intention selected from among the respective plurality of predefined intentions” [Yan, 0065]. It also discloses, as noted above, a preprocessing step of “generating a tokenized utterance based on the natural language input. The tokenized utterance may include one or more words and one or more metadata tokens respectively associated with the one or more words” [Yan, 0052]. The training utterances map to the “plurality of training texts” of the claim as noted above, and the tokenized training utterances map to the “word segmentation results of the plurality of training texts” of the claim. The training utterance intentions that are included in the training utterances map to the “first annotation intents” as noted above, and intention detector is the “intent recognition model” of the claim.)
wherein training the neural network model according to the word segmentation results of the plurality of training texts and the first annotation intents of the plurality of training texts to obtain the intent recognition model comprises: inputting the word segmentation results of the plurality of training texts to the neural network model to obtain the first intent result outputted by the neural network model for each training text; ([Yan, 0034, 0052, 0067]: As noted above, Yan discloses training utterances mapping to the “plurality of training texts” of the claim and tokenized training utterances mapping to the “word segmentation results of the plurality of training texts” of the claim. The output of the intention detector on a training utterance maps to the “first intent result” of the claim.)
Yan might not distinctly disclose:
[the first recognition layer being configured to output, according to] a semantic vector of a candidate intent and [a first semantic vector of each segmented word in a training text outputted by the feature extraction layer, a first intent result of the training text] and a score between each segmented word in the training text and the candidate intent, wherein the semantic vector of the candidate intent is configured to represent semantics of the candidate intent;
calculating a loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts;
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated loss function value, until the neural network model converges, to obtain the intent recognition model, so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model, to obtain the intent recognition model.
Galassi is in the field of natural language processing. It discusses the use of attention in a variety of natural processing tasks [Galassi, table 1], including a number of classification tasks (of which the intent detection task of Yan as disclosed above is an example). The input to the attention mechanism consists of keys (i.e., the columns of the matrix K), values (i.e., the columns of the matrix V), and a query text [Galassi, figure 4]. In the combination, the word embedding vectors which are extracted from each word in the tokenized utterance [Yan, 0029] are used as the keys of Galassi, the output of the second bidirectional LSTM network [Yan, 0033] are used as the values of Galassi, and the “candidate intent” as mapped above is used as the query text of Galassi. In other words, an attention mechanism of Galassi is inserted into the intention detector of Yan after the second bidirectional LSTM layer as a part of the “first recognition layer” of the claim. Then Yan in view of Galassi discloses:
[the first recognition layer being configured to output, according to] a semantic vector of [a candidate intent and a first semantic vector of each segmented word in a training text outputted by the feature extraction layer, a first intent result of the training text] and a score between each segmented word in the training text and the candidate intent, wherein the semantic vector of the candidate intent is configured to represent semantics of the candidate intent; ([Galassi, figure 4]: Galassi discloses representing the query text (i.e., the “candidate intent” of the claim) as a vector q in R^{n_q} [Galassi, figure 4], so q in R^{n_q} is the “semantic vector of [the] candidate intent” of the claim. This vector “represent[s] semantics of the candidate intent” as recited by the claim. The attention mechanism computes “[a]ttention weights” [Galassi, figure 4] which “represent the relevance of each element to the given task, with respect to q and K” [Galassi, section II.B paragraph beginning “Such weights”]. In the combination as noted above, the word embedding vectors of the words in the tokenized utterance of Yan [Yan, 0029] are used as key vectors of Galassi, so each attention weight map to a “score between each segmented word in the training text and the candidate intent” of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to incorporate an attention mechanism as in Galassi into the intention detector of Yan because attention “offer[s] a performance gain” and “can also be used as a tool for interpreting the behaviour of neural architectures” [Galassi, section I paragraph beginning “Besides offering a performance gain”] and consequently “has become an increasingly common ingredient of neural architectures for NLP” [Galassi, section I paragraph beginning “For all these reasons”], so the combination would be more effective overall.
While Yan in view of Galassi discloses a training an intention detector implemented as a neural network, and training a neural network involves iteratively adjusting model parameters until model performance converges, model performance being measured by means of a loss function which compares output generated by the model (the “first intent results” of the claim) against true output in labeled training data (the “first annotation intents” of the claim), it may nonetheless be argued that Yan in view of Galassi does not distinctly disclose:
calculating a loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts;
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated loss function value, until the neural network model converges, to obtain the intent recognition model, so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model, to obtain the intent recognition model.
Cho is in the field of machine learning. It discusses loss functions for training neural networks for classification [Cho, abstract]. The intention detector of Yan is a neural network performing a classification task. Moreover, Yan in view of Galassi and Cho discloses:
calculating a loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts; ([Cho, abstract]: Cho discusses loss functions which measure “prediction error” (indicating, for example, that “[i]f the prediction deviates too far from real data, a loss function would generate a very large value”) [Cho, abstract]. The output of the intention detector of Yan corresponds to the prediction of Cho and to the “first intent results” of the claim, and the true training utterance intention of Yan corresponds to the real data of Cho and the “first annotation intents” of the claim. The prediction error, i.e., the value output by a loss function as in Cho, maps to the “loss function value” of the claim.)
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated loss function value, until the neural network model converges, [so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model,] to obtain the intent recognition model. ([Cho, abstract]: Cho discloses that “[p]rogressively, with the help of some optimization function, the loss function lowers the prediction error by providing the network architecture with information that can control the weights of the network architecture” [Cho, abstract]. As noted above, the prediction error is the “calculated loss function value” of the claim. The intention detector of Yan in view of Galassi is the “neural network model” of the claim, and the weights of the intention detector are the “parameters of the neural network model and the semantic vector of the candidate intent” of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to use loss functions as described in Cho to train the intention detector of Yan in view of Galassi because “loss functions [play] an important role in training the network architecture” [Cho, abstract].
Galassi discloses that the query vector (i.e., the “semantic vector of the candidate intent” of the claim) is produced by an annotation function denoted “qaf” [Galassi, table II] and discusses architectures where “the layers in charge of annotation are trained together with the attention model” [Galassi, section IV.A first paragraph]. While this would have rendered obvious to a person of ordinary skill in the art before the effective filing date of the invention the idea of updating the semantic vector “during the training of the neural network model” as recited by the claim, it may be argued that this feature is not distinctly disclosed by Yan in view of Galassi and Cho. In other words, Yan and Galassi and Cho might not distinctly disclose:
so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model,
Agarwal is in the field of natural language processing. It discusses a “a word to vector model, wherein [a] sequence of words is replaced by corresponding vectors” [Agarwal, 0010]. In other words, in the combination, this word to vector model of Agarwal corresponds to the annotation function qaf of Galassi as mentioned above [Galassi, table II]. Moreover, Yan in view of Galassi, Cho, and Agarwal discloses:
so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model, ([Agarwal, 0010]: As noted above, Agarwal discloses a system having “a word to vector model, wherein [a] sequence of words is replaced by corresponding vectors” [Agarwal, 0010]. It further discloses that “the corresponding vectors are continually updated during training” [Agarwal, 0010]. In other words, in the combination, the corresponding vectors produced by the word to vector model of Agarwal correspond to q in Galassi and the “semantic vector of the candidate intent” of the claim. The continual updating during training as described in Agarwal maps to the ”updating” step of the claim. The applicant is invited to consult the conclusion of a previous Office action for several other references which also disclose this limitation.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the intention detector disclosed by Yan in view of Galassi and Cho with the natural language processing techniques of Agarwal because it “present[s] technological improvements as solutions to one or more of the above-mentioned technical problems” (e.g., the problem of outputting “grammatically wrong sentences, while the answers are required to be legally correct”) [Agarwal, 0003-0004], thereby resulting in a more effective system overall.
Claims 11-12 and 14-16 inherit limitations from claim 10 and recite additional limitations which are substantially similar to those recited by claims 2-3 and 5-7, respectively, so they are rejected by the same rationale.
Claim 17
Yan discloses:
A non-transitory computer readable storage medium with computer instructions stored thereon, wherein the computer instructions are used for causing ([Yan, 0078]: Yan discloses “Non-volatile storage device 606 includ[ing] one or more physical devices configured to hold instructions executable by the logic processors to implement the methods and processes described herein” [Yan, 0078]. The non-volatile storage device maps to the “non-transitory computer readable medium” of the claim.) a method for natural language processing, wherein the method comprises: ([Yan, 0022]: Yan discloses a method which receives “natural language input” [Yan, 0022].)
acquiring training data comprising a plurality of training texts and first annotation intents of the plurality of training texts; ([Yan, 0065]: Yan discloses a “training dataset 330” which contains “training utterances 332” and “[e]ach training utterance 332 may further include a training utterance intention selected from among the respective plurality of predefined intentions 42” [Yan, 0065]. The training utterances map to the “plurality of training texts” of the claim, and the training utterance intentions of the training utterances are the “first annotation intents” of the claim.)
constructing a neural network model comprising a feature extraction layer and a first recognition layer, ([Yan, 0023, 0029, 0034, and figure 4]: In the system disclosed by Yan, “natural language input 20, in the form of a text input” is sent “into a preprocessor 22. The preprocessor 22 may be configured to generate a tokenized utterance 24 based on the natural language input 20” [Yan, 0023]. Then an “intention detector 40… receive[s] the tokenized utterance 24 from the preprocessor 22” [Yan, 0029] and ultimately “select[s] a predefined intention 42 that estimates an intention of the user who enters the natural language input 20” [Yan, 0034]. The intention detector is a neural network having several components performing computations in a sequence [Yan, figure 4 elements 210, 212, 214, 216, 218]. The intention detector maps to the “neural network” of the claim, its components up through the second bidirectional LSTM network [Yan, figure 4 elements 210, 212, 214] map to the “feature extraction layer” of the claim, and the remainder [Yan, figure 4 elements 216, 218] maps to the “first recognition layer” of the claim.)
the first recognition layer being configured to output, according to [a semantic vector of] a candidate intent and a first semantic vector of each segmented word in a training text outputted by the feature extraction layer, a first intent result of the training text ([Yan, 0033-0034, 0052, and figure 4]: Yan explains that the tokenized utterance generated by the preprocessor “may include one or more words and one or more metadata tokens respectively associated with the one or more words” [Yan, 0052]. The words in the tokenized utterance map to the “segmented words” of the claim, and the “output of the second bidirectional LSTM network 214” [Yan, 0033] maps to the “first semantic vector of each segmented word” of the claim (since the components of the intention detector up through the second bidirectional LSTM network [Yan, figure 4 elements 210, 212, 214] are mapped above to the “feature extraction layer” of the claim). The output of the second bidirectional LSTM network enters the remainder of the network, which, as noted above, “select[s] a predefined intention 42 that estimates an intention of the user who enters the natural language input 20” [Yan, 0034]. Any of the possible predefined intentions which may be selected by the intention detector map to the “candidate intent” of the claim, and the predefined intention that is actually selected by the intention detector maps to the “first intent result” of the claim.)
training the neural network model according to word segmentation results of the plurality of training texts and the first annotation intents of the plurality of training texts to obtain an intent recognition model; ([Yan, 0052, 0065, 0067]: Yan discloses “train[ing] the intention detector 40 using the plurality of training utterances 332” [Yan, 0067] where, as noted above, “[e]ach training utterance 332 may further include a training utterance intention selected from among the respective plurality of predefined intentions” [Yan, 0065]. It also discloses, as noted above, a preprocessing step of “generating a tokenized utterance based on the natural language input. The tokenized utterance may include one or more words and one or more metadata tokens respectively associated with the one or more words” [Yan, 0052]. The training utterances map to the “plurality of training texts” of the claim as noted above, and the tokenized training utterances map to the “word segmentation results of the plurality of training texts” of the claim. The training utterance intentions that are included in the training utterances map to the “first annotation intents” as noted above, and intention detector is the “intent recognition model” of the claim.)
wherein training the neural network model according to the word segmentation results of the plurality of training texts and the first annotation intents of the plurality of training texts to obtain the intent recognition model comprises: inputting the word segmentation results of the plurality of training texts to the neural network model to obtain the first intent result outputted by the neural network model for each training text; ([Yan, 0034, 0052, 0067]: As noted above, Yan discloses training utterances mapping to the “plurality of training texts” of the claim and tokenized training utterances mapping to the “word segmentation results of the plurality of training texts” of the claim. The output of the intention detector on a training utterance maps to the “first intent result” of the claim.)
Yan might not distinctly disclose:
[the first recognition layer being configured to output, according to] a semantic vector of a candidate intent and [a first semantic vector of each segmented word in a training text outputted by the feature extraction layer, a first intent result of the training text] and a score between each segmented word in the training text and the candidate intent, wherein the semantic vector of the candidate intent is configured to represent semantics of the candidate intent;
calculating a loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts;
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated loss function value, until the neural network model converges, to obtain the intent recognition model, so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model, to obtain the intent recognition model.
Galassi is in the field of natural language processing. It discusses the use of attention in a variety of natural processing tasks [Galassi, table 1], including a number of classification tasks (of which the intent detection task of Yan as disclosed above is an example). The input to the attention mechanism consists of keys (i.e., the columns of the matrix K), values (i.e., the columns of the matrix V), and a query text [Galassi, figure 4]. In the combination, the word embedding vectors which are extracted from each word in the tokenized utterance [Yan, 0029] are used as the keys of Galassi, the output of the second bidirectional LSTM network [Yan, 0033] are used as the values of Galassi, and the “candidate intent” as mapped above is used as the query text of Galassi. In other words, an attention mechanism of Galassi is inserted into the intention detector of Yan after the second bidirectional LSTM layer as a part of the “first recognition layer” of the claim. Then Yan in view of Galassi discloses:
[the first recognition layer being configured to output, according to] a semantic vector of [a candidate intent and a first semantic vector of each segmented word in a training text outputted by the feature extraction layer, a first intent result of the training text] and a score between each segmented word in the training text and the candidate intent, wherein the semantic vector of the candidate intent is configured to represent semantics of the candidate intent; ([Galassi, figure 4]: Galassi discloses representing the query text (i.e., the “candidate intent” of the claim) as a vector q in R^{n_q} [Galassi, figure 4], so q in R^{n_q} is the “semantic vector of [the] candidate intent” of the claim. This vector “represent[s] semantics of the candidate intent” as recited by the claim. The attention mechanism computes “[a]ttention weights” [Galassi, figure 4] which “represent the relevance of each element to the given task, with respect to q and K” [Galassi, section II.B paragraph beginning “Such weights”]. In the combination as noted above, the word embedding vectors of the words in the tokenized utterance of Yan [Yan, 0029] are used as key vectors of Galassi, so each attention weight map to a “score between each segmented word in the training text and the candidate intent” of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to incorporate an attention mechanism as in Galassi into the intention detector of Yan because attention “offer[s] a performance gain” and “can also be used as a tool for interpreting the behaviour of neural architectures” [Galassi, section I paragraph beginning “Besides offering a performance gain”] and consequently “has become an increasingly common ingredient of neural architectures for NLP” [Galassi, section I paragraph beginning “For all these reasons”], so the combination would be more effective overall.
While Yan in view of Galassi discloses a training an intention detector implemented as a neural network, and training a neural network involves iteratively adjusting model parameters until model performance converges, model performance being measured by means of a loss function which compares output generated by the model (the “first intent results” of the claim) against true output in labeled training data (the “first annotation intents” of the claim), it may nonetheless be argued that Yan in view of Galassi does not distinctly disclose:
calculating a loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts;
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated loss function value, until the neural network model converges, to obtain the intent recognition model, so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model, to obtain the intent recognition model.
Cho is in the field of machine learning. It discusses loss functions for training neural networks for classification [Cho, abstract]. The intention detector of Yan is a neural network performing a classification task. Moreover, Yan in view of Galassi and Cho discloses:
calculating a loss function value according to the first intent results of the plurality of training texts and the first annotation intents of the plurality of training texts; ([Cho, abstract]: Cho discusses loss functions which measure “prediction error” (indicating, for example, that “[i]f the prediction deviates too far from real data, a loss function would generate a very large value”) [Cho, abstract]. The output of the intention detector of Yan corresponds to the prediction of Cho and to the “first intent results” of the claim, and the true training utterance intention of Yan corresponds to the real data of Cho and the “first annotation intents” of the claim. The prediction error, i.e., the value output by a loss function as in Cho, maps to the “loss function value” of the claim.)
and adjusting parameters of the neural network model and the semantic vector of the candidate intent according to the calculated loss function value, until the neural network model converges, [so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model,] to obtain the intent recognition model. ([Cho, abstract]: Cho discloses that “[p]rogressively, with the help of some optimization function, the loss function lowers the prediction error by providing the network architecture with information that can control the weights of the network architecture” [Cho, abstract]. As noted above, the prediction error is the “calculated loss function value” of the claim. The intention detector of Yan in view of Galassi is the “neural network model” of the claim, and the weights of the intention detector are the “parameters of the neural network model and the semantic vector of the candidate intent” of the claim.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to use loss functions as described in Cho to train the intention detector of Yan in view of Galassi because “loss functions [play] an important role in training the network architecture” [Cho, abstract].
Galassi discloses that the query vector (i.e., the “semantic vector of the candidate intent” of the claim) is produced by an annotation function denoted “qaf” [Galassi, table II] and discusses architectures where “the layers in charge of annotation are trained together with the attention model” [Galassi, section IV.A first paragraph]. While this would have rendered obvious to a person of ordinary skill in the art before the effective filing date of the invention the idea of updating the semantic vector “during the training of the neural network model” as recited by the claim, it may be argued that this feature is not distinctly disclosed by Yan in view of Galassi and Cho. In other words, Yan and Galassi and Cho might not distinctly disclose:
so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model,
Agarwal is in the field of natural language processing. It discusses a “a word to vector model, wherein [a] sequence of words is replaced by corresponding vectors” [Agarwal, 0010]. In other words, in the combination, this word to vector model of Agarwal corresponds to the annotation function qaf of Galassi as mentioned above [Galassi, table II]. Moreover, Yan in view of Galassi, Cho, and Agarwal discloses:
so that the semantic vector of the candidate intent of the neural network model is updated during the training of the neural network model, ([Agarwal, 0010]: As noted above, Agarwal discloses a system having “a word to vector model, wherein [a] sequence of words is replaced by corresponding vectors” [Agarwal, 0010]. It further discloses that “the corresponding vectors are continually updated during training” [Agarwal, 0010]. In other words, in the combination, the corresponding vectors produced by the word to vector model of Agarwal correspond to q in Galassi and the “semantic vector of the candidate intent” of the claim. The continual updating during training as described in Agarwal maps to the ”updating” step of the claim. The applicant is invited to consult the conclusion of a previous Office action for several other references which also disclose this limitation.)
Before the effective filing date of the invention, it would have been obvious to a person of ordinary skill in the art to combine the intention detector disclosed by Yan in view of Galassi and Cho with the natural language processing techniques of Agarwal because it “present[s] technological improvements as solutions to one or more of the above-mentioned technical problems” (e.g., the problem of outputting “grammatically wrong sentences, while the answers are required to be legally correct”) [Agarwal, 0003-0004], thereby resulting in a more effective system overall.
Claims 18-19 inherit limitations from claim 17 and recite additional limitations which are substantially similar to those recited by claims 2-3, respectively, so they are rejected by the same rationale.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Shishir AGRAWAL whose telephone number is +1 703-756-1183. The examiner can normally be reached Monday through Thursday, 08:30-14:30 Pacific Time.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey SHMATOV can be reached on +1 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is +1 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at +1 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call +1 800-786-9199 (IN USA OR CANADA) or +1 571-272-1000.
/S.A./Examiner, Art Unit 2123
/ALEXEY SHMATOV/Supervisory Patent Examiner, Art Unit 2123