DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
2. Claims 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter as follows. Claim 20 defines a “computer program element” embodying functional descriptive material. However, the claim does not define a “non-transitory” computer-readable element, “non-transitory” computer-readable medium, or a “non-transitory” computer-readable memory and is thus non-statutory for that reason (i.e., “When functional descriptive material is recorded on some computer-readable medium it becomes structurally and functionally interrelated to the medium and will be statutory in most cases since use of technology permits the function of the descriptive material to be realized” – Guidelines Annex IV). The scope of the presently claimed invention encompasses products that are not necessarily computer readable, and thus NOT able to impart any functionality of the recited program. The examiner suggests amending the claim(s) to embody the program on a “non-transitory computer-readable medium” or equivalent; assuming the specification does NOT define the computer readable medium as a “signal”, “carrier wave”, or “transmission medium” which are deemed non-statutory (refer to “note” below). Any amendment to the claim should be commensurate with its corresponding disclosure.
Claim Rejections - 35 USC § 103
1. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
2. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
3. Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over DeFelice (US 20190236139 A1 hereinafter, DeFelice ‘139) in view of Bellegarda (US 20200104369 hereinafter, Bellegarda ‘369).
Regarding claim 1; DeFelice ‘139 discloses a system (Fig. 2, Generative System 200) for recognizing a sentiment of a user's feedback (i.e. A personality and sentiment model is created for a target individual or class of individuals and the personality model is used as an input to a text generation system. A feedback loop is also incorporated, allowing for learning to occur both with regard to the personality model and the generated text. Paragraph 0009)
comprising:
an encoder (Fig. 2, Encoder 222) configured to receive an input text for the user's feedback (i.e. A VAE consists of paired encoder and generator networks which encode a text into to a latent representation and generate samples from the latent space. Paragraph 0007),
break up the input text into text tokens (i.e. The hidden layers are connected by feedforward networks 1111a-d, each of which provides an intermediate prediction 1107a-e mapping the computed output of the RNN to a probability distribution over candidate tokens by applying a softmax transformation, and the corresponding prediction output 1109a-e which picks the maximum likelihood token given the probability distribution. Paragraph 0088)
and encode each text token (i.e. Identified pinned texts are then re-encoded as a single multi-word token instead of a series of tokens with an underlying probability. Paragraph 0107);
and a decoder (Fig. 2, Decoder 224) connected to the encoder (i.e. Fig. 10 shows one embodiment of encoder 222 and decoder 224 together with code 223. Paragraph 0087);
and configured to output a sentiment score for the input text (i.e. The ordering of the information flow between the discriminator 226 and evaluator 228 is contingent both upon the desired intuitive model as well as the underlying hardware available and whether parallel application is reasonable. If the discriminator 226 is first in time, then its score as to the “humanness” of the generated text is used as an input to the evaluator 228. The intuition for this model is that a more human-like input should also score better relative to the target messaging, sentiment values, and information content measured by the evaluator 228. Paragraph 0048)
wherein the encoder is further configured to read the encoded text token in a bidirectional manner to obtain a forward node value and a backward node value (i.e. The outputs of forward/backward RNN encoder n (1162e) are connected to the inputs of forwards/backwards RNN decoder 0 (at 1172a). The forward/backward decoder networks 1172a-e each create a prediction at 1177a-e and outputs 1179a-e and the final prediction output 1179e is a representation of the highest-likelihood next token considering the full context of the sentence based upon the latent representation learned by the encoder 1160. Paragraph 0090),
and concatenate the forward node value and the backward node value together to obtain a vector representing semantic meaning of the input text (i.e. The encoder 22 network encodes the words within the source text 201 as a list of vectors, where each vector represents the contextual meaning of the words within the text. Paragraph 0046)
and the decoder is configured to receive the vector from the encoder (i.e. The attention score for each encoded vector is calculated by normalizing the vectors via softmax function 1220 and then multiplying the output by its weight, which is fed to each layer of the decoder RNN. Paragraph 0092).
Examiner reasonably believes that DeFelice ‘139 discloses the sentiment score as expressed above. However, Examiner cites Bellegarda ‘369 to cure any deficiencies of DeFelice ‘139).
Bellegarda ‘369 discloses a sentiment score (i.e. Sentiment prediction model 806 is configured to process the vector representations of each word in the word sequence determined by encoder 804. The vector representations are processed by sentiment prediction model 806 in the order that each respective word appears in the word sequence. Based on the vector representations, sentiment prediction model 806 determines likelihood scores for a plurality of predicted candidate sentiments (e.g., sentiment states). Paragraph 0253).
obtain a probability of each sentiment score by mapping the vector into each sentiment score (i.e. Sentiment prediction model 806 is configured to process the vector representations of each word in the word sequence determined by encoder 804. The vector representations are processed by sentiment prediction model 806 in the order that each respective word appears in the word sequence. Based on the vector representations, sentiment prediction model 806 determines likelihood scores for a plurality of predicted candidate sentiments (e.g., sentiment states). Each likelihood score indicates, for example, the likelihood that the word sequence conveys a respective predicted candidate sentiment. Paragraph 0253)
and determine the sentiment score for the input text based on the probability (i.e. The predicted candidate sentiment having the highest likelihood score among the plurality of predicted candidate sentiments is, for example, selected as the sentiment most likely represented by the word sequence. Paragraph 0253)
DeFelice ‘139 and Bellegarda ‘369 are combinable because they are from same field of endeavor of speech systems (Bellegarda ‘369 at “Field”).
Before the effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the speech system as taught by DeFelice ‘139 by adding the limitations as taught by Bellegarda ‘369. The motivation for doing so would have been advantageous because it can help to achieve a higher degree of intelligence in human-computer interaction. Therefore, it would have been obvious to combine DeFelice ‘139 with Bellegarda ‘369 to obtain the invention as specified.
Regarding claim 2; DeFelice ‘139 discloses wherein the input text relates to a sentence, and the encoder is configured to break up the input text into the text tokens each corresponding to a plurality of words included in the sentence (i.e. Identified pinned texts are then re-encoded as a single multi-word token instead of a series of tokens with an underlying probability. Paragraph 0107);
Regarding claim 3; Bellegarda ‘369 discloses wherein the encoder is further configured to input the encoded text token into a Bidirectional Long Short-Term Memory Network (Bi-LSTM) model configured to read the encoded text token in the bidirectional manner (i.e. The described techniques utilize a sentiment prediction model having bidirectional long short-term memory (LSTM) networks with one or more convolution-and-pooling stages. The bidirectional LSTM networks process vector representations of words in a textual word sequence to determine forward and backward word-level context feature vectors. See Abstract)
Regarding claim 4; DeFelice ‘139 discloses wherein the encoder comprises a series of Bi-LSTM layers (Fig. 11c, Each Block 1162a-e) in which the Bi-LSTM model is operated (i.e. Turning now to Fig. 11c, the forward/backward RNN model is shown in an encoder/decoder configuration, with group 1160 corresponding to encoder 1010 of Fig. 10 (itself corresponding to encoder 222 of Fig. 2) and group 1170 corresponding to decoder 1020 of Fig 10 (itself corresponding to decoder 224 of Fig. 2). Each block 1162a-e is a forwards/backwards RNN layer as described relative to Fig. 11b, arranged in an n-deep stack of encoding layers where each layer is approximately ½ the size of the previous layer. Paragraph 0090;
in a first Bi-LSTM layer of the Bi-LSTM layers, the encoder is configured to input the encoded text token into the Bi-LSTM model, and the Bi-LSTM model is configured to obtain a first forward node value and a first backward node value and concatenate the first forward node value and the first backward node value together, and in a next Bi-LSTM layer of the Bi-LSTM layers, the encoder is configured to input the concatenated first forward node value and first backward node value into the Bi-LSTM model, and the Bi-LSTM model is configured to obtain a next forward node value and a next backward node value and concatenate the next forward node value and the next backward node value together (i.e. The outputs of forward/backward RNN encoder n (1162e) are connected to the inputs of forwards/backwards RNN decoder 0 (at 1172a). The forward/backward decoder networks 1172a-e each create a prediction at 1177a-e and outputs 1179a-e and the final prediction output 1179e is a representation of the highest-likelihood next token considering the full context of the sentence based upon the latent representation learned by the encoder 1160. Also, in one embodiment, each RNN 1172 is also provided with the value of the prospect personality model at input 1171. Paragraph 0090-0091);
Regarding claim 5; Bellegarda ‘369 discloses wherein the encoder comprises a max pooling layer in which a max pooling function is operated, in a final Bi-LSTM layer of the Bi-LSTM layers, the Bi-LSTM model is configured to obtain a final forward node value and a final backward node value, and concatenate the final forward node value and the final backward node value together, and in the max pooling layer, the max pooling function is configured to process the concatenated final forward node value and final backward node value to obtain the vector (i.e. A pooled phrase-level feature vector is determined by pooling the first forward phrase-level feature vector, the second forward phrase-level feature vector, the first backward phrase-level feature vector, and the second backward phrase-level feature vector. Based on the pooled phrase-level feature vector, a sentiment conveyed by the word sequence is determined. A result is generated by performing an action in accordance with the determined sentiment and the generated result is outputted. See Abstract & Paragraphs 0007-0008, 0255 & 0264-0282).
Regarding claim 6; Bellegarda ‘369 discloses wherein the encoder is further configured to encode the each text token using a Byte Pair Encoding model (i.e. Encoder 804 is configured to process each word (or token) in the word sequence and determine a respective vector representation for each word. The determined vector representation of a word (or token) in the word sequence can embed information indicating whether the word corresponds to the beginning or end of a textual unit (e.g., phrase, sentence, paragraph, etc.). In some examples, vector space models are used to determine the vector representations of words in the word sequence. Sentiment prediction model 806 is configured to process the vector representations of each word in the word sequence determined by encoder 804. Paragraphs 0252-0253).
Regarding claim 7; Bellegarda ‘369 discloses wherein the encoder is further configured to encode a stop token representing an end of the input text using the Byte Pair Encoding model, and input the encoded stop token into the Bi-LSTM model (i.e. During operation, vector representations of words (or tokens) in the word sequences are provided to forward LSTM network 902 and backward LSTM network 904 for processing. Paragraph 0256)
Regarding claim 8; DeFelice ‘139 discloses wherein the decoder is configured to input the vector into a Feedforward Neural Network (FFNN) model configured to map the vector to the each sentiment score (i.e. A particular encoded vector ei at decoding step hj is multiplied by parameters w1 (at 1212a-c), w2 (at 1214a-c), with the outputs combined via tanh activation function (at 1216a-c) and weighted by v (at 1218a-c), with w1, w2, and v as learned parameters. The attention score for each encoded vector is calculated by normalizing the vectors via softmax function 1220 and then multiplying the output by its weight, which is fed to each layer of the decoder RNN. Paragraph 0092).
Regarding claim 9; DeFelice ‘139 discloses wherein the decoder comprises a series of FFNN layers in which the FFNN model is operated, and a Softmax layer in which a Softmax function is operated, the decoder is configured to input the vector into the FFNN model in the series of FFNN layers to obtain an output, and input the output of the FFNN model into the Softmax function, and the Softmax function is configured to obtain the probability of the each sentiment score and determine the sentiment score for the input text based on the probability (i.e. The attention score for each encoded vector is calculated by normalizing the vectors via softmax function 1220 and then multiplying the output by its weight, which is fed to each layer of the decoder RNN. Paragraph 0092)
Regarding claim 10; Claim 10 contains substantially the same subject matter as claim 1, Therefore claim 10, is rejected on the same grounds as claim 1.
Regarding claim 11; Claim 11 contains substantially the same subject matter as claim 2, Therefore claim 11, is rejected on the same grounds as claim 2.
Regarding claim 12; Claim 12 contains substantially the same subject matter as claim 3, Therefore claim 12, is rejected on the same grounds as claim 3.
Regarding claim 13; Claim 13 contains substantially the same subject matter as claim 4, Therefore claim 13, is rejected on the same grounds as claim 4.
Regarding claim 14; Claim 14 contains substantially the same subject matter as claim 5, Therefore claim 14, is rejected on the same grounds as claim 5.
Regarding claim 15; Claim 15 contains substantially the same subject matter as claim 6, Therefore claim 15, is rejected on the same grounds as claim 6.
Regarding claim 16; Claim 16 contains substantially the same subject matter as claim 7, Therefore claim 16, is rejected on the same grounds as claim 7.
Regarding claim 17; Claim 17 contains substantially the same subject matter as claim 8, Therefore claim 17, is rejected on the same grounds as claim 8.
Regarding claim 18; Claim 18 contains substantially the same subject matter as claim 9, Therefore claim 18, is rejected on the same grounds as claim 9.
Regarding claim 19; DeFelice ‘139 discloses data processing apparatus (Fig. 2, The Primary Components of Generative System 200) configured to perform the method of claim 10 (i.e. The primary components of the system are the prospect modeling component 210 and the text generation component 220. Together with the campaign component 230, the entire system 200 can be can be considered as a specialized source-to-source translation engine for taking a source text 201 as an input 221 into the text generation component 220 and rendering a “translation” as generated text 203 that may or may not differ in output language but intentionally varies in sentiment and content according to a continuously-refined model of a possible reader as predicted by the prospect modeling component 210. Paragraph 0045).
Regarding claim 20; Bellegarda ‘369 discloses a computer program element comprising program instructions, which, when executed by one or more processors, cause the one or more processors to perform the method of any one of claim 10 (i.e. The computer-readable storage mediums are, for example, tangible and non-transitory. Memory 202 includes high-speed random access memory and also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Memory controller 222 controls access to memory 202 by other components of device 200.A non-transitory computer-readable storage medium of memory 202 is used to store instructions for performing aspects of processes. Paragraphs 0052-0053).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARCUS T. RILEY, ESQ. whose telephone number is (571)270-1581. The examiner can normally be reached 9-5 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Hai Phan can be reached at 571-272-6338. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MARCUS T. RILEY, ESQ.
Primary Examiner
Art Unit 2654
/MARCUS T RILEY/Primary Examiner, Art Unit 2654