DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Should applicant desire to obtain the benefit of foreign priority under 35 U.S.C. 119(a)-(d) prior to declaration of an interference, a certified English translation of the foreign application must be submitted in reply to this action. 37 CFR 41.154(b) and 41.202(e).
Failure to provide a certified translation may result in no benefit being accorded for the non-English application.
Claim Objections
Claims 7 and 18 are objected to because of the following informalities: The claims recite a integer variable N with a value greater than 2, and a dependent integer variable i, with a value greater than 1 and less than N-1. For the instance of N=3, i cannot simultaneously be greater than 1 and less than 2 (3-1).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite a mental process that can be performed in the human mind or with the aid of pen and paper. This judicial exception is not integrated into a practical application because a computer is invoked merely as a tool to execute an abstract idea. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because an abstract idea is merely applied on a generic computer without any element that would otherwise preclude performance of the abstrac.
Regarding claim 1, the claim recites “A method for sentiment analysis, comprising:obtaining a feature sequence corresponding to a text, wherein the feature sequence comprises encoded features;processing, by using an attention mechanism, each of encoded features in the feature sequence, to obtain an attention feature of the text;transferring the attention feature to a spatial transform feature of the text; andrecognizing an entity attribute of the spatial transform feature, and performing sentiment mapping based on the spatial transform feature to obtain a sentiment polarity of the entity attribute.”
The limitations as drafted cover mental activities which can be performed in the mind or with the aid of pen and paper. Taken individually, or as a whole, these limitations describe acts which are equivalent to human mental work of reading a text and identifying opinions of the writer.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be performed mentally, and no additional features in the claims would preclude them from being performed as such. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 2, the claim depends from claim 1, and thus recites the limitations of claim 1, “wherein obtaining the feature sequence corresponding to the text comprises:encoding each of tokens in the text, to obtain an encoded feature of the token;encoding a sentiment classification identifier in the text, to obtain an encoded feature of the sentiment classification identifier; andgenerating, based on the encoded features of the tokens and the encoded feature of the sentiment classification identifier, the feature sequence of the text;wherein the encoded feature of the sentiment classification identifier is at a start position in the feature sequence, and an order of the encoded features of the tokens in the feature sequence is same as an order of the tokens in the text.”
Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of transcribing a written text. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 3, the claim depends from claim 2, and thus recites the limitations of claims 1 and 2, “wherein encoding each of the tokens in the text, to obtain the encoded feature of the token comprises:performing following processing on each of the tokens:vectorizing the token to obtain a word vector corresponding to the token;performing position encoding on the token according to a position of the token in the text, to obtain a position vector of the token;determining an attribution vector of the token according to a sentence to which the token belongs in the text; andobtaining the encoded feature of the token according to the word vector of the token, the position vector of the token and the attribution vector of the token.”
Taken individually, or as a whole with the preceding claims, these limitations describe acts which are equivalent to human mental work of transcribing a written text. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 4, the claim depends from claim 1, and thus recites the limitations of claim 1, “wherein processing, by using the attention mechanism, the each of the encoded features in the feature sequence, to obtain the attention feature of the text comprises:performing linear transform on the each of the encoded features in the feature sequence, to obtain a query feature, a key feature and a value feature that correspond to the each of the encoded features;performing association processing based on the query feature and the key feature corresponding to the each of the encoded features, to obtain an attention weight of the each of the encoded features; andperforming a weighted sum of the value features of the encoded features based on the attention weights of the encoded features, to obtain the attention feature of the text.”
Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of transcribing text and performing calculations. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 5, the claim depends from claim 4, and thus recites the limitations of claims 1 and 4, “wherein performing association processing based on the query feature and the key feature corresponding to the each of the encoded features, to obtain the attention weight of the each of the encoded feature comprises:performing following processing on the each of the encoded features:performing association processing based on the query feature and the key feature corresponding to the each of the encoded features to obtain a first influence factor of the encoded feature, wherein the first influence factor is positively correlated to the query feature and a transposition of the key feature, respectively, and the first influence factor is negatively correlated to a length of the encoded feature;obtaining a maximum prediction length of the feature sequence and a scaling coefficient for controlling a scaling degree of the maximum prediction length, and determining a second influence factor of the encoded feature based on the scaling coefficient, the maximum prediction length and the first influence factor; andperforming maximum likelihood processing based on the first influence factor and the second influence factor, to obtain the attention weight of the encoded feature.”
Taken individually, or as a whole with the preceding claims, these limitations describe acts which are equivalent to human mental work of performing calculations. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 6, the claim depends from claim 1, and thus recites the limitations of claim 1, “wherein transferring the attention feature to the spatial transform feature of the text comprises:obtaining at least one spatial transform layer; andtransferring, through the at least one spatial transform layer, the attention feature to the spatial transform feature of the text.”
Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of performing calculations. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 7, the claim depends from claim 6, and thus recites the limitations of claims 1 and 6, “wherein each of the at least one spatial transform layer comprises a linear transform layer and a nonlinear transform layer, and transferring, through the at least one spatial transform layer, the attention feature to the spatial transform feature of the text comprises:in a case that a number of the at least one spatial transform layer is N, performing linear transform on the attention feature through a linear transform layer in a first spatial transform layer to obtain a first intermediate transform feature, and performing nonlinear mapping on the first intermediate transform feature through a nonlinear transform layer in the first spatial transform layer to obtain a first spatial transform feature;performing, through a linear transform layer in an i-th spatial transform layer, linear transform on an (i-1)-th spatial transform feature output by an (i-1)-th spatial transform layer to obtain an i-th intermediate transform feature, and performing, through a nonlinear transform layer in the i-th spatial transform layer, nonlinear mapping on the i-th intermediate transform feature to obtain an i-th spatial transform feature; and determining an N-th spatial transform feature output by an N-th spatial transform layer as the spatial transform feature of the text;wherein N is a positive integer greater than 2, and i is a positive integer greater than 1 and smaller than N-1.”
Taken individually, or as a whole with the preceding claims, these limitations describe acts which are equivalent to human mental work of performing calculations. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 8, the claim depends from claim 1, and thus recites the limitations of claim 1, “wherein recognizing an entity attribute of the spatial transform feature comprises:performing entity attribute mapping on the spatial transform feature to obtain a corresponding first mapping feature; andoffsetting the corresponding first mapping feature to obtain the entity attribute corresponding to the text.”
Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of identifying opinions. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 9, the claim depends from claim 8, and thus recites the limitations of claims 1 and 8, “wherein performing sentiment mapping based on the spatial transform feature to obtain the sentiment polarity of the entity attribute comprises:performing sentiment polarity mapping on the spatial transform feature to obtain a corresponding second mapping feature;offsetting the corresponding second mapping feature to obtain a sentiment classification corresponding to the text; andperforming sentiment mapping on the entity attribute based on the sentiment classification, to obtain the sentiment polarity of the entity attribute.”
Taken individually, or as a whole with the preceding claims, these limitations describe acts which are equivalent to human mental work of transcribing text and identifying opinions. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 10, the claim depends from claim 1, and thus recites the limitations of claim 1, “wherein the method for sentiment analysis is implemented by calling a sentiment analysis model, the sentiment analysis model comprises:a feature extraction layer, an attention layer, a spatial transform layer, an entity recognition layer and a sentiment classification layer, and the method further comprises:extracting, through the feature extraction layer, a feature sequence corresponding to a training sample, wherein the feature sequence comprises encoded features, and the training sample is labeled with an entity attribute tag and a sentiment classification tag;processing, through the attention layer by using an attention mechanism, each of the encoded features in the feature sequence corresponding to the training sample, to obtain an attention feature corresponding to the training sample;transferring, through the spatial transform layer, the attention feature corresponding to the training sample to a spatial transform feature of the training sample;recognizing, through the entity recognition layer, entity attribute of the spatial transform feature of the training sample to obtain an entity attribute recognition result corresponding to the training sample, and performing, through the sentiment classification layer, sentiment mapping based on the spatial transform feature of the training sample to obtain a sentiment classification result corresponding to the training sample; andupdating model parameters of the sentiment analysis model based on the entity attribute recognition result and the entity attribute tag which correspond to the training sample, and based on the sentiment classification result and the sentiment classification tag which correspond to the training sample.”
The limitations of “extracting…” “processing…” “transferring…” and “recognizing…” as drafted covers mental activities which can be performed in the mind or with the aid of pen and paper.
The recitation of model layers and the limitation of “updating model parameters…” describe features that are well-understood and readily available to a person having ordinary skill in machine learning.
Taken individually, or as a whole with claim 1, these limitations describe acts which are equivalent to human mental work of reading text and performing calculations, implemented on a generic computer. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claim 11, the claim depends from claim 10, and thus recites the limitations of claims 1 and 10, “wherein updating the model parameters of the sentiment analysis model based on the entity attribute recognition result and the entity attribute tag which correspond to the training sample, and based on the sentiment classification result and the sentiment classification tag which correspond to the training sample comprises:performing one-hot encoding on the entity attribute tag and the sentiment classification tag corresponding to the training sample, respectively, to obtain an encoded feature of the entity attribute tag and an encoded feature of the sentiment classification tag that correspond to the training sample;determining a first loss function of the entity recognition layer according to the entity attribute recognition result and the encoded feature of the entity attribute tag;determining a second loss function of the sentiment classification layer according to the sentiment classification result and the encoded feature of the sentiment classification tag; anddetermining a third loss function of the sentiment analysis model according to the first loss function and the second loss function, and updating the model parameters of the sentiment analysis model based on the third loss function.”
The limitations of “performing one-hot encoding…” “determining a [first/second/third] loss function…” and “updating the model parameters…” as drafted covers features that are well-understood and readily available to a person having ordinary skill in the art of machine learning.
Taken individually, or as a whole with the preceding claims, these limitations describe acts which are equivalent to the human mental activity of performing calculation, implemented on a generic computer. Accordingly, the claim is directed to an abstract idea without significantly more. The claim is not patent eligible.
Regarding claims 12-19, system claims 12-19 and method claims 1-8 are related as a method and system of using the same, with each system element’s function corresponding to the method step. Accordingly, claims 12-19 are similarly rejected under the same rationale as applied to claims 1-8.
Regarding claim 20, computer-readable medium claim 20 and method claim 1 are related as method and computer-readable medium for performing the same, with each computer-readable medium element’s function corresponding to the method step. Accordingly, claim 20 is similarly rejected under the same rationale as applied to claim 1.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 6, 8-9, 12, 17 and 19-20 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by U.S. Patent Application Publication 2020/0159863 to Wang et al. (hereinafter, "Wang").
Regarding claims 1, 12 and 20, Wang teaches a method, system and computer readable medium comprising: obtaining a feature sequence corresponding to a text, wherein the feature sequence comprises encoded features (paragraph [0033], "In some examples, the input data includes sentences. In some examples, a sentence can be dentoed [sic] as a sequence of tokens (words)
s
i
=
{
w
i
1
,
w
i
2
,
…
,
w
i
n
}
, and can be represented as a
D
×
n
i
matrix
X
i
=
[
x
i
1
,
…
,
x
i
n
]
, where
x
i
j
∈
R
D
is a feature vector for the j-th token of the sentence.");
processing, by using an attention mechanism, each of encoded features in the feature sequence, to obtain an attention feature of the text (paragraph [0038], "In some implementations, a MNCA includes, for each sentence, constructing a pair of attentions. In some examples, an aspect attention is provided for aspect term extraction, and an opinion attention is provided for opinion term extraction. Each of the attentions aims to learn a general prototype vector, a token-level feature vector, and a token-level attention score for each word in the sentence.");
transferring the attention feature to a spatial transform feature of the text (paragraph [0050], "By updating the prototype vector ut+1a with extracted information from the tth layer, the following is provided:
u
t
+
1
a
=
tanh
Q
a
u
t
a
+
Σ
j
a
t
a
h
j
where highly interactive hj contributes more to the prototype updates. Since the final feature representation rT,ja for each word is generated from the above tensor interactions, it transforms the normal feature space hj to interaction space rT,j, compared to simple RNNs that only computes hj"); and
recognizing an entity attribute of the spatial transform feature (paragraph [0052], "In some implementations, the multi-task memory network includes: a category-specific MNCA to co-extract aspect and opinion terms for each category…"), and
performing sentiment mapping based on the spatial transform feature to obtain a sentiment polarity of the entity attribute (paragraph [0039], "In some examples, the MNCA captures direct relations between aspect terms, and opinion terms," and paragraph [0065], "The input data is processed by the MTMN (708). A set of aspect terms and a set of opinion terms with respective categories are output (710).").
Hereinafter, note that Wang’s teaching of “aspect terms” and “opinion terms,” denoted by element superscripts a and p, are read as analogous to the claimed entity attributes and sentiments, respectively.
Regarding claims 6 and 17, Wang teaches a method and system wherein transferring the attention feature to the spatial transform feature of the text comprises:
obtaining at least one spatial transform layer (paragraph [0050], "In accordance with implementations of the present disclosure, the proposed memory network is able to attend to relevant words that are highly interactive given the prototypes. This is achieved by tensor interactions, for example,
h
j
T
G
a
u
t
a
between jth word and the aspect prototype."); and
transferring, through the at least one spatial transform layer, the attention feature to the spatial transform feature of the text (paragraph [0050], "Since the final feature representation rT,ja for each word is generated from the above tensor interactions, it transforms the normal feature space hj to interaction space rT,j, compared to simple RNNs that only computes hj").
Regarding claims 8 and 19, Wang teaches a method and system wherein recognizing an entity attribute of the spatial transform feature comprises:
performing entity attribute mapping on the spatial transform feature to obtain a corresponding first mapping feature (paragraph [0043], "To obtain
r
j
a
, a composition vector
β
j
a
∈
R
k
that encodes the extent of correlations between hj and the prototype vector ua through a tensor operator is computed."); and
offsetting the corresponding first mapping feature to obtain the entity attribute corresponding to the text (paragraph [0045], "An attention score
α
j
a
for token wj is computed… Since
r
j
a
is a correlation fearure vector,
v
a
∈
R
k
can be deemed as a weight vector that weighs each feature accordingly. In this manner,
α
j
a
becomes the normalized score, where a higher score indicates a higher correlation with the prototype, and a higher change of being attended."). The claimed “offsetting” is read as analogous to Wang’s weighting of features, wherein preferred features are offset by combining a weighting vector.
Regarding claim 9, Wang teaches the method of claim 8, wherein performing sentiment mapping based on the spatial transform feature to obtain the sentiment polarity of the entity attribute comprises:
performing sentiment polarity mapping on the spatial transform feature to obtain a corresponding second mapping feature (paragraph [0043], "To obtain
r
j
a
, a composition vector
β
j
a
∈
R
k
that encodes the extent of correlations between hj and the prototype vector ua through a tensor operator is computed." Paragraph [0045], "The procedure for opinion attention is similar. In the subsequent sections, a superscript p is used to denote the opinion attention.");
offsetting the corresponding second mapping feature to obtain a sentiment classification corresponding to the text (paragraph [0045], "An attention score
α
j
a
for token wj is computed… Since
r
j
a
is a correlation fearure vector,
v
a
∈
R
k
can be deemed as a weight vector that weighs each feature accordingly. In this manner,
α
j
a
becomes the normalized score, where a higher score indicates a higher correlation with the prototype, and a higher change of being attended." Paragraph [0045], "The procedure for opinion attention is similar. In the subsequent sections, a superscript p is used to denote the opinion attention."); and
performing sentiment mapping on the entity attribute based on the sentiment classification, to obtain the sentiment polarity of the entity attribute (paragraph [0047], "The composition vectors
β
j
a
and
β
j
p
are computed… where [:] denotes concatenation of two vectors. Intuitively, Ga or Dp is to capture the K syntactic relations within aspect terms or opinion terms themselves, while Gp and Da are to capture syntactic relations between aspect terms and opinion terms for dual propagation."). Wang discloses, at paragraph [0045], that functions regarding aspects and opinions are differentiated by the superscripts a and p, respectively, and thus the teachings of aspect mapping similarly teach opinion mapping. The claimed “offsetting” is read as analogous to Wang’s weighting of features, wherein preferred features are offset by combining a weighting vector.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-3 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of U.S. Patent Application Publication 2023/0229847 to Rosedale (hereinafter, "Rosedale").
Regarding claims 2 and 13, Wang teaches a method and system wherein obtaining the feature sequence corresponding to the text comprises:
encoding each of tokens in the text, to obtain an encoded feature of the token (paragraph [0033], "In some examples, the input data includes sentences. In some examples, a sentence can be dentoed [sic] as a sequence of tokens (words)
s
i
=
{
w
i
1
,
w
i
2
,
…
,
w
i
n
}
, and can be represented as a
D
×
n
i
matrix
X
i
=
[
x
i
1
,
…
,
x
i
n
]
, where
x
i
j
∈
R
D
is a feature vector for the j-th token of the sentence.");
encoding a sentiment classification identifier in the text, to obtain an encoded feature of the sentiment classification identifier (paragraph [0043], "In the aspect attention, a prototype vector ua is generated, which can be viewed as a general feature representation for aspect terms." Paragraph [0045], "The procedure for opinion attention is similar. In the subsequent sections, a superscript p is used to denote the opinion attention."); and
generating, based on the encoded features of the tokens and the encoded feature of the sentiment classification identifier, the feature sequence of the text (paragraph [0054], "The overall representations of the sentence for category c in terms of aspects and opinions, denoted by
o
c
a
and
o
c
p
, respectively, are computed using (6), which will be further used to produce the prototype vectors
u
c
,
t
+
1
a
,
u
c
,
t
+
1
p
in the next layer using (5)."); and
wherein the encoded feature of the sentiment classification identifier is at a start position in the feature sequence (paragraph [0005], "each token-level label comprises one of beginning of an aspect, inside of an aspect, beginning of an opinion, inside of an opinion, and none;").
Wang is silent to the order of its tokens and features, and thus, Rosedale is introduced. Rosedale teaches an order of the encoded features of the tokens in the feature sequence is same as an order of the tokens in the text (paragraph [0018], "Accordingly, the present invention also provides for computer-implemented methods executed by one or more processors for representing a document temporally in an arbitrary well-defined semantic space, the method comprising: generating, by a text parser operating upon a plaintext document, an ordered list of tokens, wherein the order of tokens in the list corresponds precisely to their order in the document, and each token in the list has content in the form of a string and index in the form of a non-negative integer;").
Wang and Rosedale are considered analogous because they are each concerned with natural language processing. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Wang with the teachings of Rosedale given that one of ordinary skill in the art could have pursued the finite potential arrangements of tokens with a reasonable expectation of success. A person of ordinary skill has good reason to pursue the known options within the field of the invention. If this leads to the anticipated success, it is likely that product is not of innovation but of ordinary skill and common sense.
Regarding claims 3 and 14, Rosedale further teaches a method and system wherein encoding each of the tokens in the text, to obtain the encoded feature of the token comprises:
performing following processing on each of the tokens:
vectorizing the token to obtain a word vector corresponding to the token (paragraph [0023], "The present invention also provides for computer-implemented methods executed by one or more processors for representing a document temporally in an arbitrary well-defined semantic space, the methods comprising: generating, by a text parser operating upon a plaintext document, an ordered list of tokens, wherein the order of tokens in the list matches their order in the document,");
performing position encoding on the token according to a position of the token in the text, to obtain a position vector of the token (paragraph [0023], "each token in the list has content in the form of a string and index in the form of a non-negative integer;");
determining an attribution vector of the token according to a sentence to which the token belongs in the text (paragraph [0023], "defining, by a computational process, a frame object with attributes start, end, and size, and methods for retrieving start, end, and size attributes and altering start and end attributes;"); and
obtaining the encoded feature of the token according to the word vector of the token, the position vector of the token and the attribution vector of the token (paragraph [0023], "iteratively generating, by an increment generator using the frame object and the output of the text parser, a collection of increment objects possessing identical docID attribute values; and generating, from the collection of increment objects, a Root Document Trace," and paragraph [009], "The present invention provides, fundamentally, methods for representing a document as a bijection, henceforth referred to as a 'Root Document Trace'. Representing a document, especially a full text document, as a Root Document Trace has a variety of uses, for example, evaluating narrative and argument structures within and between documents and generating higher-order mathematical document representations quickly and efficiently.").
Wang and Rosedale are considered analogous because they are each concerned with natural language processing. Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Wang with the teachings of Rosedale for the purpose of improving document processing efficiency. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Claims 4 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of WIPO Publication 2023/226783 to Cai et al. (hereinafter, "Cai").
Regarding claims 4 and 15, Wang does not explicitly teach a method or system “wherein processing, by using the attention mechanism, the each of the encoded features in the feature sequence, to obtain the attention feature of the text comprises: performing linear transform on the each of the encoded features in the feature sequence, to obtain a query feature, a key feature and a value feature that correspond to the each of the encoded features; performing association processing based on the query feature and the key feature corresponding to the each of the encoded features, to obtain an attention weight of the each of the encoded features; and performing a weighted sum of the value features of the encoded features based on the attention weights of the encoded features, to obtain the attention feature of the text,” and thus, Cai is introduced.
Cai teaches performing linear transform on the each of the encoded features in the feature sequence, to obtain a query feature, a key feature and a value feature that correspond to the each of the encoded features (page 11, "Specifically, as shown in Figure 6, the independent superimposed attention network determines the query vector (Query, Q),key value vector (Key, K) and value vector (Value, V) based on the input data, and then based on Q, K , V carries axis 1square Calculate the attention in the direction of ~ axis N to obtain the feature vector of each element corresponding to axis 1~ axis N, and then perform a weighted sum of the feature vectors of each element corresponding to axis 1 ~ axis N to obtain the feature vector of each element.");
performing association processing based on the query feature and the key feature corresponding to the each of the encoded features, to obtain an attention weight of the each of the encoded features (page 11, "Specifically, as shown in Figure 6, the independent superimposed attention network determines the query vector (Query, Q),key value vector (Key, K) and value vector (Value, V) based on the input data, and then based on Q, K , V carries axis 1square Calculate the attention in the direction of ~ axis N to obtain the feature vector of each element corresponding to axis 1~ axis N, and then perform a weighted sum of the feature vectors of each element corresponding to axis 1 ~ axis N to obtain the feature vector of each element."); and
performing a weighted sum of the value features of the encoded features based on the attention weights of the encoded features, to obtain the attention feature of the text (page 11, "Specifically, as shown in Figure 6, the independent superimposed attention network determines the query vector (Query, Q),key value vector (Key, K) and value vector (Value, V) based on the input data, and then based on Q, K , V carries axis 1square Calculate the attention in the direction of ~ axis N to obtain the feature vector of each element corresponding to axis 1~ axis N, and then perform a weighted sum of the feature vectors of each element corresponding to axis 1 ~ axis N to obtain the feature vector of each element.").
Wang and Cai are considered analogous because they are each concerned with natural language processing (see Cai page 2, "For example, this application can be applied to computer vision or natural language processing. Including machine translation, automatic summary generation, opinion extraction, text classification, question answering, text semantic comparison, speech recognition, image recognition (Image Classification), object detection (Object Detection), semantic segmentation (Semantic Segmentation) and image generation (Image Generation)." and page 8, "Self-attention mechanism can play a key role in machine reading, abstract summary or image description generation. Taking the application of self-attention network to natural language processing as an example, the self-attention network processes input data of any length and generates new feature expressions of the input data, and then converts the feature expressions into target words."). Therefore, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to have modified Wang with the teachings of Cai for the purpose of improving system performance. Given that all the claimed elements were known in the prior art, one skilled in the art could have combined the elements by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Allowable Subject Matter
Claims 5, 7, objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Patent 12,093,646 to Miao et al.
U.S. Paten Application Publication 2020/0073937 to Zhao et al.
U.S. Patent Application Publication 2021/0312135 to Meng et al.
China Invention Application 114791950 to Kong et al.
China Invention Application 114781390 to Lu et al.
China Invention Application 116361462 to Yang et al.
China Invention Application 110781273 to Jiang et al.
China Invention Application 105975594 to Xu et al.
China Invention Application 116738999 to Zhang et al.
China Patent 112860894 to Li.
China Patent 107092596 to Zhang et al.
Korean Invention Application 2019-0008514 to Akerib.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SEAN T SMITH whose telephone number is (571)272-6643. The examiner can normally be reached Monday - Friday 8:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, PIERRE-LOUIS DESIR can be reached at (571) 272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SEAN THOMAS SMITH/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659