DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is responsive to: Amendment filed 30 Mar. 2026
Claims 1-20 are pending in this case. Claims 1, 7, 12, 17, 18, 19 and 20 are independent claims
Applicant’s Respons8
In Applicant’s Response dated 30 Mar. 2026, Applicant amended claims 1, 7, 12 and 16; argued against all rejections previously set forth in the Office Action dated 29 Dec. 2025.
Allowable Subject Matter
Claims 8 and 9 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 10-15, 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zheng et al. (Pub. No.: US 2021/0174024 A1; Effectively Filed: Dec. 7, 2018) (hereinafter “Zheng”) in view of Nair et al. (Pub. No.: US 2022/0101837 A1; Filed: Sep. 25, 2020) (hereinafter “Nair”).
Regarding independent claim 1, Zheng disclose a text processing method, comprising:
filtering one or more target words with highest first attention scores from words in a piece of text using a first attention layer of a text processing model, wherein the text is an input text (0010; 0019-0026; 0109-0110; 0172; 0189; 0226; 0246);
calculating second attention scores of the target words using a second attention layer of the text processing model (0109-0110; 0172; 0226; 0246); and
obtaining a processing result of the text from the processing model based on the second attention scores of the target words (0109-0110; 0172; 0226; 0246).
Regarding dependent claim 2, Zheng disclose the text processing method according to claim 1, wherein the filtering one or more target words with the highest first attention scores from the words in the piece of text using the first attention layer of the text processing model comprises:
determining one or more words with the highest first attention scores as the target words (0109-0110; 0172; 0226; 0246).
Zheng does not expressly disclose performing a dimensionality reduction processing on the words in the text;
calculating the first attention scores of the dimensionality- reduced words in the text using the first attention layer of the text processing model.
Nair teaches performing a dimensionality reduction processing on the words in the text (0055-0056);
calculating the first attention scores of the dimensionality- reduced words in the text using the first attention layer of the text processing model (0055-0056).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Nair with Zheng for the benefit of generating a plurality of reduced-dimensionality vectors by reducing a dimensionality of the plurality of multi-dimensional vectors (0004).
Regarding dependent claim 3, Zheng in view of Nair disclose the text processing method according to claim 2, wherein the dimensionality of the dimensionality-reduced words is less than 512 (0055).
Regarding dependent claim 4, Zheng in view of Nair disclose the text processing method according to claim 3, wherein the dimensionality of the dimensionality-reduced words is 64. (0055).
Regarding dependent claim 10, Zheng disclose the training method according to claim 7, wherein a number of the target words is equal to a first parameter, and the training method further comprises:
determining a number of the filtered target words based on a sum of the first attention scores of the target words (0109-0110; 0172; 0226; 0246).
Regarding dependent claim 11, Zheng in view of Nair disclose the training method according to claim 10, wherein the determining the number of the filtered target words based on the sum of the first attention scores of the target words comprises:
reducing the number of the filtered target words in response to the sum of the first attention scores of the target words being not less than a score threshold (0055).
Regarding independent claim 12, Zheng disclose a text processing device, comprising:
a memory (0254); and
a processor coupled to the memory, the processor configured to, based on instructions stored in the memory, carry out a text processing method comprising (0254):
filtering one or more target words with highest first attention scores from words in a piece of text using a first attention layer of a text processing model, wherein the text is an input text (0010; 0019-0026; 0109-0110; 0172; 0189; 0226; 0246);
calculating second attention scores of the target words using a second attention layer of the text processing model (0109-0110; 0172; 0226; 0246); and
obtaining a processing result of the text from the processing model based on the second attention scores of the target words (0109-0110; 0172; 0226; 0246).
Regarding dependent claim 13, Zheng disclose the text processing device according to claim 12, wherein the processor is further configured to:
determine one or more words with the highest first attention scores as the target words (0109-0110; 0172; 0226; 0246).
Zheng does not expressly disclose perform a dimensionality reduction processing on the words in the text;
calculate the first attention scores of the dimensionality- reduced words in the text using the first attention layer of the text processing model.
Nair teaches perform a dimensionality reduction processing on the words in the text (0055-0056);
calculate the first attention scores of the dimensionality- reduced words in the text using the first attention layer of the text processing model (0055-0056).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Nair with Zheng for the benefit of generating a plurality of reduced-dimensionality vectors by reducing a dimensionality of the plurality of multi-dimensional vectors (0004).
Regarding dependent claim 14, Zheng in view of Nair disclose the text processing device according to claim 13, wherein the dimensionality of the dimensionality-reduced words is less than 512 (0055).
Regarding dependent claim 15, Zheng in view of Nair disclose the text processing device according to claim 14, wherein the dimensionality of the dimensionality-reduced words is 64 (0055).
Regarding independent claim 17, Zheng a training device, comprising:
a memory (0254); and
a processor coupled to the memory, the processor configured to, based on instructions stored in the memory, carry out the training method according to claim 7 (0254).
Regarding independent claim 19, Zheng disclose a non-transitory computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the text processing method according to claim 1.
Claims 5-7, 16, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zheng in view of Bhusan et al. (Pub. No.: US 2023/0067976 A1; Filed: Jul. 27, 2021) (hereinafter “Bhusan”).
Regarding dependent claim 5, Zheng does not expressly disclose the text processing method according to claim 1, wherein the text processing model is a neural network model comprising Transformer.
Bhusan teaches wherein the text processing model is a neural network model comprising Transformer (0153).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Bhusan with Zheng for the benefit of generating a plurality of reduced-dimensionality vectors by reducing a dimensionality of the plurality of multi-dimensional vectors (0004).
Regarding dependent claim 6, Zheng does not expressly disclose the text processing method according to claim 1, wherein the text processing comprises at least one of text translation, text classification or text matching.
Bhusan teaches wherein the text processing comprises at least one of text translation, text classification or text matching (0006-0008; 0038; 0042; 0057; 0088-0090).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Bhusan with Zheng for the benefit of improving the performance of the models in the final tasks (0004).
Regarding independent claim 7, Zheng disclose a training method for text processing, comprising:
filtering one or more target words with highest first attention scores from words in a piece of training text using a first attention layer of a text processing model, wherein the training text is an input text (0010; 0019-0026; 0109-0110; 0172; 0189; 0226; 0246);
calculating second attention scores of the target words using a second attention layer of the text processing model (0109-0110; 0172; 0226; 0246);
obtaining a processing result of the training text from the text processing model based on the second attention scores of the target words (0109-0110; 0172; 0226; 0246); and
Zheng does not expressly disclose training the text processing model based on the processing result of the training text and annotation information of the text.
Bhusan teaches training the text processing model based on the processing result of the training text and annotation information of the text (0116; 0118; 0123; 0143).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Bhusan with Zheng for the benefit of improving the performance of the models in the final tasks (0004).
Regarding dependent claim 16, Zheng does not expressly disclose the text processing device according to claim 12, wherein the text processing model is a neural network model comprising Transformer.
Bhusan teaches wherein the text processing model is a neural network model comprising Transformer (0153).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Bhusan with Zheng for the benefit of improving the performance of the models in the final tasks (0004).
Regarding independent claim 18, Zheng disclose a text processing system, comprising:
the text processing device according to claim 12; and the training device, comprising:
a memory (0254); and
a processor coupled to the memory, the processor configured to, based on instructions stored in the memory, carry out a training method comprising (0254):
filtering one or more target words with highest first attention scores from words in a piece of training text using a first attention layer of a text processing model, wherein the training text is an input text (0010; 0019-0026; 0109-0110; 0172; 0189; 0226; 0246);
calculating second attention scores of the target words using a second attention layer of the text processing model (0109-0110; 0172; 0226; 0246);
obtaining a processing result of the training text from the text processing model based on the second attention scores of the target words (0109-0110; 0172; 0226; 0246); and
Zheng does not expressly disclose training the text processing model based on the processing result of the training text and annotation information of the text.
Bhusan teaches training the text processing model based on the processing result of the training text and annotation information of the text (0116; 0118; 0123; 0143).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Bhusan with Zheng for the benefit of improving the performance of the models in the final tasks (0004).
Regarding independent claim 20, Zheng disclose a non-transitory computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the training method according to claim 7.
NOTE
It is noted that any citations to specific, pages, columns, lines, or figures in the prior art references and any interpretation of the reference should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. See MPEP 2123.
Response to Arguments
Applicant’s arguments have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES J DEBROW whose telephone number is (571)272-5768. The examiner can normally be reached on 09:00 - 06:00.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Bashore can be reached on 571-272-4088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center or Private PAIR to authorized users only. Should you have questions about access to Patent Center or the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/James J Debrow/
Primary Patent Examiner
Art Unit 2174
571-272-5768
Prosecution Insights