DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This Office Action is responsive to: Application filed 09 Feb. 2024
Claims 1-20 are pending in this case. Claims 1, 7, 12, 17, 18, 19 and 20 are independent claims
Allowable Subject Matter
Claims 8 and 9 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 12 and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Tan et al. (Pub. No.: US 2021/0303606 A1; Filed: Jan. 24, 2019) (hereinafter “Tan”)
Regarding independent claim 1,Tan disclose a text processing method, comprising:
filtering one or more target words with highest first attention scores from words in a piece of text using a first attention layer of a text processing model (0102-0104; 0111; 0126; 0133);
calculating second attention scores of the target words using a second attention layer of the text processing model (0099; 0104; 0111; 0130-0132); and
obtaining a processing result of the text from the processing model based on the second attention scores of the target words (0104; 0111; 0130-0132).
Regarding independent claim 12, Tan disclose a text processing device, comprising:
a memory (0007); and
a processor coupled to the memory, the processor configured to, based on instructions stored in the memory, carry out a text processing method comprising (0007):
filtering one or more target words with highest first attention scores from words in a piece of text using a first attention layer of a text processing model (0102-0104; 0111; 0126; 0133);
calculating second attention scores of the target words using a second attention layer of the text processing model (0099; 0104; 0111; 0130-0132); and
obtaining a processing result of the text from the processing model based on the second attention scores of the target words (0104; 0111; 0130-0132).
Regarding independent claim 19, Tan disclose a non-transitory computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the text processing method according to claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2-4, 10, 11, 13-15 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Tan in view of Nair et al. (Pub. No.: US 2022/0101837 A1; Filed: Sep. 25, 2020) (hereinafter “Nair”).
Regarding dependent claim 2, Tan disclose the text processing method according to claim 1, wherein the filtering one or more target words with the highest first attention scores from the words in the piece of text using the first attention layer of the text processing model comprises:
determining one or more words with the highest first attention scores as the target words (0102-0104; 0111; 0126; 0133).
Tan does not expressly disclose performing a dimensionality reduction processing on the words in the text;
calculating the first attention scores of the dimensionality- reduced words in the text using the first attention layer of the text processing model.
Nair teaches performing a dimensionality reduction processing on the words in the text (0055-0056);
calculating the first attention scores of the dimensionality- reduced words in the text using the first attention layer of the text processing model (0055-0056).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Nair with Tan for the benefit of generating a plurality of reduced-dimensionality vectors by reducing a dimensionality of the plurality of multi-dimensional vectors (0004).
Regarding dependent claim 3, Tan in view of Nair disclose the text processing method according to claim 2, wherein the dimensionality of the dimensionality-reduced words is less than 512 (0055).
Regarding dependent claim 4, Tan in view of Nair disclose the text processing method according to claim 3, wherein the dimensionality of the dimensionality-reduced words is 64. (0055).
Regarding dependent claim 10, Tan disclose the training method according to claim 7, wherein a number of the target words is equal to a first parameter, and the training method further comprises:
determining a number of the filtered target words based on a sum of the first attention scores of the target words (0097-0098; 0111; 0126).
Regarding dependent claim 11, Tan in view of Nair disclose the training method according to claim 10, wherein the determining the number of the filtered target words based on the sum of the first attention scores of the target words comprises:
reducing the number of the filtered target words in response to the sum of the first attention scores of the target words being not less than a score threshold (0055).
Regarding dependent claim 13, Tan disclose the text processing device according to claim 12, wherein the processor is further configured to:
determine one or more words with the highest first attention scores as the target words (0102-0104; 0111; 0126; 0133).
Tan does not expressly disclose perform a dimensionality reduction processing on the words in the text;
calculate the first attention scores of the dimensionality- reduced words in the text using the first attention layer of the text processing model.
Nair teaches perform a dimensionality reduction processing on the words in the text (0055-0056);
calculate the first attention scores of the dimensionality- reduced words in the text using the first attention layer of the text processing model (0055-0056).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Nair with Tan for the benefit of generating a plurality of reduced-dimensionality vectors by reducing a dimensionality of the plurality of multi-dimensional vectors (0004).
Regarding dependent claim 14, Tan in view of Nair disclose the text processing device according to claim 13, wherein the dimensionality of the dimensionality-reduced words is less than 512 (0055).
Regarding dependent claim 15, Tan in view of Nair disclose the text processing device according to claim 14, wherein the dimensionality of the dimensionality-reduced words is 64 (0055).
Regarding independent claim 17, Tan a training device, comprising:
a memory (0007); and
a processor coupled to the memory, the processor configured to, based on instructions stored in the memory, carry out the training method according to claim 7 (0007).
Claims 5-7, 16, 18 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Tan in view of Bhusan et al. (Pub. No.: US 2023/0067976 A1; Filed: Jul. 27, 2021) (hereinafter “Bhusan”).
Regarding dependent claim 5, Tan does not expressly disclose the text processing method according to claim 1, wherein the text processing model is a neural network model comprising Transformer.
Bhusan teaches wherein the text processing model is a neural network model comprising Transformer (0153).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Nair with Tan for the benefit of improving the performance of the models in the final tasks (0004).
Regarding dependent claim 6, Tan does not expressly disclose the text processing method according to claim 1, wherein the text processing comprises at least one of text translation, text classification or text matching.
Bhusan teaches wherein the text processing comprises at least one of text translation, text classification or text matching (0006-0008; 0038; 0042; 0057; 0088-0090).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Nair with Tan for the benefit of improving the performance of the models in the final tasks (0004).
Regarding independent claim 7, Tan disclose a training method for text processing, comprising:
filtering one or more target words with highest first attention scores from words in a piece of training text using a first attention layer of a text processing model (0102-0104; 0111; 0126; 0133);
calculating second attention scores of the target words using a second attention layer of the text processing model (0099; 0104; 0111; 0130-0132);
obtaining a processing result of the training text from the text processing model based on the second attention scores of the target words (0104; 0111; 0130-0132); and
Tan does not expressly disclose training the text processing model based on the processing result of the training text and annotation information of the text.
Bhusan teaches training the text processing model based on the processing result of the training text and annotation information of the text (0116; 0118; 0123; 0143).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Nair with Tan for the benefit of improving the performance of the models in the final tasks (0004).
Regarding dependent claim 16, Tan does not expressly disclose the text processing device according to claim 12, wherein the text processing model is a neural network model comprising Transformer.
Bhusan teaches wherein the text processing model is a neural network model comprising Transformer (0153).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Nair with Tan for the benefit of improving the performance of the models in the final tasks (0004).
Regarding independent claim 18, Tan disclose a text processing system, comprising:
the text processing device according to claim 12; and the training device, comprising:
a memory (0007); and
a processor coupled to the memory, the processor configured to, based on instructions stored in the memory, carry out a training method comprising (0007):
filtering one or more target words with highest first attention scores from words in a piece of training text using a first attention layer of a text processing model (0102-0104; 0111; 0126; 0133);
calculating second attention scores of the target words using a second attention layer of the text processing model (0099; 0104; 0111; 0130-0132);
obtaining a processing result of the training text from the text processing model based on the second attention scores of the target words (0104; 0111; 0130-0132); and
Tan does not expressly disclose training the text processing model based on the processing result of the training text and annotation information of the text.
Bhusan teaches training the text processing model based on the processing result of the training text and annotation information of the text (0116; 0118; 0123; 0143).
Therefore, before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to combine Nair with Tan for the benefit of improving the performance of the models in the final tasks (0004).
Regarding independent claim 20, Tan disclose a non-transitory computer-readable storage medium having stored thereon a computer program that, when executed by a processor, implements the training method according to claim 7.
NOTE
It is noted that any citations to specific, pages, columns, lines, or figures in the prior art references and any interpretation of the reference should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. See MPEP 2123.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JAMES J DEBROW whose telephone number is (571)272-5768. The examiner can normally be reached on 09:00 - 06:00.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Bashore can be reached on 571-272-4088. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center or Private PAIR to authorized users only. Should you have questions about access to Patent Center or the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/James J Debrow/
Primary Patent Examiner
Art Unit 2174
571-272-5768