DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claim 1 recites the limitation: "… generating, for each first padded input sequence, a respective first padding mask identifying the part of the first padded input sequence that contains the padding values; generating a first attention mask from each first padding mask, wherein the generating comprises an outer product operation applied to the first padding mask; processing the first padded input sequences and the first attention masks through the test neural network;…" (emphasis added to accentuate insufficient antecedent basis).
The limitation lacks clarity for the following reasons:
No prior reference to “the part”.
The “first attention mask” begins with singular form and moves to a plural form, “the first attention masks”.
For the purposes of examination, the limitation is interpreted as the following:
"… generating, for each first padded input sequence, a respective first padding mask identifying a part of the first padded input sequence that contains the padding values; generating a first attention mask from each first padding mask, thereby generating a set of first attention masks, wherein the generating comprises …".
REASONS FOR ALLOWANCE
The following is an examiner’s statement of reasons for allowance: The instant invention is related to efficiently implementing attention-based neural networks, such as Transformers, on fixed-function hardware accelerators by processing padded test sequences to collect statistical data regarding value ranges at various network layers. The data is then used to select optimal numerical formats for the hardware implementation.
Prior art for was found for the claims as follows:
Re. Claim 1,
Henderson (US 2021/0141798 A1) discloses the following limitations:
A computer-implemented method for selecting numerical formats for use in configuring a hardware implementation of an attention-based neural network (Henderson: Abstract.), the method comprising:
obtaining a representation of the attention-based neural network (Henderson: Para. [0108] discloses in the example illustrated in FIG. 6 (a), a neural network architecture based on a 'Transformer Network' is used.);
implementing the representation as a test neural network (Henderson: Para. [0183] discloses applying quantization to the dual-encoder during training allows reduction of the size of the model whilst maintaining accuracy. The training is performed in a 'quantization-aware' manner.);
obtaining a dataset of first test input sequences for the attention-based neural network, wherein the dataset includes first test input sequences of varying length (Henderson: Para. [0110], [0142] disclose the training data is limited to sentences with a maximum of 128 characters. In case the length is greater than 60 units, then the sequence is truncated. For an example, the length of the sequence of embeddings output from the first model 205 fed into the second model 207 is truncated to 60 units.);
padding each first test input sequence with padding values to produce a respective first padded input sequence of a first fixed length (Henderson: Para. [0110], [0142] disclose in case the output 507 is shorter than N=60, the sequence is padded accordingly so that all the sequences are of the same length.);
generating, for each first padded input sequence, a respective first padding mask identifying the part of the first padded input sequence that contains the padding values (Henderson: Para. [0110] discloses "Padding masking is applied to mitigate any effect of the padding on further computations … Masking may be implemented by including indicator variables which denote which parts of the sequence have been padded and which come from the original sequence in a mask tensor.".);
processing the first padded input sequences and the first attention masks through the test neural network (Henderson: Para. [0110] discloses "Padding masking is applied to mitigate any effect of the padding on further computations … The sequence of vectors output from the positional encodings step 601 is then fed into a first block of a set of M repeating blocks 603.".);
Bourges-Sevenier et al., (US 2019/0325314 A1) disclose the following limitations:
collecting statistics describing ranges of values obtained during said processing, wherein the statistics describe ranges of values for at least two different layers of the attention-based neural network (Bourges-Sevenier: Para. [0026], [0029] disclose the example model analyzer 225 analyzes the trained parameters/activations of the model accessed by the model accessor 220 to infer weighting magnitude(s) from the model and before applying the quantization function, the value distribution of each operand (e.g., within a DNN model layer) are shifted to an appropriate order of magnitude.); and
selecting numerical formats for the at least two different layers based on the collected statistics (Bourges-Sevenier: Paras. [0015], [0028] discloses quantizer 235 identifies (e.g., selects) a layer of the model for processing, and performs quantization of the layer based on the constraints. The weights may be quantized to an 8-bit integer value, without an appreciable loss of accuracy of the model. Quantization results in a model that is approximately quarter the size, as compared to a model that is not quantized. More importantly, because the model uses smaller bit-widths (e.g., 8 bit values, as opposed to 16 bit, 32 bit, 64 bit, 128 bit, etc. values).).
Applicant uniquely claimed a distinct feature in the instant invention, which is not found in the prior art, either singularly or in combination. The feature is [Claim 1] “… generating a first attention mask from each first padding mask, wherein the generating comprises an outer product operation applied to the first padding mask; …”. This feature is not found or suggested in the prior art.
Claims 1-20 would be allowable provided that the rejections under 35 U.S.C. 112(b) are overcome.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure and can be seen in the list of cited references.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEET DHILLON whose telephone number is (571)270-5647. The examiner can normally be reached M-F: 5am-1:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sath V. Perungavoor can be reached on 571-272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PEET DHILLON/Primary Examiner
Art Unit: 2488
Date: 02-14-2026