DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is made non-final.
Claims 1-20 are pending. Claims 1 and 11 are independent claims.
Claim Objections
Claim 17 objected to because of the following informalities: Claim 17 is self-referential. To advance prosecution, in claim 17, “The method of claim 17” will be assumed to refer to claim 16 instead. Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: a hierarchical coarse-grain sparsity (HCGS) selector configured to randomly select… in claim 1. The HCGS selector is described in Section III of the specification: selection of relevant blocks is done through the implementation of block multiplexers – and will be interpreted to be block multiplexers. The same interpretation applies to any dependent claims.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 17 recites the limitation "each subsequent compression". There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP 2106.03. Claim 1 is directed to an apparatus(Step 1: YES).
Step 2A prong 1: Does the claim recite a judicial exception? Claim 1 recites: A neural network accelerator… randomly select block-wise weights (randomly selecting block-wise weights is a mental process). These steps can be performed mentally or are mathematical calculations (Step 2A prong 1: YES).
Step 2A prong 2: Does the claim recite additional elements? Do those additional elements, considered individually and in combination, integrate the judicial exception into a practical application? Claim 1 recites: comprising an input buffer; an output buffer, and a hierarchical coarse-grain sparsity (HCGS) selector configured to… from the input buffer for training a neural network. Describing the input buffer, output buffer, and selector, all configured for neural network training, is recited at a high level of generality, i.e., as a generic computer performing generic computer functions, and represents no more than mere instructions to implement the abstract idea which is equivalent to adding the words “apply it” to the recited judicial exception (Step 2A prong 2: NO).
Step 2B: These elements are recited at such a high level of generality that they fail to integrate the abstract idea into a practical application, since they provide nothing more than mere instructions to implement an abstract idea on a generic computer (MPEP 2106.05(f)). These limitations, taken either alone or in combination, fail to provide an inventive concept (Step 2B: NO). Thus, the claim is not patent eligible.
Regarding claims 2-10, they recite limitations which further narrow the abstract idea by specifying more details of the mental and mathematical process that occurs (Claim 2, using a long short-term memory LSTM as the neural network is recited at a high level of generality, i.e., an attempt to use the neural network (NN) by merely applying the abstract idea (i.e., perform the mental process) without placing any limits on how the NN operates; Claim 3, using a recurrent neural network (RNN) as the neural network is also recited at a high level of generality, i.e., an attempt to use the neural network (NN) by merely applying the abstract idea (i.e., perform the mental process) without placing any limits on how the NN operates; Claim 4, storing weights on chip is an additional element specifying a field of use (AI accelerator chips) without significantly more; Claim 5, storing 50% or more weights on chip is an additional element specifying a field of use without significantly more; Claim 6, compressing weights of a NN is a mental process, i.e., removing weights and reducing the size of the matrix/tensor once or more; Claim 7, compressing weights of a NN is a mental process, i.e., removing weights and reducing the size of the matrix/tensor once or more; Claim 8, randomly selecting weights to enforce block-wise sparsity is a mental process; Claim 9, using low precision quantization is a mental process or mathematical formula, i.e., rounding; Claim 10, stating that the NN is used for on-device automatic speech recognition is limiting the field of use without significantly more).
Regarding claim 11,
Step 1: This part of the eligibility analysis evaluates whether the claim falls within any statutory category. See MPEP 2106.03. Claim 11 is directed to a method (Step 1: YES).
Step 2A prong 1: Does the claim recite a judicial exception? Claim 11 recites: A method for compressing a neural network, the method comprising: randomly selecting a hierarchical structure of block-wise weights… selecting a same number of random blocks for every block row (randomly selecting block-wise weights is a mental process, and selecting a same number of random blocks for every block row is also a mental process). These steps can be performed mentally or are mathematical calculations (Step 2A prong 1: YES).
Step 2A prong 2: Does the claim recite additional elements? Do those additional elements, considered individually and in combination, integrate the judicial exception into a practical application? Claim 11 recites: in the neural network; and training the neural network by… Training the neural network which has weights is recited at a high level of generality, i.e., as a generic computer performing generic computer functions, and represents no more than mere instructions to implement the abstract idea (i.e., mental processes of selecting), which is equivalent to adding the words “apply it” to the recited judicial exception (Step 2A prong 2: NO).
Step 2B: These elements are recited at such a high level of generality that they fail to integrate the abstract idea into a practical application, since they provide nothing more than mere instructions to implement an abstract idea on a generic computer (MPEP 2106.05(f)). These limitations, taken either alone or in combination, fail to provide an inventive concept (Step 2B: NO). Thus, the claim is not patent eligible.
Regarding claims 12-20, they recite limitations which further narrow the abstract idea by specifying more details of the mental and mathematical process that occurs (Claim 12, accelerating the NN on an ASIC is recited at a high level of generality, equivalent to adding the words “apply it” to the recited judicial exception; Claim 13, storing weights on chip is an additional element specifying a field of use (AI accelerator chips) without significantly more; Claim 14, storing 50% or more weights on chip is an additional element specifying a field of use without significantly more; Claim 15, compressing weights of a NN is a mental process, i.e., removing weights and reducing the size of the matrix/tensor once or more; Claim 16, compressing weights of a NN is a mental process, i.e., removing weights and reducing the size of the matrix/tensor once or more; Claim 17, selecting smaller and smaller block sizes is a mental process; Claim 18, compressing weights once using one block size and compressing weights again with a second block size is a mental process; Claim 19, performing compression twice using a larger, then smaller block sizes is a mental process; Claim 20, performing low precision quantization is a mental process or mathematical calculation).
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kadetotad et al. (“A 8.93-TOPS/W LSTM Recurrent Neural Network Accelerator Featuring Hierarchical Coarse-Grain Sparsity With All Parameters Stored On-Chip”, 2019, INCLUDED IN IDS), herein Kadetotad.
Regarding claim 1, Kadetotad teaches: A neural network accelerator, comprising: an input buffer; an output buffer; and a hierarchical coarse-grain sparsity (HCGS) selector (pg. 120, Section III, It consists of the HCGS selector, input and output buffers) configured to randomly select block-wise weights from the input buffer for training a neural network (pg. 120, Section II, The hierarchical structure of block-wise weights is randomly selected).
Regarding claim 2, Kadetotad teaches: The neural network accelerator of claim 1, wherein the neural network is a long short-term memory (LSTM) (pg. 119, Section I, hierarchical coarse-grain sparsity (HCGS) scheme that structurely compresses LSTM weights).
Regarding claim 3, Kadetotad teaches: The neural network accelerator of claim 1, wherein the neural network is a recurrent neural network (RNN) (pg. 120, Section III, The hierarchical structure of block-wise weights is randomly selected before the RNN training process starts).
Regarding claim 4, Kadetotad teaches: The neural network accelerator of claim 1, wherein the weights are stored on-chip (pg. 121, Section III, we first reduced the weight precision down to 6-bit to keep all weights on-chip).
Regarding claim 5, Kadetotad teaches: The neural network accelerator of claim 4, wherein greater than 50% of the weights are stored on-chip (pg. 121, Section III, we first reduced the weight precision down to 6-bit to keep all weights on-chip).
Regarding claim 6, Kadetotad teaches: The neural network accelerator of claim 1, further comprising an HCGS hierarchy having at least one level of weight compression (pg. 119, Section II, Two-level HCGS is illustrated in Fig. 2, where the first level compresses weights (e.g., 4× compression)).
Regarding claim 7, Kadetotad teaches: The neural network accelerator of claim 6, further comprising an HCGS hierarchy having more than one level of compression (pg. 119, Section II, Two-level HCGS is illustrated in Fig. 2, where the first level compresses weights (e.g., 4× compression) using a larger block size (e.g., 32×32) and the remaining weights in the large blocks go through the second level of compression).
Regarding claim 8, Kadetotad teaches: The neural network accelerator of claim 1, wherein the HGSC selector is configured for block-wise sparsity (pg. 119, Section I, structured sparsity techniques have been proposed with row-/column-wise sparsity for recurrent neural networks (RNNs) [3], with block-wise sparsity for multilayer perceptrons).
Regarding claim 9, Kadetotad teaches: The neural network accelerator of claim 1, wherein the HCGS selector is configured for low-precision quantization (pg. 122, section V, Exploiting the hierarchical block-wise sparsity and low-precision quantization, our accelerator stores the entire compressed weights).
Regarding claim 10, Kadetotad teaches: The neural network accelerator of claim 1, wherein the neural network is trained for on-device automatic speech recognition (ASR) (pg. 122, section V, This letter presented a hierarchically compressed, energy-efficient LSTM accelerator for ASR… our accelerator stores the entire compressed weights of 3-layer, 512-cell LSTMs in 288 kB of on-chip SRAM).
Regarding claim 11, Kadetotad teaches: A method for compressing a neural network, the method comprising: randomly selecting a hierarchical structure of block-wise weights in the neural network; and training the neural network by selecting a same number of random blocks for every block row (pg. 120, Section II, The hierarchical structure of block-wise weights is randomly selected before the RNN training process starts, and is maintained throughout training and classification phases. HCGS selects the same number of blocks for every block-row).
Regarding claim 12, Kadetotad teaches: The method of claim 11, further comprising accelerating the neural network on an application-specific integrated circuit (ASIC) (pg. 122, Section IV, Compared to the RNN ASIC works of [6] and [7], this letter shows 2.90× and 1.75× higher energy-efficiency (TOPS/W), respectively. Table I shows the detailed comparison with prior ASIC/FPGA works for RNNs).
Regarding claim 13, Kadetotad teaches: The method of claim 11, further comprising the step of storing the weights on-chip (pg. 121, Section III, we first reduced the weight precision down to 6-bit to keep all weights on-chip).
Regarding claim 14, Kadetotad teaches: The method of claim 13, wherein greater than 50% of the weights are stored on-chip (pg. 121, Section III, we first reduced the weight precision down to 6-bit to keep all weights on-chip).
Regarding claim 15, Kadetotad teaches: The method of claim 11, further comprising the step of compressing the weights at least once (pg. 119, Section II, Two-level HCGS is illustrated in Fig. 2, where the first level compresses weights (e.g., 4× compression)).
Regarding claim 16, Kadetotad teaches: The method of claim 15, further comprising the step of compressing the weights more than once (pg. 119, Section II, Two-level HCGS is illustrated in Fig. 2, where the first level compresses weights (e.g., 4× compression) using a larger block size (e.g., 32×32) and the remaining weights in the large blocks go through the second level of compression).
Regarding claim 17, Kadetotad teaches: The method of claim 17 (*assumed to be 16), further comprising the step of recursively selecting smaller block sizes for each subsequent compression (Abstract, Aided by HCGS-based block-wise recursive weight compression, we demonstrate LSTM networks with up to 16× fewer weights).
Regarding claim 18, Kadetotad teaches: The method of claim 11, further comprising the step of compressing weights a first time using a first block size, and compressing weights a second time using a second block size (pg. 119, Section II, Two-level HCGS is illustrated in Fig. 2, where the first level compresses weights (e.g., 4× compression) using a larger block size (e.g., 32×32) and the remaining weights in the large blocks go through the second level of compression).
Regarding claim 19, Kadetotad teaches: The method of claim 18, wherein the first block size is larger than the second block size (pg. 119, Section II, Two-level HCGS is illustrated in Fig. 2, where the first level compresses weights (e.g., 4× compression) using a larger block size (e.g., 32×32) and the remaining weights in the large blocks go through the second level of compression).
Regarding claim 20, Kadetotad teaches: The method of claim 11, further comprising the step of low precision quantization of the block-wise weights (pg. 122, section V, Exploiting the hierarchical block-wise sparsity and low-precision quantization, our accelerator stores the entire compressed weights).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Zhang et al. (US 20210150362 A1), which discloses storing compressed weights on-chip, as well as block sparsity, and Seo et al. (US 20190164538 A1), which discloses an ASIC based DNN compressor for automatic speech recognition that takes advantage of block-wise sparsity.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HARRISON CHAN YOUNG KIM whose telephone number is (571)272-0713. The examiner can normally be reached Monday - Thursday 9:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, CESAR PAULA can be reached at (571) 272-4128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HARRISON C KIM/ Examiner, Art Unit 2145
/CESAR B PAULA/ Supervisory Patent Examiner, Art Unit 2145