Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on April 28, 2025 has been entered.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 10-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 10,
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 10 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“reordering the two or more weights into ascending order”
“populating a weights vector with the two or more weights in ascending order”
“performing a convolution cycle”
“performing convolutional multiplications”
As drafted, under their broadest reasonable interpretations, cover mental processes, i.e., concepts performed in the human mind (including an observation, evaluation, judgement, opinion). The above limitations in the context of this claim correspond to mental processes, e.g., evaluation and judgement with assistance of pen and paper. Additionally or alternatively, as drafted, under their broadest reasonable interpretations, cover mathematical concepts, i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations. The above limitations in the context of this claim additionally or alternatively correspond to mathematical relationships and/or calculations.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are mere instructions to apply an exception (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)).
The limitations:
“providing a filter in the one or more convolutional layers, wherein each filter comprises individual positive and negative weights”
“providing a controller, wherein said controller manages timing and organization of the stochastic multiplication”
“providing an input buffer, wherein said input buffer fetches input data from a memory of the neural network”
“providing an output buffer, wherein said output buffer stores one or more corresponding results of each convolution operation performed”
“providing an accelerator capable of performing two-dimensional convolution”
As drafted, are additional elements that amount to no more than mere instructions to apply an exception for the abstract ideas. See MPEP 2106.05(f).
The limitations:
“storing indices of the two or more weights' original order in an index buffer”
“wherein the controller fetches one weight from the input buffer”
“wherein the controller … broadcasts said weight to one or more counters in the accelerator”
“storing the one or more results in the output buffer”
As drafted, are additional elements that amount to no more than insignificant extra-solution activity. See MPEP 2106.05(g).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements are “mere instructions to apply” and “insignificant extra-solution activity”. Specifically, the storing and fetching from a buffer limitations recite the well-understood, routine, and conventional activity of storing and retrieving information in memory. MPEP 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015) (storing and retrieving information in memory). Further, the broadcasting limitations recite the well-understood, routine, and convention activity of receiving or transmitting data over a network. MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network). Mere instructions to apply and insignificant extra-solution activity cannot provide an inventive concept. As an ordered whole, the claim is directed to a mentally performable or mathematical process of performing stochastic multiplication. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Regarding Claim 11,
Claim 11 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 11 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: The limitations:
“reordering the two or more weights into ascending order”
“populating a weights vector with the two or more weights in ascending order”
“performing a convolution cycle”
“performing convolutional multiplications”
As drafted, under their broadest reasonable interpretations, cover mental processes, i.e., concepts performed in the human mind (including an observation, evaluation, judgement, opinion). The above limitations in the context of this claim correspond to mental processes, e.g., evaluation and judgement with assistance of pen and paper. Additionally or alternatively, as drafted, under their broadest reasonable interpretations, cover mathematical concepts, i.e., mathematical relationships, mathematical formulas or equations, and mathematical calculations. The above limitations in the context of this claim additionally or alternatively correspond to mathematical relationships and/or calculations.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are mere instructions to apply an exception (See MPEP 2106.05(f)) and insignificant extra-solution activity (See MPEP 2106.05(g)).
The limitations:
“providing a filter comprising two or more weights in the one or more convolutional layers, wherein each filter comprises positive and negative individual weights”
“providing a controller, wherein said controller manages timing and organization of the stochastic multiplication”
“providing an input buffer, wherein said input buffer fetches input data from a memory of the neural network”
“providing an output buffer, wherein said output buffer stores one or more corresponding results of each convolution operation performed
“providing an accelerator capable of performing two-dimensional convolution, comprising:(a) an index buffer;(b) a weight buffer;(c) a binary number-to-stochastic number converter;(d) a sign holder; and(e) a counters and summation unit, comprising one or more counters”
As drafted, are additional elements that amount to no more than mere instructions to apply an exception for the abstract ideas. See MPEP 2106.05(f).
The limitations:
“storing indices of the two or more weights' original order in an index buffer”
“wherein the controller fetches one weight from the input buffer”
“wherein the controller …broadcasts said weight to one or more counters in the accelerator”
“storing the one or more results in the output buffer”
As drafted, are additional elements that amount to no more than insignificant extra-solution activity. See MPEP 2106.05(g).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements are “mere instructions to apply” and “insignificant extra-solution activity”. Specifically, the storing and fetching from a buffer limitations recite the well-understood, routine, and conventional activity of storing and retrieving information in memory. MPEP 2106.05(d)(II); Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015) (storing and retrieving information in memory). Further, the broadcasting limitations recite the well-understood, routine, and convention activity of receiving or transmitting data over a network. MPEP 2106.05(d)(II); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network). Mere instructions to apply and insignificant extra-solution activity cannot provide an inventive concept. As an ordered whole, the claim is directed to a mentally performable or mathematical process of performing stochastic multiplication. Nothing in the claim provides significantly more than this. As such, the claim is not patent eligible.
Regarding Claim 12,
Claim 12 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 12 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: See corresponding analysis of claim 11.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are additional details that do not apply the exception in a meaningful way (See MPEP 2106.05(e)).
The limitations:
“wherein the weights vector is stored in the weight buffer”
As drafted, are additional elements that do not apply an exception for the abstract ideas in a meaningful way. See MPEP 2106.05(e).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements do not apply the exception in a meaningful way. The claim is not patent eligible.
Regarding Claim 13,
Claim 13 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 13 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: See corresponding analysis of claim 11.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are additional details that do not apply the exception in a meaningful way (See MPEP 2106.05(e)).
The limitations:
“wherein the one or more counters comprise an up counter and a down counter”
As drafted, are additional elements that do not apply an exception for the abstract ideas in a meaningful way. See MPEP 2106.05(e).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements do not apply the exception in a meaningful way. The claim is not patent eligible.
Regarding Claim 14,
Claim 14 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 14 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: See corresponding analysis of claim 11.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are additional details that do not apply the exception in a meaningful way (See MPEP 2106.05(e)).
The limitations:
“wherein the one or more counters count in descending order if the weight fetched by the controller is negative”
As drafted, are additional elements that do not apply an exception for the abstract ideas in a meaningful way. See MPEP 2106.05(e).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements do not apply the exception in a meaningful way. The claim is not patent eligible.
Regarding Claim 15,
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 15 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: See corresponding analysis of claim 11.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are additional details that do not apply the exception in a meaningful way (See MPEP 2106.05(e)).
The limitations:
“wherein the one or more counters count in ascending order if the weight fetched by the controller is positive”
As drafted, are additional elements that do not apply an exception for the abstract ideas in a meaningful way. See MPEP 2106.05(e).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements do not apply the exception in a meaningful way. The claim is not patent eligible.
Regarding Claim 16,
Claim 16 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 16 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: See corresponding analysis of claim 10.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are additional details that do not apply the exception in a meaningful way (See MPEP 2106.05(e)).
The limitations:
“wherein the input data to the first convolutional layer is one or more pixels of an image and comprise one or more values greater than zero”
As drafted, are additional elements that do not apply an exception for the abstract ideas in a meaningful way. See MPEP 2106.05(e).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements do not apply the exception in a meaningful way. The claim is not patent eligible.
Regarding Claim 17,
Claim 17 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 17 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: See corresponding analysis of claim 11.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are additional details that do not apply the exception in a meaningful way (See MPEP 2106.05(e)).
The limitations:
“wherein the sign holder instructs the counters and summations unit whether to count upwards or downwards”
As drafted, are additional elements that do not apply an exception for the abstract ideas in a meaningful way. See MPEP 2106.05(e).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements do not apply the exception in a meaningful way. The claim is not patent eligible.
Regarding Claim 18,
Claim 18 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 18 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: See corresponding analysis of claim 11.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are additional details that do not apply the exception in a meaningful way (See MPEP 2106.05(e)).
The limitations:
“wherein the one or more counters multiply one or more input feature maps by individual weights and convert one or more results of said multiplication to binary numbers”
As drafted, are additional elements that do not apply an exception for the abstract ideas in a meaningful way. See MPEP 2106.05(e).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements do not apply the exception in a meaningful way. The claim is not patent eligible.
Regarding Claim 19,
Claim 19 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1 Analysis: Claim 19 is directed to a method of performing stochastic multiplication in one or more convolutional layers of a neural network, which is directed to a process, one of the statutory categories.
Step 2A Prong One Analysis: See corresponding analysis of claim 18.
Step 2A Prong Two Analysis: The judicial exceptions are not integrated into a practical application. In particular, the claim recited additional elements that are additional details that do not apply the exception in a meaningful way (See MPEP 2106.05(e)).
The limitations:
“wherein the one or more counters are followed by the summation unit adding the one or more results with respect to the original order of the weights in the index buffer”
As drafted, are additional elements that do not apply an exception for the abstract ideas in a meaningful way. See MPEP 2106.05(e).
Therefore, the additional elements do not integrate the abstract ideas into a practical application.
Step 2B Analysis: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract ideas into a practical application, all of the additional elements do not apply the exception in a meaningful way. The claim is not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 10 and 16 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Akhlaghi et al. (“SnaPEA: Predictive Early Activation for Reducing Computation in Deep Convolutional Neural Networks”) (“Akhlaghi”).
Regarding claim 10, Akhlaghi teaches a method of performing stochastic multiplication in one or more convolutional layers of a neural network comprising: providing a filter in the one or more convolutional layers (Akhlaghi Section II.A SnaPEA Software Workflow “Figure 3 depicts the software workflow of SnaPEA which takes a CNN model, an acceptable accuracy loss, and an optimization dataset as its inputs. The CNN goes through the multiple passes of this workflow. The first pass, called Convolution Layer Extraction, elicits the convolution kernels of the CNN. Then, the weights of each kernel are re-ordered through the remaining passes, depending on the operating mode, exact or predictive” Akhlaghi provides kernels corresponding to filters in the one or more convolutional layers.), wherein each filter comprises individual positive and negative weights (Akhlaghi Section II.A SnaPEA Software Workflow “Given this insight, in the exact mode, Sign-Based Weight Reordering pass reorders the weights of convolution kernels based on their sign such that the positive subset are followed by the negative subset.” Akhlaghi provides individual positive and negative weights within a kernel corresponding to a filter.); storing indices of the two or more weights' original order in an index buffer (Akhlaghi Section II.B SnaPEA Hardware Architecture “The SnaPEA architecture comprises number of identical Processing Engines (PEs), each of which is designed to compute a convolution using the reordered weights. To support computation with the reorderings, each PE is equipped with an index buffer that hold the indices of weights in the original kernel.” Akhlaghi provides an index buffer for storing the indices of weights in the original kernel corresponding to storing indices of the two or more weights' original order in an index buffer.); reordering the two or more weights into ascending order (Akhlaghi Section IV SnaPEA Software Optimization “To mitigate the mentioned issue, SnaPEA sorts the weights in ascending order, partitions them into a number of smaller groups, and selects the weight with the largest magnitude from each group.” Akhlaghi provides reordering the weights into ascending order.); populating a weights vector with the two or more weights in ascending order (Akhlaghi Section IV.B Problem Formulation “Let assume that the reordered weights are stored in a 1D array such that the Nk l speculation weights are placed at the beginning of the array while the remaining positive weights followed by the remaining negative weights are placed at the end.”; Section V. Weight and index buffers “The weight buffer contains the weight values of the convolution kernels in the pre-determined order (See Section IV). The weights are ordered offline and loaded into the memory with the proper ordering. Since the ordering of the weights are changed, we also need to add an index buffer to properly index the input buffer.” Akhlaghi provides 1D weight arrays corresponding to weight vectors and sorting weights in ascending order and subsequently loading them into the weight buffer, corresponding to populating a weights vector with the two or more weights in ascending order.); providing a controller, wherein said controller manages timing and organization of the stochastic multiplication (Akhlaghi Section V. Processing Engine “In every cycle, the PE controller reads one weight value from the weight buffer and broadcasts it to all the compute (MAC units) lanes. The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer.” Akhlaghi provides an architecture comprising a controller, which performs operations at each cycle, thus managing timing and organizing the architecture's multiplication operations.); providing an input buffer, wherein said input buffer fetches input data from a memory of the neural network (Akhlaghi Section V. Processing Engine “The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer. Upon receiving the index, the input buffer reads a set of values (one value per each MAC unit) and sends them to the MAC unit for processing” Akhlaghi provides an input buffer, which receives fetched input data, corresponding to the input buffer comprises functionality to fetch one or more inputs from a memory of the neural network.); providing an output buffer, wherein said output buffer stores one or more corresponding results of each convolution operation performed (Akhlaghi Section V. Input/Output Buffers “Upon completion of all the computations, the results are written into the output buffer” Akhlaghi provides an output buffer which stores one or more results computed by the architecture.); providing an accelerator capable of performing two-dimensional convolution (Akhlaghi Section II. SnaPEA Hardware-Software Solution “To utilize these transformations, the SnaPEA comes with an accelerator design that can efficiently execute the CNN with reordered convolution weights with support for early termination of convolution.” Akhlaghi provides an architecture comprising an accelerator corresponding to providing an accelerator capable of performing two-dimensional convolution.); performing a convolution cycle, wherein the controller fetches one weight from the input buffer and broadcasts said weight to one or more counters in the accelerator (Akhlaghi Section V. Processing Engine “In every cycle, the PE controller reads one weight value from the weight buffer and broadcasts it to all the compute (MAC units) lanes. The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer. Upon receiving the index, the input buffer reads a set of values (one value per each MAC unit) and sends them to the MAC unit for processing.” Akhlaghi provides in every cycle, the controller reads a weight from the weight buffer and broadcasts it to the MAC corresponding to performing a convolution cycle, wherein the controller fetches one weight from the input buffer and broadcasts said weight to one or more counters in the accelerator.); performing convolutional multiplications (Akhlaghi Section V. Processing Engine “That is, each MAC unit performs the multiplication of one input and weight for each convolution window and sends the results to the accumulation register. The accumulation register accumulates the partial sums for each convolution window.” Akhlaghi provides a MAC and performing convolutional multiplications.); and storing the one or more results in the output buffer (Akhlaghi Section V. Input/Output Buffers “Upon completion of all the computations, the results are written into the output buffer” Akhlaghi provides an output buffer which stores one or more results computed by the architecture.).
Regarding claim 16, Akhlaghi teaches the method of claim 10, wherein the input data to the first convolutional layer is one or more pixels of an image and comprise one or more values greater than zero (Akhlaghi Section IV.B Problem Formulation “Let od l,k be the result of a single convolution window obtained by kernel k in layer l with the speculation parameters Thk l and Nk l for the input image d. The number of MAC operations to compute od l,k can be calculated by the function Op shown in (1). Let assume that the reordered weights are stored in a 1D array such that the Nk l speculation weights are placed at the beginning of the array while the remaining positive weights followed by the remaining negative weights are placed at the end.” Akhlaghi provides input images for a convolutional neural network corresponding to the input data to the first convolutional layer is one or more pixels of an image and comprise one or more values greater than zero.).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 4-5, 7-8, 11-15, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Akhlaghi et al. (“SnaPEA: Predictive Early Activation for Reducing Computation in Deep Convolutional Neural Networks”) (“Akhlaghi”) in view of Sim et al. (“A New Stochastic Computing Multiplier with Application to Deep Convolutional Neural Networks”) (“Sim”).
Regarding claim 1, Akhlaghi teaches an architecture for performing stochastic multiplication in one or more convolutional layers of a convolutional neural network comprising: (a) a controller (Akhlaghi Section V. Processing Engine “In every cycle, the PE controller reads one weight value from the weight buffer and broadcasts it to all the compute (MAC units) lanes. The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer.” Akhlaghi provides an architecture comprising a controller.); (b) an input buffer (Akhlaghi Section V. Processing Engine “In every cycle, the PE controller reads one weight value from the weight buffer and broadcasts it to all the compute (MAC units) lanes. The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer.” Akhlaghi provides an architecture comprising an input buffer.); (c) an output buffer (Akhlaghi Section V. Processing Engine “Figure 6 (b) depicts the microarchitecture of one PE in the SnaPEA architecture. Each PE comprises multiple compute lanes, a weight and index buffer, an input/output buffer, and multiple Predictive Activation Units” Akhlaghi provides an architecture comprising an output buffer.); (d) an accelerator (Akhlaghi Section II. SnaPEA Hardware-Software Solution “To utilize these transformations, the SnaPEA comes with an accelerator design that can efficiently execute the CNN with reordered convolution weights with support for early termination of convolution.” Akhlaghi provides an architecture comprising an accelerator.); and (e) a multiply-and-accumulate unit (Akhlaghi Section V. Processing Engine “Each compute lane consists of one dedicated Multiply-and-Accumulate (MAC) unit and one Predictive Activation Unit (PAU).” Akhlaghi provides an architecture comprising a multiply-and-accumulate unit.), comprising:(i) a weight buffer, comprising a plurality of weights (Akhlaghi Section V. Weight and index buffers “The weight buffer contains the weight values of the convolution kernels in the pre-determined order (See Section IV).” Akhlaghi provides an architecture comprising a weight buffer, comprising a plurality of weights.) …wherein the plurality of weights are ordered according to each weight's individual size in relation to other weights in the weight buffer (Akhlaghi Section IV SnaPEA Software Optimization “To mitigate the mentioned issue, SnaPEA sorts the weights in ascending order, partitions them into a number of smaller groups, and selects the weight with the largest magnitude from each group.”; Section V. Weight and index buffers “The weight buffer contains the weight values of the convolution kernels in the pre-determined order (See Section IV). The weights are ordered offline and loaded into the memory with the proper ordering. Since the ordering of the weights are changed, we also need to add an index buffer to properly index the input buffer.” Akhlaghi provides sorting weights in ascending order and subsequently loading them into the weight buffer, corresponding to the plurality of weights are ordered according to each weight's individual size in relation to other weights in the weight buffer.); wherein the controller is configured to manage timing and organize the architecture's operation (Akhlaghi Section V. Processing Engine “In every cycle, the PE controller reads one weight value from the weight buffer and broadcasts it to all the compute (MAC units) lanes. The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer.” Akhlaghi provides an architecture comprising a controller, which manages timing and organizes the architecture's operation over a plurality of cycles.); wherein the input buffer comprises functionality to fetch one or more inputs from a memory of the neural network (Akhlaghi Section V. Processing Engine “The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer. Upon receiving the index, the input buffer reads a set of values (one value per each MAC unit) and sends them to the MAC unit for processing” Akhlaghi provides the index buffer and sends the fetched index to the input buffer, corresponding to the input buffer comprises functionality to fetch one or more inputs from a memory of the neural network.); and wherein the output buffer stores one or more results computed by the architecture (Akhlaghi Section V. Input/Output Buffers “Upon completion of all the computations, the results are written into the output buffer” Akhlaghi provides an output buffer which stores one or more results computed by the architecture.).
Akhlaghi fails to teach a multiply-and-accumulate unit comprising …(ii) a down counter; (iii) an up counter; and (iv) a stochastic number generator unit, comprising: (x) a multiplexer; and (y) a finite-state-machine.
However, Sim teaches a multiply-and-accumulate unit (Sim Section 2.2 Our Proposed SC-MAC “Our SC multiplier illustrated in Fig. 1(c) can also be called SC-MAC, since the counter naturally accumulates results from consecutive multiplications. The counter only needs to have a wider width.” Sim provides multiply-and-accumulate units.) comprising …(ii) a down counter (Sim Section 3.1 BISC-MVM: Vectorization of Our SC-MAC “Fig. 3(a) illustrates our BISC-MVM, which contains p parallel SCMACs of N-bit multiplier precision. Each SC-MAC requires a mux and an up-down counter, whose width is N +A bits (A additional bits are for accumulation). All muxes share the same control input, hence the same FSM. The down counter can be shared as well if the other operand, w, is common to all, as is the case with our BISC-MVM” Sim provides a MAC comprising a down counter.); (iii) an up counter (Sim Section 3.1 BISC-MVM: Vectorization of Our SC-MAC “Fig. 3(a) illustrates our BISC-MVM, which contains p parallel SCMACs of N-bit multiplier precision. Each SC-MAC requires a mux and an up-down counter, whose width is N +A bits (A additional bits are for accumulation). All muxes share the same control input, hence the same FSM. The down counter can be shared as well if the other operand, w, is common to all, as is the case with our BISC-MVM” Sim provides a MAC comprising an up counter.); and (iv) a stochastic number generator unit (Sim Section 1 Introduction “Accuracy is enhanced over previous work (i) by our new SC multiply algorithm and (ii) through the use of our novel low-discrepancy SNG (Stochastic Number Generator) scheme” Sim provides a stochastic number generator.), comprising: (x) a multiplexer (Sim Section 2.3 Enhancing Accuracy via Low-Discrepancy Code “This allows us to use a simple and deterministic bit shuffling scheme via an N-bit FSM (Finite-State Machine) and one mux, which is in fact simpler than the conventional LFSR-comparator-based SNG and far simpler than a Halton sequence generator” Sim provides a mux corresponding to a multiplexer.); and (y) a finite-state-machine (Sim Section 2.3 Enhancing Accuracy via Low-Discrepancy Code “This allows us to use a simple and deterministic bit shuffling scheme via an N-bit FSM (Finite-State Machine) and one mux, which is in fact simpler than the conventional LFSR-comparator-based SNG and far simpler than a Halton sequence generator” Sim provides a Finite-State Machine.).
Akhlaghi and Sim are both considered analogous to the claimed invention because they are in the same field of artificial intelligence and more specifically convolutional neural networks. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Akhlaghi with the above teachings of Sim. Doing so would allow for an alternative, more error-resilient way of representing numbers for when device reliability is no longer guaranteed (Sim Section 1 Introduction “Stochastic computing (SC) provides an alternative, more error-resilient way of representing numbers for when device reliability is no longer guaranteed.”).
Regarding claim 2, Akhlaghi teaches an architecture for performing stochastic multiplication in one or more convolutional layers of a convolutional neural network comprising: (a) a controller (Akhlaghi Section V. Processing Engine “In every cycle, the PE controller reads one weight value from the weight buffer and broadcasts it to all the compute (MAC units) lanes. The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer.” Akhlaghi provides an architecture comprising a controller.); (b) an input buffer (Akhlaghi Section V. Processing Engine “In every cycle, the PE controller reads one weight value from the weight buffer and broadcasts it to all the compute (MAC units) lanes. The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer.” Akhlaghi provides an architecture comprising an input buffer.); (c) an output buffer (Akhlaghi Section V. Processing Engine “Figure 6 (b) depicts the microarchitecture of one PE in the SnaPEA architecture. Each PE comprises multiple compute lanes, a weight and index buffer, an input/output buffer, and multiple Predictive Activation Units” Akhlaghi provides an architecture comprising an output buffer.); and (d) an accelerator (Akhlaghi Section II. SnaPEA Hardware-Software Solution “To utilize these transformations, the SnaPEA comes with an accelerator design that can efficiently execute the CNN with reordered convolution weights with support for early termination of convolution.” Akhlaghi provides an architecture comprising an accelerator.), comprising: (i) an index buffer (Akhlaghi Section V. Weight and index buffers “The weight buffer contains the weight values of the convolution kernels in the pre-determined order (See Section IV). The weights are ordered offline and loaded into the memory with the proper ordering. Since the ordering of the weights are changed, we also need to add an index buffer to properly index the input buffer.” Akhlaghi provides an index buffer.); (ii) a weight buffer, comprising a plurality of weights (Akhlaghi Section V. Weight and index buffers “The weight buffer contains the weight values of the convolution kernels in the pre-determined order (See Section IV). The weights are ordered offline and loaded into the memory with the proper ordering. Since the ordering of the weights are changed, we also need to add an index buffer to properly index the input buffer.” Akhlaghi provides a weight buffer, comprising a plurality of weights.) …and (v) a counters and summation unit (Akhlaghi Section V. Processing Engine “Each compute lane consists of one dedicated Multiply-and-Accumulate (MAC) unit and one Predictive Activation Unit (PAU).” Akhlaghi provides an architecture comprising a multiply-and-accumulate unit corresponding to a counters and summation unit.); wherein the plurality of weights are ordered in ascending order according to each weight's individual size in relation to other weights in the weight buffer (Akhlaghi Section IV SnaPEA Software Optimization “To mitigate the mentioned issue, SnaPEA sorts the weights in ascending order, partitions them into a number of smaller groups, and selects the weight with the largest magnitude from each group.”; Section V. Weight and index buffers “The weight buffer contains the weight values of the convolution kernels in the pre-determined order (See Section IV). The weights are ordered offline and loaded into the memory with the proper ordering. Since the ordering of the weights are changed, we also need to add an index buffer to properly index the input buffer.” Akhlaghi provides sorting weights in ascending order and subsequently loading them into the weight buffer, corresponding to the plurality of weights are ordered according to each weight's individual size in relation to other weights in the weight buffer.); wherein the controller is configured to manage timing and organize the architecture's operation (Akhlaghi Section V. Processing Engine “In every cycle, the PE controller reads one weight value from the weight buffer and broadcasts it to all the compute (MAC units) lanes. The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer.” Akhlaghi provides an architecture comprising a controller, which manages timing and organizes the architecture's operation over a plurality of cycles.); wherein the input buffer comprises functionality to fetch input data from a memory of the neural network (Akhlaghi Section V. Processing Engine “The PE controller also reads one weight index from the index buffer and sends the fetched index to the input buffer. Upon receiving the index, the input buffer reads a set of values (one value per each MAC unit) and sends them to the MAC unit for processing” Akhlaghi the index buffer and sends the fetched index to the input buffer, corresponding to the input buffer comprises functionality to fetch one or more inputs from a memory of the neural network.); and wherein the output buffer stores one or more results computed by the architecture (Akhlaghi Section V. Input/Output Buffers “Upon completion of all the computations, the results are written into the output buffer” Akhlaghi provides an output buffer which stores one or more results computed by the architecture.).
Akhlaghi fails to teach an accelerator, comprising: …(iii) a binary number-to-stochastic number converter; (iv) a sign holder.
However, Sim teaches an accelerator (Sim Section 3.3 Our SC-CNN Accelerator Architecture “Our SC-CNN accelerator architecture is by design very similar to previous CNN accelerators based on conventional binary.” Sim provides an accelerator.), comprising: …(iii) a binary number-to-stochastic number converter (Sim Section 2.1 Conventional SC Multiplication “An SNG (Stochastic Number Generator), which is synonymous to BN-to-SN converter, takes an N-bit binary number (BN) and generates an SN bitstream, and it typically consists of a random number generator such as an N-bit LFSR (Linear Feedback Shift Register) and an N-bit comparator, which generates 1 if the random number is less than the input BN, and 0 otherwise” Sim teaches a binary number-to-stochastic number converter.); (iv) a sign holder (Sim Section 2.4 Extension to Support Signed Multiplication “Our scheme can be extended to support signed multiplications, where both x and w as well as the output are represented in two’s complement. The only major change is that the bitstream counter now becomes an up-down counter, counting up for input ‘1’ and down for ‘0’. The other changes are minor. The sign bit of input x is flipped and XOR-ed with the sign bit of the other input w after being converted to sign-magnitude representation, and the magnitude part is fed to the down counter as before” Sim provides operations including signed multiplication corresponding to an accelerator comprising a sign holder.);
Akhlaghi and Sim are both considered analogous to the claimed invention because they are in the same field of artificial intelligence and more specifically convolutional neural networks. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Akhlaghi with the above teachings of Sim. Doing so would allow for an alternative, more error-resilient way of representing numbers for when device reliability is no longer guaranteed (Sim Section 1 Introduction “Stochastic computing (SC) provides an alternative, more error-resilient way of representing numbers for when device reliability is no longer guaranteed.”).
Regarding claim 4, Akhlaghi in view of Sim teaches the architecture of claim 2 as discussed above in the rejection of claim 2, wherein the controller comprises functionality to fetch one or more individual weights from the weight buffer and broadcasts said weight to the counters and summation unit (Akhlaghi Section V Processing Engine “In every cycle, the PE controller reads one weight value from the weight buffer and broadcasts it to all the compute (MAC units) lanes.” Akhlaghi provides a controller fetching weights from a weight buffer and broadcasting to the MAC units, corresponding to the controller comprises functionality to fetch one or more individual weights from the weight buffer and broadcasts said weight to the counters and summation unit.).
It would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Akhlaghi in view of Sim for the same reasons disclosed above in the rejection of claim 2.
Regarding claim 5, Akhlaghi in view of Sim teaches the architecture of claim 2 as discussed above in the rejection of claim 2, wherein the sign holder comprises functionality to support signed multiplication operations (Sim Section 2.4 Extension to Support Signed Multiplication “Our scheme can be extended to support signed multiplications, where both x and w as well as the output are represented in two’s complement. The only major change is that the bitstream counter now becomes an up-down counter, counting up for input ‘1’ and down for ‘0’. The other changes are minor. The sign bit of input x is flipped and XOR-ed with the sign bit of the ot