DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Note
In regards to 35 USC § 101 - software per se rejection, has been withdrawn in light of the instant amendments to the claims.
Response to Argument
In Remarks/Arguments (pg. 9 – 10) Applicant contends: “ Claims 1, 6, 10 and 18 recite specific structure implemented as hardware circuitry and components, and further recite specific technical features corresponding to the recited hardware circuitry and components. Thus, the claims are not directed to an abstract idea.
For example, claim 1 expressly recites "a decoding unit including a selector and a product- device and a product-sum circuit that performs convolution processing " These are not sum functional placeholders; they correspond to concrete hardware modules...”
Regarding the above argument, the Examiner respectfully disagrees with Applicants contention. The recited claim features are directed to an abstract idea in the form of mental process, even though they are implemented using hardware circuitry. For example, the claim limitation: “decodes a first coefficient matrix encoded into a first zero coefficient position table and a first non-zero coefficient table” recites abstract idea: mental process, which involves decoding a matrix that has been encoded into a zero coefficient table for zero values and a non-zero coefficient table for non-zero values., see Claim Rejections - 35 USC § 101 section for other limitation classified as mental process. The recitation of a decoding unit, product-sum circuit, selector, determination circuits and other hardware components in claim 1, 6 and 10 does not alter this characterization because these components are invokes computers or other machinery merely as a tool to perform the abstract idea, MPEP 2106.05(f).
In Remarks/Arguments (pg. 10) Applicant contends: “These claimed features also solve a technical problem with a technical solution. The claimed apparatus improves neural network hardware efficiency by reducing memory footprint through sparse representation (zero/non-zero tables), eliminating unnecessary multiply-accumulate operations on zeros, and synchronizing coefficient and variable acquisition for optimized hardware pipelines. Further, these improvements are explicitly tied to specific hardware architecture, rather than being generically or abstractly recited.”
Regarding the above argument, the Examiner notes that the claim lacks sufficient technical details to support a conclusion that they recite improvement to neural network hardware. Although the claims assert that they improve neural network hardware efficiency by reducing memory footprint, eliminating multiply-accumulate operations to zeros, and synchronizing data acquisition, the claim do not recite the specific technical details of how the hardware itself is structurally modified to achieve these improvements. Instead, they describe results such as sparse representation, voidance of zero operations and pipeline optimization at a functional level. To determine if the disclosure provides sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. The claim needs to reflect the particular way of achieving that improvement and the specification need not explicitly set forth the improvement, but it must describe the invention such that the improvement would be apparent to one of ordinary skill in the art. Conversely, if the specification explicitly sets forth an improvement but in a conclusory manner (i.e., a bare assertion of an improvement without the detail necessary to be apparent to a person of ordinary skill in the art), the examiner should not determine the claim improves technology. Second, if the specification sets forth an improvement in technology, the claim must be evaluated to ensure that the claim itself reflects the disclosed improvement.,” MPEP 2106.04(d)(1).
In Remarks/Arguments (pg. 11 – 12) Applicant contends: “For example, the relied upon references at least fail to disclose "a decoding unit including a selector and a product-sum device… configured to decode a first coefficient matrix encoded into a first zero coefficient position table and a first non-zero coefficient table," as claimed by Applicant. Neither Woo nor Tanaka discloses reconstructing a coefficient matrix from two separate tables (zero positions + non-zero values). Woo's bitmap/index applies to activations, not weights, and does not rebuild a matrix. Tanaka assigns non-zero elements to calculators but does not teach table-based decoding.”
Regarding the above argument, the Examiner respectfully disagrees with Applicant’s assertion and notes the claim limitation as claimed in light of the teaching of the prior art disclosed in Woo, (col. 7 line [65 – 67] – col. 8 line [1- 12]): using a bitmap to identify zero and non-zero positions and storing only the corresponding non-zero values in memory, which separates positional information (bitmap) from numerical values (sored activations), which is structurally equivalent to two table sparse representation (indices + values). Although described for activations, the mechanism itself teaches the same positional/value separation required to reconstruct a matrix by placing stored values at bitmap-indicated positions and filling remaining positions with zero. In addition, the rejected claim requires only the encoding structure, not any specific application of the encoded matrix, it does not require that the matrix be applied as weights.
In Remarks/Arguments (pg. 12) Applicant contends: “The relied upon references also fail to disclose "the selector outputs a zero when a value input to a control terminal is the first value, and outputs one of the second coefficients the product-sum device restores the first coefficient matrix by arranging zeros and the second coefficients input from the selector," as claimed by Applicant. This hardware-based restoration mechanism is absent from Woo (which omits MACs) and Tanaka (which performs MAC scheduling without selector logic).”
Regarding the above argument, the Examiner respectfully disagrees with Applicant’s assertion and notes the claim limitation as claimed in light of the teaching of the prior art disclosed in Woo, (col. 9 line [38 – 51]): the noted section explains that controller 302 reads bitmaps 303 to identify memory addresses associated with non-zero activation values and also identify addresses corresponding to zero activation values. It further determines corresponding weights and memory addresses for input activations. The demonstrates a hardware-controlled mechanism in which position information (bitmap) is used to identify stored values and coordinate their use during computation. Thus, the noted section describes hardware logic that interprets positional encoding and retrieves corresponding data constitutes a hardware based restoration and mapping mechanism. In addition, Woo, (col. 6 line [58 – 64]), describes: one or more multiply accumulate (MAC) units 304 within compute system 300. It further explains that control signals can cause MAC unit 304 to perform computations using input activations from input activation bus 306 and weights form memory bank 110, producing output activations on output activation bus 308.
In Remarks/Arguments (pg. 12) Applicant contends: “Applicant's claim 1 also recites sequential feeding of zero coefficient position table values to the selector for decoding-a feature not taught by Woo's bitmap or Tanaka's parallel assignment.”
Regarding the above argument, the Examiner respectfully disagrees with Applicant’s assertion and notes the claim limitation as claimed in light of the teaching of the prior art disclosed in Woo, (col. 2 line [17 – 28]): disclosed that activations are accessed from memory address locations associated with an index (e.g.: via the bitmap) and provided onto the data bus, while zero-valued activations may be skipped and not provided. This necessarily requires the controller to read positional/index information in sequence to determine which activation to access and whether to provide or suppress it. Thus, the cited section supports indexed, sequential selection and decoding of position associated data, even if zero valued entries are conditionally skipped.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
a decoding unit, in claim 1, 6 and 10.,
an encoding unit in claim 8.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim(s) 1, 3 – 16 and 18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e. an abstract idea) without significantly more.
In step 1, of the 101-analysis set forth in the MPEP 2106, the examiner has determined
that the following limitations recite a process that, under the broadest reasonable interpretation, falls within one or more statutory categories (processes).
In step 2A prong 1, of the 101-analysis set forth in MPEP 2106, the examiner has determined
that the following limitations recite a process that, under broadest reasonable interpretation, covers
a mental process but for the recitation of generic computer components:
Regarding claim 1:
decode a first coefficient matrix encoded into a first zero coefficient position table and a first non-zero coefficient table,
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves decoding a matrix that has been encoded into a zero coefficient table for zero values and a non-zero coefficient table for non-zero values. See (MPEP 2106.04)).
the first zero coefficient position table indicating positions of first coefficients each having a zero value in the first coefficient matrix by a first value
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves observing entries in a matrix, evaluating whether each entry equals zero and marking the positions that satisfy that condition with an indicator. See (MPEP 2106.04)).
indicating positions of second coefficients each having a non-zero value in the first coefficient matrix by a second value, the first non-zero coefficient table holding the second coefficients in the first coefficient matrix;
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves assigning positions of zero and non-zero coefficients within a matrix using allocated indicator values. See (MPEP 2106.04)).
… decodes the first coefficient matrix by storing the second coefficients stored in the first non-zero coefficient table at the positions on the first zero coefficient position table indicated by the second value;
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves rebuilding a coefficient matrix by placing non-zero coefficients at specified positions according to positional indicators. See (MPEP 2106.04)).
… restores the first coefficient matrix by arranging zeros and the second coefficients input from the selector,
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves reconstructing a matrix by arranging zero and non-zero coefficients in their respective positions. See (MPEP 2106.04)).
If the claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process, but for the recitation of generic computer components, then it falls within the mental process. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 of the 101-analysis, set forth in MPEP 2106, the examiner has determined that
the following additional elements do not integrate this judicial exception into a practical application:
A neural network processing apparatus including: a decoding unit including a selector and a product-sum device, the decoding unit being configured to
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
a product-sum circuit that performs convolution processing on the first coefficient matrix decoded by the decoding unit and a first variable matrix,
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
the selector outputs a zero when a value input to a control terminal is the first value, and outputs one of the second coefficients corresponding to one of the positions on the first zero coefficient position table indicated by the second value when a value input to the control terminal is the second value
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity. See MPEP (2106.05(g))).
… sequentially inputs values constituting the first zero coefficient position table to the selector
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity. See MPEP (2106.05(g))).
In Step 2B of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the
claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception:
Regarding limitation (I and II), recite mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception, see MPEP 2106.05(f).
Regarding limitation (IV), additional elements considered extra/post solution activity, as analyzed above, are activity that are well-understood routine and conventional, specifically: the courts have recognized the computer functions as well‐understood, routine, and conventional functions.
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TL| Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
Regarding limitation (III), additional elements considered extra/post solution activity, as analyzed above, are activity that are well-understood routine and conventional, specifically: the courts have recognized the computer functions as well‐understood, routine, and conventional functions.
Data gathering and outputting: see Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015).
As analyzed above, the additional elements, analyzed above, do not integrate the noted judicial exception into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Regarding claim 3, dependent upon claim 1, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
wherein when sequentially inputting values constituting the first zero coefficient position table to the selector, the decoding unit acquires variables stored at positions on the first variable matrix corresponding to the positions of the values to be input to the selector on the first zero coefficient position table.
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves retrieving variables from specific positions within a matrix based on corresponding positional indicators from another table. See (MPEP 2106.04)).
Regarding claim 4, dependent upon claim 1, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
wherein the decoding unit acquires the second coefficients corresponding to the positions on the first zero coefficient position table indicated by the second value, acquires variables stored at positions on the first variable matrix corresponding to the positions on the first zero coefficient position table indicated by the second value
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves obtaining numerical coefficients and variables from specified positions in matrices based on positional indicators. See (MPEP 2106.04)).
inputs the acquired second coefficients and the acquired variables to the product-sum circuit
The recitation in the additional limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity and well-understood routine and conventional (2106.05(d)).
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TL| Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
The additional limitations as analyze failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above.
the product-sum circuit
Deemed insufficient to transform the judicial exception to a patentable invention because the limitation is directed to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and are considered to adding the words “apply it” (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to using the computer as a tool for implementing an abstract idea cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
performs the convolution processing on the first coefficient matrix and the first variable matrix by sequentially multiplying the second coefficients and the variables input from the decoding unit and adding up multiplication results.
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mathematical concept: It involves performing convolution by sequentially multiplying coefficients and variables and summing the result, which is multiplication and adding of numerical values to compute convolution outputs. See (MPEP 2106.04)).
Regarding claim 5, dependent upon claim 4, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
wherein the decoding unit includes a priority encoder having a plurality of inputs for which priorities are set, respectively, and outputs a value set to an input having a highest priority among one or more inputs to which the second value is input,
The recitation in the additional limitation simply links the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).
Limitations directed to field of use cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
the decoding unit inputs values constituting the first zero coefficient position table to the plurality of inputs in parallel, and
The recitation in the additional limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity and well-understood routine and conventional (2106.05(d)).
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TL| Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
The additional limitations as analyze failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above.
acquires one of the second coefficients from the first non-zero coefficient table based on the value output from the priority encoder with respect to the plurality of inputs.
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves using positional values to retrieve corresponding coefficients based on the output from priority encoder. See (MPEP 2106.04)).
Regarding claim 6:
In step 2A prong 1, of the 101-analysis set forth in MPEP 2106, the Examiner has determined that the following limitations recite a process that, under broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components:
decode a first coefficient matrix encoded into a first zero coefficient position table and a first non-zero coefficient table,
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves decoding a matrix that has been encoded into a zero coefficient table for zero values and a non-zero coefficient table for non-zero values. See (MPEP 2106.04)).
the first zero coefficient position table indicating positions of first coefficients each having a zero value in the first coefficient matrix by a first value
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves observing entries in a matrix, evaluating whether each entry equals zero and marking the positions that satisfy that condition with an indicator. See (MPEP 2106.04)).
indicating positions of second coefficients each having a non-zero value in the first coefficient matrix by a second value, the first non-zero coefficient table holding the second coefficients in the first coefficient matrix;
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves assigning positions of zero and non-zero coefficients within a matrix using allocated indicator values. See (MPEP 2106.04)).
… decodes the first coefficient matrix by storing the second coefficients stored in the first non-zero coefficient table at the positions on the first zero coefficient position table indicated by the second value;
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves rebuilding a coefficient matrix by placing non-zero coefficients at specified positions according to positional indicators. See (MPEP 2106.04)).
the decoding unit performs logical operations on values constituting the first zero coefficient position table and values constituting the first zero variable position table, acquires the second coefficients and the second variables, which do not produce zero when multiplied based on results of the logical operations, from the first non-zero coefficient table and the non-zero variable table, respectively
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mathematical concept: It involves performing logical and arithmetic operations on numerical values from coefficient and variable tables and selecting non-zero results based on these operations. See (MPEP 2106.04)).
If the claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process, but for the recitation of generic computer components, then it falls within the mental process. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 of the 101-analysis, set forth in MPEP 2106, the examiner has determined that
the following additional elements do not integrate this judicial exception into a practical application:
A neural network processing apparatus comprising: a decoding unit including a selector and a product-sum device, the decoding unit being configured to
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
the first variable matrix is encoded into a first zero variable position table and a non-zero variable table, the first zero variable position table indicating positions of first variables each having a zero value in the first variable matrix by the first value and indicating positions of second variables each having a non-zero value in the first variable matrix by the second value, the non- zero variable table holding the second variables in the first variable matrix,
Deemed insufficient to transform the judicial exception to a patentable invention because the additional limitation simply links the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).
inputs the acquired second coefficients and the acquired second variables to the product-sum circuit.
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity. See MPEP (2106.05(g)).
In Step 2B of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the
claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception:
Regarding limitation (I), recite mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception, see MPEP 2106.05(f).
Regarding limitation (II), additional elements are deemed insufficient to transform the judicial exception to a patentable invention to a patentable invention because they generally link the judicial exception to the technology environment, see MPEP 2106.05(h).
Regarding limitation (III), additional elements considered extra/post solution activity, as analyzed above, are activity that are well-understood routine and conventional, specifically: the courts have recognized the computer functions as well‐understood, routine, and conventional functions.
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TL| Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
As analyzed above, the additional elements, analyzed above, do not integrate the noted judicial exception into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Regarding claim 7, dependent upon claim 6, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
wherein the decoding unit includes a priority encoder having a plurality of inputs for which priorities are set, respectively, and outputs a value set to an input having a highest priority among one or more inputs to which the second value is input
The recitation in the additional limitation simply links the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).
Limitations directed to field of use cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
the decoding unit inputs values constituting the first zero coefficient position table to the plurality of inputs in parallel, and acquires one of the second coefficients from the first non-zero coefficient table based on the value output from the priority encoder with respect to the plurality of inputs, and the decoding unit inputs values constituting the first zero variable position table to the plurality of inputs in parallel, and acquires one of the second variables from the non-zero variable table based on the value output from the priority encoder with respect to the plurality of inputs.
The recitation in the additional limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity and well-understood routine and conventional (2106.05(d)).
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TL| Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
The additional limitations as analyze failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above.
Regarding claim 8, dependent upon claim 1, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
an encoding unit that encodes a second variable matrix output from the product-sum circuit into a second zero variable position table and a second non-zero coefficient table
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves encoding a numerical matrix by separating its elements into tables representing zero and non-zero values. See (MPEP 2106.04)).
the second zero variable position table indicating positions of first variables each having a zero value in the second variable matrix by the first value and indicating positions of second variables each having a non- zero value in the second variable matrix by the second value, the second non-zero coefficient table holding the second coefficients in the second variable matrix.
The recitation in the additional limitation simply links the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).
Limitations directed to field of use cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Regarding claim 9, dependent upon claim 8, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
wherein the encoding unit includes: a determination circuit that determines whether or not a value input thereto is zero;
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: metal process: It involves evaluating an input value to determine whether is equals zero. See (MPEP 2106.04)).
a first buffer that stores the first value when the determination circuit determines that the value is zero, and stores the second value when the determination circuit determines that the value is not zero; and a second buffer that stores the second variable when the determination circuit determines that the value is not zero,
The recitation in the additional limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity and well-understood routine and conventional (2106.05(d)).
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
The additional limitations as analyze failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above.
the encoding unit sequentially inputs variables constituting the second variable matrix to the determination circuit.
The recitation in the additional limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity and well-understood routine and conventional (2106.05(d)).
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TL| Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
The additional limitations as analyze failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above.
Regarding claim 10:
In step 2A prong 1, of the 101-analysis set forth in MPEP 2106, the Examiner has determined that the following limitations recite a process that, under broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components:
decode a first coefficient matrix encoded into a first zero coefficient position table and a first non-zero coefficient table,
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves decoding a matrix that has been encoded into a zero coefficient table for zero values and a non-zero coefficient table for non-zero values. See (MPEP 2106.04)).
the first zero coefficient position table indicating positions of first coefficients each having a zero value in the first coefficient matrix by a first value
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves observing entries in a matrix, evaluating whether each entry equals zero and marking the positions that satisfy that condition with an indicator. See (MPEP 2106.04)).
indicating positions of second coefficients each having a non-zero value in the first coefficient matrix by a second value, the first non-zero coefficient table holding the second coefficients in the first coefficient matrix;
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves assigning positions of zero and non-zero coefficients within a matrix using allocated indicator values. See (MPEP 2106.04)).
… decodes the first coefficient matrix by storing the second coefficients stored in the first non-zero coefficient table at the positions on the first zero coefficient position table indicated by the second value;
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves rebuilding a coefficient matrix by placing non-zero coefficients at specified positions according to positional indicators. See (MPEP 2106.04)).
that determines whether or not a value input thereto is zero
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves observing to make a determination whether or not an input value is a zero or not. See (MPEP 2106.04)).
If the claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process, but for the recitation of generic computer components, then it falls within the mental process. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 of the 101-analysis, set forth in MPEP 2106, the examiner has determined that
the following additional elements do not integrate this judicial exception into a practical application:
A neural network processing apparatus comprising: a decoding unit including a selector and a product-sum device, the decoding unit being configured to:
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
a product-sum circuit that performs convolution processing on the first coefficient matrix decoded by the decoding unit and a first variable matrix, wherein the decoding unit
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
an encoding unit that encodes a second variable matrix output from the product-sum circuit into a second zero variable position table and a second non-zero coefficient table
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
a determination circuit…
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
the second zero variable position table indicating positions of first variables each having a zero value in the second variable matrix by the first value and indicating positions of second variables each having a non- zero value in the second variable matrix by the second value, the second non-zero coefficient table holding the second coefficients in the second variable matrix
Deemed insufficient to transform the judicial exception to a patentable invention because the additional limitation simply links the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h))
a first buffer that stores the first value when the determination circuit determines that the value is zero, and stores the second value when the determination circuit determines that the value is not zero; and a second buffer that stores the second variable when the determination circuit determines that the value is not zero, and wherein the encoding unit sequentially inputs variables constituting the second variable matrix to the determination circuit,
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity. See MPEP (2106.05(g)).
encoding unit further includes a register that stores a second zero coefficient position table to be used for convolution processing in a next layer, when a value stored at a position on the second zero coefficient position table corresponding to a position on the second variable matrix of a variable determined to be non-zero by the determination circuit is the first value, the first buffer stores the first value instead of the second value, and when a value stored at a position on the second zero coefficient position table corresponding to a position on the second variable matrix of a variable determined to be non-zero by the determination circuit is the first value, the second buffer does not store the variable.
Deemed insufficient to transform the judicial exception to a patentable invention because the additional limitation simply links the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h))
In Step 2B of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the
claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception:
Regarding limitation (I, II, III and IV), recite mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception, see MPEP 2106.05(f).
Regarding limitation (V and VII), additional elements are deemed insufficient to transform the judicial exception to a patentable invention to a patentable invention because they generally link the judicial exception to the technology environment, see MPEP 2106.05(h).
Regarding limitation (VI), additional elements considered extra/post solution activity, as analyzed above, are activity that are well-understood routine and conventional, specifically: the courts have recognized the computer functions as well‐understood, routine, and conventional functions.
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
As analyzed above, the additional elements, analyzed above, do not integrate the noted judicial exception into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Regarding claim 11, dependent upon claim 10, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
wherein the register stores a third zero coefficient position table obtained by calculating a logical sum of a plurality of second zero coefficient position tables to be used for the convolution processing in the next layer.
The recitation in the additional limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity and well-understood routine and conventional (2106.05(d)).
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
The additional limitations as analyze failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above.
Regarding claim 12, dependent upon claim 11, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
wherein the register stores the plurality of second zero coefficient position tables to be used for the convolution processing in the next layer.
The recitation in the additional limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity and well-understood routine and conventional (2106.05(d)).
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
The additional limitations as analyze failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above.
Regarding claim 13, dependent upon claim 11, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
wherein the plurality of second zero coefficient position tables to be used for the convolution processing in the next layer are grouped into one or more groups, and
The recitation in the additional limitation simply links the judicial exception to a field of use and/or technology environment, see MPEP 2106.05(h).
Limitations directed to field of use cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
the register stores a third zero coefficient position table obtained by calculating a logical sum of the second zero coefficient position tables for each of the groups.
The recitation in the additional limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity and well-understood routine and conventional (2106.05(d)).
Storing and retrieving information in memory, Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015); OIP Techs., 788 F.3d at 1363, 115 USPQ2d at 1092-93;
The additional limitations as analyze failed to integrate a judicial exception into a practical application at Step 2A and provide an inventive concept in Step 2B, per the analysis above.
Regarding claim 14, dependent upon claim 1, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
An information processing apparatus including: the neural network processing apparatus according to claim 1; and a processor core connected to the neural network processing apparatus via a bus.
Deemed insufficient to transform the judicial exception to a patentable invention because the limitation is directed to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and are considered to adding the words “apply it” (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to using the computer as a tool for implementing an abstract idea cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Regarding claim 15, dependent upon claim 14, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
An information processing system including: the information processing apparatus according to claim 14; and one or more sensors connected to the information processing apparatus.
Deemed insufficient to transform the judicial exception to a patentable invention because the limitation is directed to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and are considered to adding the words “apply it” (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to using the computer as a tool for implementing an abstract idea cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Regarding claim 16, dependent upon claim 14, and fail to resolve the deficiencies identified above by integrating the judicial exception into a practical application, or introducing significantly more than the judicial exception. The claim recites:
An electronic device including the information processing apparatus according to claim 14.
Deemed insufficient to transform the judicial exception to a patentable invention because the limitation is directed to mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea and are considered to adding the words “apply it” (or an equivalent) with the judicial exception, See MPEP 2106.05(f).
Limitations directed to using the computer as a tool for implementing an abstract idea cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Regarding claim 18
In step 2A prong 1, of the 101-analysis set forth in MPEP 2106, the Examiner has determined that the following limitations recite a process that, under broadest reasonable interpretation, covers a mental process but for the recitation of generic computer components:
Decoding [ ] a coefficient matrix encoded into a zero coefficient position table and a non-zero coefficient table
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves decoding a matrix that has been encoded into a zero coefficient table for zero values and a non-zero coefficient table for non-zero values. See (MPEP 2106.04)).
the zero coefficient position table indicating positions of first coefficients each having a zero value in the coefficient matrix by a first value and
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves observing entries in a matrix, evaluating whether each entry equals zero and marking the positions that satisfy that condition with an indicator. See (MPEP 2106.04)).
indicating positions of second coefficients each having a non-zero value in the coefficient matrix by a second value, the non-zero coefficient table holding the second coefficients in the coefficient matrix;
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves assigning positions of zero and non-zero coefficients within a matrix using allocated indicator values. See (MPEP 2106.04)).
wherein, the coefficient matrix is decoded by storing the second coefficients stored in the non-zero coefficient table at the positions on the zero coefficient position table indicated by the second value
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves rebuilding a coefficient matrix by placing non-zero coefficients at specified positions according to positional indicators. See (MPEP 2106.04)).
restoring, [ ] the first coefficient matrix by arranging zeros and the second coefficients input from the selector;
(i.e.: the broadest reasonable interpretation, the claim recites abstract idea: mental process: It involves reconstructing a matrix by arranging zero and non-zero coefficients in their respective positions. See (MPEP 2106.04)).
If the claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process, but for the recitation of generic computer components, then it falls within the mental process. Accordingly, the claim recites an abstract idea.
Step 2A Prong 2 of the 101-analysis, set forth in MPEP 2106, the examiner has determined that
the following additional elements do not integrate this judicial exception into a practical application:
A non-transitory computer readable medium storing a program, the program being executable by a processor for causing a computer to execute to perform operations comprising:
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
… by a decoding unit including a selector and a product-sum device …
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
performing, by a product-sum circuit, convolution processing on the coefficient matrix decoded by the decoding processing and a variable matrix,
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
outputting, by the selector, a zero when a value input to a control terminal is the first value, and outputs one of the second coefficients corresponding to one of the positions on the first zero coefficient position table indicated by the second value when a value input to the control terminal is the second value;
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation directed to mere data gathering and outputting as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity. See MPEP (2106.05(g)).
… by the product-sum device, …
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation which does not amount to more than a recitation of the words "apply it" (or an equivalent), such as mere instructions to implement an abstract idea on a computer. See MPEP 2106.05(f).
sequentially inputting, by the decoding unit, values constituting the first zero coefficient position table to the selector.
Deemed insufficient to transform the judicial exception to a patentable invention because the claim recites limitation directed to mere data gathering as deemed insufficient to transform the judicial exception because claimed elements are considered insignificant extra-solution activity. See MPEP (2106.05(g)).
In Step 2B of the 101-analysis set forth in the 2019 PEG, the examiner has determined that the
claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception:
Regarding limitation (I, II, III and V), recite mere application of the abstract idea or mere instructions to implement an abstract idea on a computer are deemed insufficient to transform the judicial exception to a patentable invention because the limitations generally apply the use of a generic computer and/or process with the judicial exception, see MPEP 2106.05(f).
Regarding limitation (IV), additional elements considered extra/post solution activity, as analyzed above, are activity that are well-understood routine and conventional, specifically: the courts have recognized the computer functions as well‐understood, routine, and conventional functions.
Data gathering and outputting: see Mayo, 566 U.S. at 79, 101 USPQ2d at 1968; OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1092-93 (Fed. Cir. 2015).
Regarding limitation (VI), additional elements considered extra/post solution activity, as analyzed above, are activity that are well-understood routine and conventional, specifically: the courts have recognized the computer functions as well‐understood, routine, and conventional functions.
Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TL| Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016) (using a telephone for image transmission); OIP Techs., Inc., v. Amazon.com, Inc., 788 F.3d 1359, 1363, 115 USPQ2d 1090, 1093 (Fed. Cir. 2015) (sending messages over a network); buySAFE, Inc. v. Google, Inc., 765 F.3d 1350, 1355, 112 USPQ2d 1093, 1096 (Fed. Cir. 2014) (computer receives and sends information over a network). See MPEP 2106.05(d)(II).
As analyzed above, the additional elements, analyzed above, do not integrate the noted judicial exception into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Therefore, the claim is directed to an abstract idea.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 3, 14 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Woo et al. Pub. No.: US9818059B1 in view of Tanaka et al., Pub. No.: US11281746B2.
Regarding claim 1, Woo teaches: A neural network processing apparatus including:
(Woo, (col. 3 [59 – 64]), “The subject matter described in this specification relates to reducing computations that occur within a compute unit or tile of an example neural network hardware computing system [A neural network processing apparatus]. In general, as part of computing a neural network inference, an input activation is multiplied with a parameter or weight value to produce an output activation.”)
a decoding unit including [ ] and a product-sum device, the decoding unit being configured to
(Woo, (col. 1 [34 – 42]), “One way of computing convolution calculations requires numerous matrix multiplications in a large dimensional space. A processor or controller device of a compute unit can compute matrix multiplications through a brute force method. For example, although compute-intensive and time-intensive, the processor can repeatedly calculate individual sums and products for convolution calculations [a decoding unit including [ ] and a product-sum device, the decoding unit being configured to]. The degree to which the processor parallelizes calculations is limited due to its architecture.”)
… a selector …
(Woo, (col.9 line [38 – 39]), “As discussed above, controller 302 [a selector] identifies memory addresses for non-zero activation values based, in part, on bitmap 303.”)
decode a first coefficient matrix encoded into a first zero coefficient position table and a first non-zero coefficient table,
(Woo, (col. 7 line [65 – 67] – col. 8 line [1 – 12]), “In some implementations, controller 302 uses bitmap 303 to determine which input activations to write to memory bank 108. For example, analysis of bitmap 303 can indicate that only activation values corresponding to bitmap positions 1, 3, 5, 7 (non-zero values) [and a first non-zero coefficient table] should be written to address locations in memory bank 108. Moreover, data values associated with bitmap positions 2, 4, 6, 8 (zero values) [a decoding unit that decodes a first coefficient matrix encoded into a first zero coefficient position table] can either be discarded or written to memory address locations which may or may not be accessed by controller 302 when activation values are provided to input bus 306. Thus, bitmap 303 can be used as a basis to compress zero activation values in which compression occurs when zero value input activations are not written to memory address locations, thereby reducing the overall memory usage and freeing address locations for storing other data values.”)
the first zero coefficient position table indicating positions of first coefficients each having a zero value in the first coefficient matrix by a first value and indicating positions of second coefficients each having a non-zero value in the first coefficient matrix by a second value,
(Woo, (col. 7 line [45 – 54]), “As shown in FIG. 3, bitmap 303 can use binary values to map detected zero value input activations and detected non-zero value input activations. For example, a binary value of “0” can correspond to a detected zero input activation value [the first zero coefficient position table indicating positions of first coefficients each having a zero value in the first coefficient matrix by a first value] and a binary value of “1” can correspond to a detected non-zero input activation value [and indicating positions of second coefficients each having a non-zero value in the first coefficient matrix by a second value]. For example, bitmap 303 can be an 8-bit bitmap in which odd numbered bit positions that include a binary “1” correspond to non-zero activation values and even numbered bit positions that include a binary “0” correspond to zero activation values.”)
the first non-zero coefficient table holding the second coefficients in the first coefficient matrix; and
(Woo, (col. 7 line [55 – 64]), “Controller 302 can cause input activations to be stored in memory bank 108. In general, data values stored in memory bank 108 are typically each written to a respective memory address location. The address location in memory bank 108 can then be accessed by an example control device (e.g., controller 302) when a data value such as an input activation is needed to perform a particular compute operation. Controller 302 can use bitmap 303 to create an index of memory address locations that include non-zero input activation values [the first non-zero coefficient table holding the second coefficients in the first coefficient matrix].”)
wherein the decoding unit decodes the first coefficient matrix by storing the second coefficients stored in the first non-zero coefficient table at the positions on the first zero coefficient position table indicated by the second value.
(Woo, (col. 7 line [55 – 64]), “Controller 302 can cause input activations to be stored in memory bank 108. In general, data values stored in memory bank 108 are typically each written to a respective memory address location. The address location in memory bank 108 can then be accessed by an example control device (e.g., controller 302) when a data value such as an input activation is needed to perform a particular compute operation. Controller 302 can use bitmap 303 to create an index of memory address locations that include non-zero input activation values [wherein the decoding unit decodes the first coefficient matrix by storing the second coefficients stored in the first non-zero coefficient table at the positions on the first zero coefficient position table indicated by the second value].”)
a selector that outputs a zero when a value input to a control terminal is the first value, and outputs one of the second coefficients corresponding to one of the positions on the first zero coefficient position table indicated by the second value when a value input to the control terminal is the second value; and
(Woo, (col.9 line [38 – 51]), “As discussed above, controller 302 [a selector] identifies memory addresses for non-zero activation values based, in part, on bitmap 303. In some implementations, controller 302 reads bitmap 303 and determines, for example, at least two memory addresses that have non-zero activation values [outputs one of the second coefficients corresponding to one of the positions on the first zero coefficient position table indicated by the second value when a value input to the control terminal is the second value]. If controller 302 is configured to provide, and subsequently skip or disable computes for, zero activation values, then controller 302 may also determine at least one memory address that has a zero activation value [that outputs a zero when a value input to a control terminal is the first value]. In this implementation, controller 302 can reference the above mentioned registers to determine a corresponding weight (and memory address) for the first input activation and to determine a corresponding weight (and memory address) for the second input activation.”)
… that restores the first coefficient matrix by arranging the zeros and the second coefficients input from the selector,
(Woo, (col. 7 line [65 – 67] – col. 8 line [1 – 12]), “In some implementations, controller 302 uses bitmap 303 to determine which input activations to write to memory bank 108. For example, analysis of bitmap 303 can indicate that only activation values corresponding to bitmap positions 1, 3, 5, 7 (non-zero values) [and the second coefficients input from the selector] should be written to address locations in memory bank 108. Moreover, data values associated with bitmap positions 2, 4, 6, 8 (zero values) can either be discarded or written to memory address locations [… that restores the first coefficient matrix by arranging the zeros] which may or may not be accessed by controller 302 when activation values are provided to input bus 306. Thus, bitmap 303 can be used as a basis to compress zero activation values in which compression occurs when zero value input activations are not written to memory address locations, thereby reducing the overall memory usage and freeing address locations for storing other data values.”)
and the decoding unit sequentially inputs values constituting the first zero coefficient position table to the selector.
(Woo, (col. 2 line [7 – 16]), “In some implementations, the method further comprises, mapping, by the controller and to a first unit, a first portion of a tensor computation that uses a first input activation and mapping, to a second unit that differs from the first unit, a second portion of the tensor computation that also uses the first input activation. In some implementations, the method further comprises, sequentially providing a single input activation onto the data bus [the decoding unit sequentially inputs values], the single input activation being accessed and selected from memory address locations that are associated with the index. In some implementations, providing further comprises, not providing input activations that have a zero value [constituting the first zero coefficient position table to the selector].”)
Woo does not teach:
a product-sum circuit that performs convolution processing on the first coefficient matrix decoded by the decoding unit and a first variable matrix
Tanaka teaches:
a product-sum circuit that performs convolution processing on the first coefficient matrix decoded by the decoding unit and a first variable matrix,
(Tanaka, (col. 4 line [65 – 67] – col. 5 line [1 – 5]), “Generally, a matrix operation performed in the convolutional operation [that performs convolution processing] is expressed by a formula (1). That is, an output vector f of the convolutional operation is obtained by multiplying coefficient matrix A [on the first coefficient matrix decoded by the decoding unit] by input vector x from the right, and adding a bias vector b [and a first variable matrix] to an arithmetic operation result. Here, a feature of coefficient matrix A lies in that coefficient matrix A includes a comparatively large number of elements each having a value of 0.”)
Tanaka and Woo are related to the same field of endeavor (i.e.: neural network optimization). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to combine the teaching of Tanaka with teachings of Woo to efficiently perform convolution through structured matrix operations rather than just storing and retrieving activations (Tanaka, Abstract).
Regarding claim 3, Woo in view of Tanaka teach the method of claim 1.
Woo further teaches: wherein when sequentially inputting values constituting the first zero coefficient position table to the selector, the decoding unit acquires variables stored at positions on the first variable matrix corresponding to the positions of the values to be input to the selector on the first zero coefficient position table.
(Woo, (col. 2 line [7 – 16]), “In some implementations, the method further comprises, mapping, by the controller and to a first unit, a first portion of a tensor computation that uses a first input activation and mapping, to a second unit that differs from the first unit, a second portion of the tensor computation that also uses the first input activation. In some implementations, the method further comprises, sequentially providing a single input activation onto the data bus [wherein when sequentially inputting values constituting the first zero coefficient position table to the selector], the single input activation being accessed and selected from memory address locations that are associated with the index [the decoding unit acquires variables stored at positions on the first variable matrix corresponding to the positions of the values to be input]. In some implementations, providing further comprises, not providing input activations that have a zero value [to the selector on the first zero coefficient position table].”)
Regarding claim 14, Woo in view of Tanaka teach the method of claim 1.
Woo further teaches: An information processing apparatus including: the neural network processing apparatus according to claim 1; and a processor core connected to the neural network processing apparatus
(Woo, (col. 6 line [65 – 67] – col. 7 line [1 – 3]), “Controller 302 can include one or more processing units and memory. In some embodiments, processing units of controller 302 [a processor core connected to the neural network processing apparatus] can include one or more processors (e.g., microprocessors or central processing units (CPUs)), graphics processing units (GPUs), application specific integrated circuits (ASICs), or a combination of different processors.”)
via a bus (Woo, data bus 306).
Regarding claim 18, Woo teaches: A non-transitory computer readable medium storing a program, the program being executable by a processor to perform operations comprising: by a decoding unit including a selector and
(Woo, (col. 7 line [9 – 21]), “In some implementations, processing unit(s) of controller 302 executes programmed instructions stored in memory to cause controller 302 [by a decoding unit including a selector] and compute system 300 to perform one or more functions described in this specification. The memory of controller 302 can include one or more non-transitory machine-readable storage mediums. The non-transitory machine-readable storage medium [A non-transitory computer readable medium storing a program, the program being executable by a processor for causing a computer to execute to perform operations comprising:] can include solid-state memory, magnetic disk, and optical disk, a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (e.g., EPROM, EEPROM, or Flash memory), or any other tangible medium capable of storing information.”)
a product-sum device,
(Woo, (col. 1 [34 – 42]), “One way of computing convolution calculations requires numerous matrix multiplications in a large dimensional space. A processor or controller device of a compute unit can compute matrix multiplications through a brute force method. For example, although compute-intensive and time-intensive, the processor can repeatedly calculate individual sums and products for convolution calculations [a product-sum device]. The degree to which the processor parallelizes calculations is limited due to its architecture.”)
decoding [ ] a coefficient matrix encoded into a zero coefficient position table and a non- zero coefficient table,
(Woo, (col. 7 line [65 – 67] – col. 8 line [1 – 12]), “In some implementations, controller 302 uses bitmap 303 to determine which input activations to write to memory bank 108. For example, analysis of bitmap 303 can indicate that only activation values corresponding to bitmap positions 1, 3, 5, 7 (non-zero values) [a non- zero coefficient table] should be written to address locations in memory bank 108. Moreover, data values associated with bitmap positions 2, 4, 6, 8 (zero values) [decoding processing for decoding a coefficient matrix encoded into a zero coefficient position table] can either be discarded or written to memory address locations which may or may not be accessed by controller 302 when activation values are provided to input bus 306. Thus, bitmap 303 can be used as a basis to compress zero activation values in which compression occurs when zero value input activations are not written to memory address locations, thereby reducing the overall memory usage and freeing address locations for storing other data values.”)
the zero coefficient position table indicating positions of first coefficients each having a zero value in the coefficient matrix by a first value and indicating positions of second coefficients each having a non-zero value in the coefficient matrix by a second value, the non-zero coefficient table holding the second coefficients in the coefficient matrix; and
(Woo, (col. 7 line [45 – 54]), “As shown in FIG. 3, bitmap 303 can use binary values to map detected zero value input activations and detected non-zero value input activations. For example, a binary value of “0” can correspond to a detected zero input activation value [the zero coefficient position table indicating positions of first coefficients each having a zero value in the coefficient matrix by a first value] and a binary value of “1” can correspond to a detected non-zero input activation value [and indicating positions of second coefficients each having a non-zero value in the coefficient matrix by a second value, the non-zero coefficient table holding the second coefficients in the coefficient matrix; and]. For example, bitmap 303 can be an 8-bit bitmap in which odd numbered bit positions that include a binary “1” correspond to non-zero activation values and even numbered bit positions that include a binary “0” correspond to zero activation values.”)
wherein, the coefficient matrix is decoded by storing the second coefficients stored in the non-zero coefficient table at the positions on the zero coefficient position table indicated by the second value.
(Woo, (col. 7 line [55 – 64]), “Controller 302 can cause input activations to be stored in memory bank 108. In general, data values stored in memory bank 108 are typically each written to a respective memory address location. The address location in memory bank 108 can then be accessed by an example control device (e.g., controller 302) when a data value such as an input activation is needed to perform a particular compute operation. Controller 302 can use bitmap 303 to create an index of memory address locations that include non-zero input activation values [wherein, in the decoding processing, the coefficient matrix is decoded by storing the second coefficients stored in the non-zero coefficient table at the positions on the zero coefficient position table indicated by the second value].”
outputting, by the selector, a zero when a value input to a control terminal is the first value, and outputs one of the second coefficients corresponding to one of the positions on the first zero coefficient position table indicated by the second value when a value input to the control terminal is the second value;
(Woo, (col.9 line [38 – 51]), “As discussed above, controller 302 [… by the selector …] identifies memory addresses for non-zero activation values based, in part, on bitmap 303. In some implementations, controller 302 reads bitmap 303 and determines, for example, at least two memory addresses that have non-zero activation values [outputs one of the second coefficients corresponding to one of the positions on the first zero coefficient position table indicated by the second value when a value input to the control terminal is the second value]. If controller 302 is configured to provide, and subsequently skip or disable computes for, zero activation values, then controller 302 may also determine at least one memory address that has a zero activation value [outputting [ ] a zero when a value input to a control terminal is the first value]. In this implementation, controller 302 can reference the above mentioned registers to determine a corresponding weight (and memory address) for the first input activation and to determine a corresponding weight (and memory address) for the second input activation.”)
the product-sum device restores the first coefficient matrix by arranging zeros and the second coefficients input from the selector,
(Woo, (col. 7 line [65 – 67] – col. 8 line [1 – 12]), “In some implementations, controller 302 uses bitmap 303 to determine which input activations to write to memory bank 108. For example, analysis of bitmap 303 can indicate that only activation values corresponding to bitmap positions 1, 3, 5, 7 (non-zero values) [and the second coefficients input from the selector,] should be written to address locations in memory bank 108. Moreover, data values associated with bitmap positions 2, 4, 6, 8 (zero values) can either be discarded or written to memory address locations [ the product-sum device restores the first coefficient matrix by arranging zeros] which may or may not be accessed by controller 302 when activation values are provided to input bus 306. Thus, bitmap 303 can be used as a basis to compress zero activation values in which compression occurs when zero value input activations are not written to memory address locations, thereby reducing the overall memory usage and freeing address locations for storing other data values.”)
sequentially inputting, by the decoding unit, values constituting the first zero coefficient position table to the selector
(Woo, (col. 2 line [7 – 16]), “In some implementations, the method further comprises, mapping, by the controller and to a first unit, a first portion of a tensor computation that uses a first input activation and mapping, to a second unit that differs from the first unit, a second portion of the tensor computation that also uses the first input activation. In some implementations, the method further comprises, sequentially providing a single input activation onto the data bus [sequentially inputting, by the decoding unit], the single input activation being accessed and selected from memory address locations that are associated with the index. In some implementations, providing further comprises, not providing input activations that have a zero value [values constituting the first zero coefficient position table to the selector].”)
Woo does not teach:
performing, by a product-sum circuit, convolution processing on the coefficient matrix decoded by the decoding processing and a variable matrix,
Tanaka teaches:
performing, by a product-sum circuit, convolution processing on the coefficient matrix decoded by the decoding processing and a variable matrix
(Tanaka, (col. 4 line [65 – 67] – col. 5 line [1 – 5]), “Generally, a matrix operation performed in the convolutional operation [performing, by a product-sum circuit, convolution processing] is expressed by a formula (1). That is, an output vector f of the convolutional operation is obtained by multiplying coefficient matrix A [on the coefficient matrix decoded by the decoding processing] by input vector x from the right, and adding a bias vector b [and a variable matrix] to an arithmetic operation result. Here, a feature of coefficient matrix A lies in that coefficient matrix A includes a comparatively large number of elements each having a value of 0.”)
Tanaka and Woo are related to the same field of endeavor (i.e.: neural network optimization). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to combine the teaching of Tanaka with teachings of Woo to efficiently perform convolution through structured matrix operations rather than just storing and retrieving activations (Tanaka, Abstract).
Claim(s) 4 and 8 – 9 are rejected under 35 U.S.C. 103 as being unpatentable over Woo in view of Tanaka and in further view of Dally et al., Pub. No.: US20180046900A1.
Regarding claim 4, Woo in view of Tanaka teach the method of claim 1.
Woo in view of Tanaka do not teach:
wherein the decoding unit acquires the second coefficients corresponding to the positions on the first zero coefficient position table indicated by the second value, acquires variables stored at positions on the first variable matrix corresponding to the positions on the first zero coefficient position table indicated by the second value, and inputs the acquired second coefficients and the acquired variables to the product-sum circuit, and the product-sum circuit performs the convolution processing on the first coefficient matrix and the first variable matrix by sequentially multiplying the second coefficients and the variables input from the decoding unit and adding up multiplication results.
Dally teaches:
wherein the decoding unit acquires the second coefficients corresponding to the positions on the first zero coefficient position table indicated by the second value, acquires variables stored at positions on the first variable matrix corresponding to the positions on the first zero coefficient position table indicated by the second value, and inputs the acquired second coefficients and the acquired variables to the product-sum circuit, and the product-sum circuit performs the convolution processing on the first coefficient matrix and the first variable matrix by sequentially multiplying the second coefficients and the variables input from the decoding unit and adding up multiplication results.
(Dally, “[0117] In one embodiment, the SCNN 200 uses a simple compressed-sparse encoding approach based on a run-length encoding scheme. A data vector may be extracted from the compressed-sparse encoded data [inputs the acquired second coefficients and the acquired variables to the product-sum circuit], where the data vector is a sequence non-zero values. An index vector may be extracted from the compressed-sparse encoded data, where the index vector is a sequence of zero-counts (the number of zeros between each non-zero element) [the decoding unit acquires the second coefficients corresponding to the positions on the first zero coefficient position table indicated by the second value]. For example, a compressed-space encoding of the data shown in FIG. 3B is (a, b, c, d, e, f) and (2, 0, 3, 4, 1, 1) representing a data vector and a corresponding index vector, where each element in the index vector is a number of zeros preceding the corresponding non-zero element.
[0118] Determining the coordinates of a location in the accumulator array 340 [acquires variables stored at positions on the first variable matrix corresponding to the positions on the first zero coefficient position table indicated by the second value] for each product output by a multiplier in the F×I multiplier array 325 requires reading the index vectors for F and I and combining the index vectors with the coordinates of a portion of the output activation space currently being processed [the product-sum circuit performs the convolution processing on the first coefficient matrix and the first variable matrix by sequentially multiplying the second coefficients and the variables input from the decoding unit and adding up multiplication results]. Four bits per index allows for up to 15 zeros to appear between any two non-zero elements. When more than 15 zeros occur between two non-zero elements, a zero-value placeholder (i.e., zero pad) is inserted as an intervening non-zero element without incurring any noticeable degradation in compression efficiency. With an expected non-zero element density of 30% there will be approximately 2 zeros between non-zero elements on average.”)
Dally, Woo and Tanaka are related to the same field of endeavor (i.e.: neural network optimization). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to combine the teaching of Dally with teachings of Woo and Tanaka to add computational stages were coordinate sets from different indexed arrays are summed and converted into linear indices to enable efficient mapping and processing of sparse activation. (Dally, Abstract).
Regarding claim 8, Woo in view of Tanaka teach the method of claim 1.
Woo further teaches: an encoding unit that encodes a second variable matrix output from the product-sum circuit into a second zero variable position table and
(Woo, (col. 7 line [65 – 67] – col. 8 line [1 – 12]), “In some implementations, controller 302 uses bitmap 303 to determine which input activations to write to memory bank 108. For example, analysis of bitmap 303 can indicate that only activation values corresponding to bitmap positions 1, 3, 5, 7 (non-zero values) should be written to address locations in memory bank 108. Moreover, data values associated with bitmap positions 2, 4, 6, 8 (zero values) can either be discarded or written to memory address locations [an encoding unit that encodes a second variable matrix output from the product-sum circuit into a second zero variable position table] which may or may not be accessed by controller 302 when activation values are provided to input bus 306. Thus, bitmap 303 can be used as a basis to compress zero activation values in which compression occurs when zero value input activations are not written to memory address locations, thereby reducing the overall memory usage and freeing address locations for storing other data values.”)
Woo in view of Tanaka do not teach:
a second non-zero coefficient table, the second zero variable position table indicating positions of first variables each having a zero value in the second variable matrix by the first value and indicating positions of second variables each having a non- zero value in the second variable matrix by the second value, the second non-zero coefficient table holding the second coefficients in the second variable matrix.
Dally teaches:
a second non-zero coefficient table, the second zero variable position table indicating positions of first variables each having a zero value in the second variable matrix by the first value and indicating positions of second variables each having a non- zero value in the second variable matrix by the second value, the second non-zero coefficient table holding the second coefficients in the second variable matrix.
(Dally, “[0117] In one embodiment, the SCNN 200 uses a simple compressed-sparse encoding approach based on a run-length encoding scheme. A data vector may be extracted from the compressed-sparse encoded data, where the data vector is a sequence non-zero values. An index vector may be extracted from the compressed-sparse encoded data, where the index vector is a sequence of zero-counts (the number of zeros [a second non-zero coefficient table, the second zero variable position table indicating positions of first variables each having a zero value in the second variable matrix by the first value] between each non-zero element [indicating positions of second variables each having a non- zero value in the second variable matrix by the second value, the second non-zero coefficient table holding the second coefficients in the second variable matrix]). For example, a compressed-space encoding of the data shown in FIG. 3B is (a, b, c, d, e, f) and (2, 0, 3, 4, 1, 1) representing a data vector and a corresponding index vector, where each element in the index vector is a number of zeros preceding the corresponding non-zero element.”)
Dally, Woo and Tanaka are related to the same field of endeavor (i.e.: neural network optimization). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to combine the teaching of Dally with teachings of Woo and Tanaka to add computational stages were coordinate sets from different indexed arrays are summed and converted into linear indices to enable efficient mapping and processing of sparse activation. (Dally, Abstract).
Regarding claim 9, Woo in view of Tanaka and Dally teach the method of claim 8.
Woo further teaches: wherein the encoding unit includes: a determination circuit that determines whether or not a value input thereto is zero;
(Woo, (col. 7 line [33 – 44]), “In some implementations, system 300 receives instructions that define a particular compute operation to be performed by system 300. Moreover, controller 302 can execute programed instructions to, for example, analyze a data stream associated with the received input activations. Analyzing the input activation data stream can enable controller 302 to detect or determine whether a value associated with each of the input activations is a zero value or a non-zero value [a determination circuit that determines whether or not a value input thereto is zero]. In some implementations, controller 302 analyzes an example input activation data stream and maps each detected zero activation value and each detected non-zero activation value to bitvector or bitmap 303.”)
a first buffer that stores the first value when the determination circuit determines that the value is zero, and stores the second value when the determination circuit determines that the value is not zero; and
(Woo, (col. 7 line [65 – 67] – col. 8 line [1 – 12]), “In some implementations, controller 302 uses bitmap 303 to determine which input activations to write to memory bank 108 [a first buffer that stores]. For example, analysis of bitmap 303 can indicate that only activation values corresponding to bitmap positions 1, 3, 5, 7 (non-zero values) [and stores the second value when the determination circuit determines that the value is not zero] should be written to address locations in memory bank 108. Moreover, data values associated with bitmap positions 2, 4, 6, 8 (zero values) [the first value when the determination circuit determines that the value is zero,] can either be discarded or written to memory address locations which may or may not be accessed by controller 302 when activation values are provided to input bus 306. Thus, bitmap 303 can be used as a basis to compress zero activation values in which compression occurs when zero value input activations are not written to memory address locations, thereby reducing the overall memory usage and freeing address locations for storing other data values.”)
a second buffer that stores the second variable when the determination circuit determines that the value is not zero, and
(Woo, (col. 7 line [65 – 67] – col. 8 line [1 – 12]), “In some implementations, controller 302 uses bitmap 303 to determine which input activations to write to memory bank 108. For example, analysis of bitmap 303 can indicate that only activation values corresponding to bitmap positions 1, 3, 5, 7 (non-zero values) [the second variable when the determination circuit determines that the value is not zero] should be written to address locations in memory bank 108 [a second buffer that stores]. Moreover, data values associated with bitmap positions 2, 4, 6, 8 (zero values) can either be discarded or written to memory address locations which may or may not be accessed by controller 302 when activation values are provided to input bus 306. Thus, bitmap 303 can be used as a basis to compress zero activation values in which compression occurs when zero value input activations are not written to memory address locations, thereby reducing the overall memory usage and freeing address locations for storing other data values.”)
the encoding unit sequentially inputs variables constituting the second variable matrix to the determination circuit.
(Woo, (col. 2 line [17 – 28]), “In some implementations, the method further comprises, mapping, by the controller and to a first unit, a first portion of a tensor computation that uses a first input activation and mapping, to a second unit that differs from the first unit, a second portion of the tensor computation that also uses the first input activation. In some implementations, the method further comprises, sequentially providing a single input activation onto the data bus [the encoding unit sequentially inputs variables constituting the second variable matrix to the determination circuit], the single input activation being accessed and selected from memory address locations that are associated with the index. In some implementations, providing further comprises, not providing input activations that have a zero value.”)
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Woo in view of Tanaka, Dally and in further view of Lee et al., Pub. No.: US11966835B2.
Woo in view of Tanaka and Dally teach the method of claim 4.
Dally further teaches: the decoding unit inputs values constituting the first zero coefficient position table to the plurality of inputs in parallel, and acquires one of the second coefficients from the first non-zero coefficient table based on the value output from the priority encoder with respect to the plurality of inputs.
(Dally, “[0117] In one embodiment, the SCNN 200 uses a simple compressed-sparse encoding approach based on a run-length encoding scheme. A data vector may be extracted from the compressed-sparse encoded data [the decoding unit inputs values constituting the first zero coefficient position table to the plurality of inputs in parallel], where the data vector is a sequence non-zero values. An index vector may be extracted from the compressed-sparse encoded data, where the index vector is a sequence of zero-counts (the number of zeros between each non-zero element) [acquires one of the second coefficients from the first non-zero coefficient table based on the value output from the priority encoder with respect to the plurality of inputs]. For example, a compressed-space encoding of the data shown in FIG. 3B is (a, b, c, d, e, f) and (2, 0, 3, 4, 1, 1) representing a data vector and a corresponding index vector, where each element in the index vector is a number of zeros preceding the corresponding non-zero element.”)
It would have been obvious to one of ordinary skill in the art before the effective filling date of the present application to combine the teachings of Dally with teachings of Woo and Tanaka for the same reasons disclosed for claim 4.
Woo in view of Tanaka and Dally do not teach:
wherein the decoding unit includes a priority encoder having a plurality of inputs for which priorities are set, respectively, and outputs a value set to an input having a highest priority among one or more inputs to which the second value is input,
Lee teaches:
wherein the decoding unit includes a priority encoder having a plurality of inputs for which priorities are set, respectively, and outputs a value set to an input having a highest priority among one or more inputs to which the second value is input, and
(Lee, (col. 4 line [18 – 35]), “The column coordinate index 304 and the row coordinate index 306 receive the channel indices of weights and input activations as inputs, respectively. The comparator array 302 compares the column coordinate index 304 to the row coordinate index 306. For example, the left-most column of the comparator array 302 compares the first entry of the column coordinate index 304 to each of the entries of the row coordinate index 306. Each comparator may signal a “1” or “0” based on whether the column and row indices match or do not match, respectively. Each column of the outputs of the comparator array 302 are sent to the array of priority encoders 308 [wherein the decoding unit includes a priority encoder having a plurality of inputs for which priorities are set, respectively, and outputs a value set to an input having a highest priority among one or more inputs to which the second value is input], which identifies matching indices from weights and inputs. Each priority encoder of the array of priority encoders 308 generates a vector of C row addresses, and an additional valid bit per address to indicate whether a match is found or not, resulting in a priority encoded sequence of (log2(C)+1) bits.”)
Lee, Woo, Tanaka and Dally are related to the same field of endeavor (i.e.: neural network optimization). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to combine the teaching of Lee with teachings of Woo, Tanaka and Dally to enable efficient sparse convolution processing by detecting and exploiting fine-grained parallelism between compacted activation and weight arrays. (Lee, Abstract).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Woo in view of Tanaka and in further view of Li et al., Pub. No.: US20200159534A1.
Woo in view of Tanaka teach the method of claim 14.
Woo in view of Tanaka do not teach:
An information processing system including: the information processing apparatus according to claim 14; and one or more sensors connected to the information processing apparatus.
Li teaches:
An information processing system including: the information processing apparatus according to claim 14; and one or more sensors connected to the information processing apparatus.
(Li, “[0037] In some embodiments the platform controller hub 130 enables peripherals to connect to memory device 120 and processor 102 via a high-speed I/O bus. The I/O peripherals include, but are not limited to, an audio controller 146, a network controller 134, a firmware interface 128, a wireless transceiver 126, touch sensors 125, a data storage device 124 (e.g., hard disk drive, flash memory, etc.). The data storage device 124 can connect via a storage interface (e.g., SATA) or via a peripheral bus, such as a Peripheral Component Interconnect bus (e.g., PCI, PCI Express). The touch sensors 125 can include touch screen sensors, pressure sensors, or fingerprint sensors [the information processing apparatus according to claim 14; and one or more sensors connected to the information processing apparatus]. The wireless transceiver 126 can be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver such as a 3G, 4G, or Long-Term Evolution (LTE) transceiver.”)
Li, Woo and Tanaka are related to the same field of endeavor (i.e.: neural network optimization). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the invention to combine the teaching of Li with teachings of Woo and Tanaka to enable computing unit to perform multiple operations efficiently in parallel, to optimize the execution of indexed, non-zero activation computations. (Li, Abstract).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Woo in view of Tanaka and Li.
Woo in view of Tanaka and Li teach the method of claim 14
Woo further teaches: An electronic device including the information processing apparatus according to claim 14.
(Woo, data bus 306 [An electronic device including the information processing apparatus according to claim 14]).
Allowable Subject Matter
Claim(s) 6 – 7, 10 – 13 would be allowable if rewritten or amended to overcome the rejection under 35 U.S.C. 101 set forth in this Office action. The prior art made of record does not teach, make obvious, or suggest the claim limitations as disclosed in applicant's claims.
Claim 6 recites:
a decoding unit including a selector and a product-sum device, the decoding unit being configured to decode a first coefficient matrix encoded into a first zero coefficient position table and a first non-zero coefficient table, the first zero coefficient position table indicating positions of first coefficients each having a zero value in the first coefficient matrix by a first value and indicating positions of second coefficients each having a non-zero value in the first coefficient matrix by a second value, the first non-zero coefficient table holding the second coefficients in the first coefficient matrix; and
a product-sum circuit that performs convolution processing on the first coefficient matrix decoded by the decoding unit and a first variable matrix, wherein the decoding unit decodes the first coefficient matrix by storing the second coefficients stored in the first non-zero coefficient table at the positions on the first zero coefficient position table indicated by the second value,
the first variable matrix is encoded into a first zero variable position table and a non-zero variable table, the first zero variable position table indicating positions of first variables each having a zero value in the first variable matrix by the first value and indicating positions of second variables each having a non-zero value in the first variable matrix by the second value, the non- zero variable table holding the second variables in the first variable matrix, and the decoding unit performs logical operations on values constituting the first zero coefficient position table and values constituting the first zero variable position table, acquires the second coefficients and the second variables, which do not produce zero when multiplied based on results of the logical operations, from the first non-zero coefficient table and the non-zero variable table, respectively, and inputs the acquired second coefficients and the acquired second variables to the product-sum circuit.
Closest prior arts:
Woo et al. Pub. No.: US9818059B1.
Woo describes a computing device receiving multiple input signals from an outside source. The device's controller checks if each signal is zero or non-zero. The method goes on to store at least one of these signals in the device's memory, creating an index that marks the addresses of non-zero values. The controller then sends at least one non-zero signal to a data bus, which can be accessed by other parts of the computer. However Woo does not teach a first variable matrix is encoded by separating zero and non-zero elements into two tables: one indicating the positions of zeros and non-zeros, and another storing only the non-zero values. The decoding unit performs logical operations between the first zero coefficient position table and the first zero variable position table to determine which coefficient-variable pairs will produce non-zero results, retrieves the corresponding second coefficient and second variable from the non-zero coefficient and variable tables, and input them to the product-sum circuit for computation.
Tanaka et al., Pub. No.: US11281746B2.
Tanaka describes an arithmetic operation method for a convolutional layer in a neural network involves several steps. First, a coefficient matrix is created from the kernel used in the convolutional layer. This matrix is linked to an input vector formed by turning a feature map into one column. Next, non-zero elements in the coefficient matrix are identified. Then, multiplications of these non-zero elements with related elements of the input vector are assigned to multiple calculators. However Tanaka does not teach a first variable matrix is encoded by separating zero and non-zero elements into two tables: one indicating the positions of zeros and non-zeros, and another storing only the non-zero values. The decoding unit performs logical operations between the first zero coefficient position table and the first zero variable position table to determine which coefficient-variable pairs will produce non-zero results, retrieves the corresponding second coefficient and second variable from the non-zero coefficient and variable tables, and input them to the product-sum circuit for computation.
Dally et al., Pub. No.: US20180046900A1
Dally describes a first instruction with two index vector operands is received. The first index vector is decoded to create first coordinate sets for a first array, identifying positions of non-zero elements. The second index vector is decoded to create second coordinate sets for a second array, also identifying positions of non-zero elements. The first coordinate sets are then summed with the second coordinate sets to generate output coordinate sets, which are converted into linear indices. However Dally does not teach a first variable matrix is encoded by separating zero and non-zero elements into two tables: one indicating the positions of zeros and non-zeros, and another storing only the non-zero values. The decoding unit performs logical operations between the first zero coefficient position table and the first zero variable position table to determine which coefficient-variable pairs will produce non-zero results, retrieves the corresponding second coefficient and second variable from the non-zero coefficient and variable tables, and input them to the product-sum circuit for computation.
Claim 7 is allowable because of its dependency to claim 6.
Claim 10 recites:
a decoding unit including a selector and a product-sum device, the decoding unit being configured to decode a first coefficient matrix encoded into a first zero coefficient position table and a first non-zero coefficient table, the first zero coefficient position table indicating positions of first coefficients each having a zero value in the first coefficient matrix by a first value and indicating positions of second coefficients each having a non-zero value in the first coefficient matrix by a second value, the first non-zero coefficient table holding the second coefficients in the first coefficient matrix;
a product-sum circuit that performs convolution processing on the first coefficient matrix decoded by the decoding unit and a first variable matrix, wherein the decoding unit decodes the first coefficient matrix by storing the second coefficients stored in the first non-zero coefficient table at the positions on the first zero coefficient position table indicated by the second value; and
an encoding unit that encodes a second variable matrix output from the product-sum circuit into a second zero variable position table and a second non-zero coefficient table, the second zero variable position table indicating positions of first variables each having a zero value in the second variable matrix by the first value and indicating positions of second variables each having a non- zero value in the second variable matrix by the second value, the second non-zero coefficient table holding the second coefficients in the second variable matrix wherein the encoding unit includes: a determination circuit that determines whether or not a value input thereto is zero;
a first buffer that stores the first value when the determination circuit determines that the value is zero, and stores the second value when the determination circuit determines that the value is not zero; and a second buffer that stores the second variable when the determination circuit determines that the value is not zero, and wherein the encoding unit sequentially inputs variables constituting the second variable matrix to the determination circuit,
wherein the encoding unit further includes a register that stores a second zero coefficient position table to be used for convolution processing in a next layer, when a value stored at a position on the second zero coefficient position table corresponding to a position on the second variable matrix of a variable determined to be non-zero by the determination circuit is the first value, the first buffer stores the first value instead of the second value, and when a value stored at a position on the second zero coefficient position table corresponding to a position on the second variable matrix of a variable determined to be non-zero by the determination circuit is the first value, the second buffer does not store the variable.
Closest prior arts:
Woo et al. Pub. No.: US9818059B1.
Woo describes a computing device receiving multiple input signals from an outside source. The device's controller checks if each signal is zero or non-zero. The method goes on to store at least one of these signals in the device's memory, creating an index that marks the addresses of non-zero values. The controller then sends at least one non-zero signal to a data bus, which can be accessed by other parts of the computer. However Woo does not teach Encoding unit includes a register that stores a second zero coefficient position table for use in convolution processing of the next layer. If a variable determined to be non-zero corresponds to a position in this table that holds the first value, the first buffer stores the first value instead of the second value. Additionally, when that position is the second zero coefficient position table holds the first value, the second buffer omits storing the corresponding variable.
Tanaka et al., Pub. No.: US11281746B2.
Tanaka describes an arithmetic operation method for a convolutional layer in a neural network involves several steps. First, a coefficient matrix is created from the kernel used in the convolutional layer. This matrix is linked to an input vector formed by turning a feature map into one column. Next, non-zero elements in the coefficient matrix are identified. Then, multiplications of these non-zero elements with related elements of the input vector are assigned to multiple calculators. However Tanaka does not teach Encoding unit includes a register that stores a second zero coefficient position table for use in convolution processing of the next layer. If a variable determined to be non-zero corresponds to a position in this table that holds the first value, the first buffer stores the first value instead of the second value. Additionally, when that position is the second zero coefficient position table holds the first value, the second buffer omits storing the corresponding variable.
Dependent claim(s): 11 – 13, are allowable because of their dependency to claim 10.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chen et al., Pub. No.: US10877752B2.
Chen teaches circuit that enables a multiply-accumulate (MAC) operation based on a current-sensing readout technique. An operational amplifier coupled with a bitline of a column of bitcells included in a memory array of the CIM circuit to cause the bitcells to act like ideal current sources for use in determining an analog voltage value outputted from the operational amplifier for given states stored in the bitcells and for given input activations for the bitcells.
Abts et al., Pub. No.: US11360934B1.
Abts teaches the functional slices are configured to perform specific operations within the processor, which includes memory slices for storing operand data and arithmetic logic slices for performing operations on received operand data (e.g., vector processing, matrix manipulation).
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner
should be directed to MATIYAS T MARU whose telephone number is (571)270-0902 or via email: matiyas.maru@uspto.gov. The examiner can normally be reached Monday - Friday (8:00am - 4:00pm) EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a
USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to
use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor,
Michelle Bechtold can be reached on (571)431-0762. The fax phone number for the organization were this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from
Patent Center. Unpublished application information in Patent Center is available to registered users.
To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit
https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and
https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional
questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like
assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA)
or 571-272-1000.
/M.T.M./ Examiner, Art Unit 2148
/MICHELLE T BECHTOLD/ Supervisory Patent Examiner, Art Unit 2148