DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Specification
The disclosure is objected to because of the following informalities. The specification uses different terminology for the same reference designator. Paragraph [0081] refers to a “min-sum decoder 650”, followed by “the decoder 650” later in the paragraph, and paragraph [0082] revers to “the MS decoder 650”. Furthermore, for subsequent use of an acronym, the first use should recite the acronym, such as a “min-sum decoder (MS) 650”.
Furthermore, paragraph [00104] appears to have a typographical error, reciting “bourdary” instead of “boundary” and two instances.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 lime 5 recites “the floating-point values”. This limitation lacks antecedent basis. It is unclear whether “the floating-point values” refers to the subset of floating-point values recited in line 3 or a different set of floating-point values. For purposes of examination, Examiner interprets as “the subset of floating-point values”. Claims 2-10 inherit the same deficiency as claim 1 based on dependence. Claim 11 recites substantially the same limitation and is rejected for the same reason. Claims 12-20 inherit the same deficiency as claim 11 based on dependence.
Claim 1 lines 8-9 recites “merging an empty region among the plurality of regions to neighbor regions to output cluster of the weights in merged regions”. It is unclear whether a single empty region is merged into more than one neighbor regions, a plural number of regions has a single empty region, wherein each of the single empty region of the plural number of regions is merged to each neighbor region or other. For purposes of examination, the Examiner interprets as a plural number of regions has a single empty region, wherein each of the single empty region of the plural number of regions is merged to each neighbor region. Claims 2-10 inherit the same deficiency as claim 1 based on dependence. Claim 11 recites substantially the same limitation and is rejected for the same reason. Claims 12-20 inherit the same deficiency as claim 11 based on dependence. Claims 2, 7, 12, and 17 recite “the merged regions” and are rejected for the reason set forth with respect to claim 1 “merged regions”. Claims 3 and 13 inherit the same deficiency as claim 2 and 12 respectively based on dependence.
Claim 5 recites “wherein the merging comprises merging each empty region among the plurality of regions with a neighbor region”. Claim 1 recites the merging as “merging an empty region”, i.e., singular region. Claim 5 recites a plurality of regions, i.e., each empty region. Examiner recommends amending to recite “wherein the merging further comprises merging further empty regions” or equivalent. Claim 15 recites substantially the same limitation and is rejected for the same reason.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Apparatus claims 11-20 will be addressed first, followed by method claim 1-10.
Regarding claim 11, under the Alice framework Step 2A prong 1, the claim recites Mathematical concepts. The claim recites mathematical calculations and mathematical relationships for quantizing and clustering values. Specifically, the claim recites the following:
calculate parameters corresponding to the DNN (see e.g., [0091] eqn 1),
use a subset of floating-point values to represent weights in the DNN (see e.g., [0098] eqn 1, fig 9A);
quantize the floating-point values onto a flexible-power-of-two (FPoT) alphabet (see e.g., [00100];
list values in the FPoT alphabet in a plurality of regions (see e.g., fig 9B, [00103]); and
merge an empty region among the plurality of regions to neighbor regions to output clusters of the weights in merged regions, the merged regions having respective centroids and boundary lines in between (see e.g., [00104-00110]).
For these reasons, these are steps in a mathematical calculation using mathematical relationships.
Under the Alice framework Step 2A prong 2 analysis, additional elements not reciting Mathematical equations and mathematical calculations thereof include: a memory system for operating a deep neural network (DNN), comprising: a data source; and a controller, wherein the controller is program to perform the steps recited in the Step 2A prong 1 analysis. This additional element does no more than generally link the additional element to the mathematical calculations in a manner that in effect merely recites “apply it” to the math, or recite mere instructions to apply the exception using generic computer components. For these reasons claim 11 is not integrated into a practical application.
Moreover, under the Alice Framework Step 2B analysis, the claim, considered individually and as an ordered combination does not include additional elements that are sufficient to amount to significantly more than the abstract idea. As discussed in the Step 2A prong 2 analysis, the claim merely generally links the additional element to the math. Furthermore as stated in the Step 2A prong 2 analysis, the claims recite mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using generic computer components cannot provide an inventive concept. For these reasons claim 11 elements considered individually and as an ordered combination does not amount to significantly more than the abstract idea.
Claims 12-20 are rejected for at least the reasons cited with respect to the claim 1 analysis. Under the Step 2A prong 1 analysis, claims 12-20 merely further mathematically limit the claim 11 mathematical elements recited. Claims 12-20 contain no further additional elements that would require further consideration under Step 2A prong 2 or Step 2B.
Claim 1 is directed to a method that would be practiced by the apparatus as in claim 11 as configured. All steps performed in the method of claim 1 would be performed by the apparatus as in claim 11 as configured. The analysis with respect to claim 11 applies equally to claim 1.
Claims 2-10 are rejected for at least the reasons cited with respect to the claim 1 analysis. Under the Step 2A prong 1 analysis, claims 2-10 merely further mathematically limit the claim 1 mathematical elements recited. Claims 2-10 contain no further additional elements that would require further consideration under Step 2A prong 2 or Step 2B.
Allowable Subject Matter
Claims 1-20 would be allowable if rewritten to overcome the rejections under 35 USC 112(b), and 35 USC 101.
The following is a statement of reasons for the indication of allowable subject matter. Applicant claims apparatus, and methods for operating a deep neural network (DNN), wherein the method as in claim 1 comprises:
using a subset of floating-point values to represent weights in the DNN;
quantizing the floating-point values onto a flexible-power-of-two (FPoT) alphabet;
listing values in the FPoT alphabet in a plurality of regions; and
merging an empty region among the plurality of regions to neighbor regions to output clusters of the weights in merged regions, the merged regions having respective centroids and boundary lines in between.
The primary reasons for indication of allowable subject matter is the combination of limitations quantizing the floating-point values onto a flexible-power-of-two (FPoT) alphabet; and merging an empty region among the plurality of regions to neighbor regions to output clusters of the weights in merged regions, the merged regions having respective centroids and boundary lines in between. Elements of the power-of-two alphabet and clustering of regions of weights having centroids and boundary lines were found in the prior art, however, no motivation to combine could be determined, furthermore, the prior art of record does not teach or suggest merging an empty region among the plurality of regions to neighbor regions to output clusters of the weights in merged regions.
Y.Li et al., Additive Powers-of-Two Quantization: An Efficient Non-Uniform Discretization for Neural Networks, arXiv:1909.13144v2 [cs>LG] 2020, (hereinafter “Li”), discloses an additive Powers-of-Two (APoT) quantization for neural networks (abstract, fig 2, fig 3). Li does not, however, teach or suggest merging an empty region among the plurality of regions to neighbor regions to output clusters of the weights in merged regions, the merged regions having respective centroids and boundary lines in between.
M. de Prado et al., QUENN: Quantization Engine for low-power Neural Networks, Computing Frontiers Conference, ACM, 2018 (hereinafter “de Prado”) discloses a modular neural network architecture including dynamic fixed point quantization comprising selecting optimum fixed point parameters for several bit widths (section 3.1, 3.2, 4.1). de Prado further discloses a k-means clustering approach (section 4.1). de Prado does not, however, teach or suggest wherein the quantizing the floating -point values is a power-of-two alphabet, listing the values in the power-of-two alphabet in a plurality of regions, or merging an empty region among the plurality of regions to neighbor regions to output clusters of the weights in merged regions.
US 20160335535 A1 Amir et al., (hereinafter “Amir”), discloses a system for mapping a neural network onto a neurosynaptic substrate, the neurosynaptic substrate including a memory system (abstract, fig 1). Amir further discloses reordering the neural network including clustering and merging rows and columns of the neural network (fig 6, fig 11, [0060-0065]). Amir further discloses reordering using a pair-wise centroid distance metric to reorder into clusters ([0093]). Amir further discloses replacing one or more entries of the block with zeros (fig 13-804, [0090]). Amir does not, however, teach or suggest, quantizing the floating-point values onto a flexible-power-of-two (FPoT) alphabet, or merging an empty region among the plurality of regions to neighbor regions to output clusters of weights in the merged regions.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to EMILY E LAROCQUE whose telephone number is (469)295-9289. The examiner can normally be reached on 10:00am - 1200pm, 2:00pm - 8pm ET M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Andrew Caldwell can be reached on 571-272-3702. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/EMILY E LAROCQUE/Primary Examiner, Art Unit 2182