DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Amendment/Request for reconsideration filed on 06/30/2025 has been entered. Claims 1, 3-6, 8, 10-13, 15, 17-19 and 21-28 are pending. No amendment to any claims. Claims 2, 7, 9, 14, 16 and 20 have been canceled without prejudice.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1,3-4, 8, 10-11, 15, 17, 19, 21, 23-25, and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Park, US Patent Application Publication No. US 2018/0253635 A1, and further in view of Dally et al. (Dally), US Patent Application Publication No. US 2018/0046916 A1.
As to independent claim 1, Park discloses an apparatus, comprising:
a location vector table configured to store data indicating a relative position of a first non-zero weight with respect to a second non-zero weight within a filter of a neural network operation (paragraphs [0052]-[0055] and Figure 3: an input feature list (location vector table) includes a first index RA (first weight) and a second index CA(second weight) which correspond to a spatial location of an input feature, wherein an index may be referred to as an address, wherein the input feature list may include an index and data which correspond to each of input features f1,1, f1,4 and f4,3, having non-zero values, and index-based neural network operation may include an index operation, which is performing an operation on each input feature index in an input feature list; paragraph [0069]: the input feature list may include an input feature index and an input feature value which correspond to each input (“input feature”), and the input may have a non-zero value, and the input feature index may indicate a location (position) of the input feature on an input feature map; paragraph [0074]: the neural network device may reduce the amount of operations by performing an operation on each input feature index and each input feature value and filtering output indices which do not influence an output result during the operation);
a memory configured to store data indicating one or more values of one or more weights in the filter (paragraphs [0060], [0065], [0168]: receiving a weight index and a weight value corresponding to the weight index from a weight list stored in the memory 24); and
one or more processing units configured to:
obtain a value of the first non-zero weight at a first memory address of the memory (paragraph [0056]: the weight list may include an index and data which correspond to each weight having a non-zero value; paragraph [0076]: the input feature list may include an input feature index and an input feature value which correspond to each of input features of the input feature matrix, wherein the index may include a first index and a second index which respectively correspond to a row (first memory address) and a column (second memory address) of the input feature matrix; paragraph [0079]: performing an operation on the input feature index corresponding to the input feature having a non-zero value; paragraph [0081]: generating an output feature value corresponding to the output feature index based on the input feature value and a weight value in operation);
determine a second memory address based on the first memory address and the data stored in the location vector table, a value of the second non-zero weight stored at the second memory address (paragraph [0076]: the input feature list may include an input feature index and an input feature value which correspond to each of input features of the input feature matrix, wherein the index may include a first index and a second index which respectively correspond to a row (first memory address) and a column (second memory address) of the input feature matrix; paragraph [0079]: performing an operation on the input feature index corresponding to the input feature having a non-zero value; paragraph [0081]: generating an output feature value corresponding to the output feature index based on the input feature value and a weight value in operation),
obtain a value of the second non-zero weight at the second memory address of the memory (paragraph [0081]: generating an output feature value corresponding to the output feature index based on the input feature value and a weight value in operation), and
generate an output of the neural network operation from the first non-zero weight and the second non-zero weight (paragraphs [0080]-[0081]: the neural network device may generate the output feature index by adding the input feature index and the weight index, the neural network device may add a first index of the input feature index and a first index of the weight index and add a second index of the input feature index and a second index of the weight index).
Park, however, does not disclose “read a value of the first non-zero weight at a first memory address” and “read the value of the second non-zero weight at the second memory address”.
In the same field of endeavor, Dally discloses a method, computer program product, and system for performing computations using a sparse convolutional neural network accelerator by receiving compressed-sparse data for input to a processing element, wherein the compress-sparse data encodes non-zero elements (first non-zero weight, second non-zero weight, etc.) and corresponding multi-dimensional positions (Abstract). Dally further discloses the non-zero elements are processed in parallel by the processing element to produce a plurality of result values, and the corresponding multi-dimensional positions are processed in parallel by the processing element to produce destination addresses for each result value in the plurality of result values, and each result value is transmitted to a destination accumulator associated with the destination address for the result value (Abstract). Dally further discloses as the weights are read by the memory interface 205 from DRAM, the weights are broadcast to the processing elements (PEs) 210 and held locally in a per-PE weight buffer 305, and the input activations may be read by the memory interface 205 from DRAM or transmitted from the output activations buffer 350 and stored locally in a per-PE input activations buffer 310 (paragraph [0074]). Dally further discloses a state machine operates on weight and input activations in the order to produce an output-channel group of Ksubc X Wsubt X Hsubt partial sums inside the accumulator array 340 (paragraph [0075]). Dally further discloses in Table 4 which describes pseudo-code for sequencing computations for a tile including read next weights, value in “Do While” loop (page 8).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claim invention to modify the system of Park to include “read a value of the first non-zero weight at a first memory address” and “read the value of the second non-zero weight at the second memory address”. Dally suggests that first read the first vector of F weights, and sequences all of the weights for another vector of input activations to produce another vector of products (Table 4).
As to dependent claim 3, Park discloses wherein the location vector table is further configured to store data indicating coordinates of all non-zero weights in the filter (Figure 3).
As to dependent claim 4, Park discloses wherein the data stored in the memory comprises a weight identification (ID) of the second non-zero weight (paragraphs [0072], [0084]-[0086] and Figure 6) and the one or more processing units are configured to obtain the value of the second non-zero weight by:
reading the weight ID from the second memory address of the memory (paragraph [0147]); and
determining the value of the second non-zero weight using the weight ID and a look up table, the look up table associating with the weight ID with the value of the second non-zero weight (paragraphs [0150]-[0151]).
As to dependent claim 23, Park discloses a third processor to determine a memory address for an input activation of the neural network operation based on the second memory address and to read the activation from the memory address, wherein the second processor is to generate the output of the neural network operation by multiplying the input activation with the second non-zero weight (paragraphs [0046], [0081]).
As to dependent claim 24, Park discloses wherein the third processor is to determine the memory address for the input activation further based on the first memory address (paragraph [0053]).
As to dependent claim 25, Park discloses wherein the data stored in the memory comprises a weight identification of the second non-zero weight (paragraphs [0072], [0084]-[0086] and Figure 6), and the first processor is to obtain the value of the second non-zero weight by:
reading the weight ID from the second memory address of the memory (paragraph [0147]); and
determining the value of the second non-zero weight using the weight ID and a look up table, the look up table associating the weight ID with the value of the second non-zero weight (paragraphs [0150]-[0151]).
As to dependent claim 27, Park discloses wherein the weight ID of the second non-zero weight is determined by quantizing the value of the second non-zero weight (paragraphs [0084]-[0087]).
Claim 10 is method claim that contains similar limitations of claim 3. Therefore, claim 10 is rejected under the same rationale.
Claims 11, 17 and 21 are method, medium and system claims, respectively. Claims 11, 17 and 21 contain similar limitations of claim 4. Therefore, claims 11, 17 and 21 are rejected under the same rationale.
Claims 5 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Park and Dally as applied to claims 1,3-4, 8, 10-11, 15, 17, 19, 21, and 23-28 above, and further in view of Teig et al., US Patent Number US 11,537,870.
As to dependent claim 5, Park and Dally, however, does not disclose wherein the first non-zero weight or the second non-zero weight is a 16-bit floating point value or a 32-bit floating point value.
In the same field of endeavor, Teig discloses neural networks involve many of weights that are calculated during training and then used when the neural network is embedded into the device, wherein these weights are generally floating-point values, e.g., 32-bit values, and each non-zero weight uses a large number of bits in floating point networks (col. 1, lines 15-26).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the systems of Park and Dally to include the first non-zero weight or the second non-zero weight is a 16-bit floating point value or a 32-bit floating point value, as taught by Teig for the purpose using a large number of bits.
Response to Arguments
In the Remarks, Applicant argues in substance that the combination of Park and Dally does not show or render obvious “one or more processing units configured to… determine a second memory address based on the first memory address and the data stored in the location vector table, a value of the second non-zero weight stored at the second memory address.”
In reply to this argument, paragraphs [0033]-[0034] of the Specification of the instant application states “the memory address of the first input activation (X.sub.0, Y.sub.0, Z.sub.0)” and Let I(X.sub.n, Y.sub.n, Z.sub.n) represent the memory address for input activation.
In this case, Park discloses Feature map is which represent various features of input data, wherein feature maps FM1, FM2, and FM3 may have a form of a two-dimensional matrix or a form of a three-dimensional matrix, and the feature maps FM1, FM2 and FM3 have a width (or a column) W, a height (or a row) H, and a depth D, which may respectively correspond to the x-axis, the y-axis, and the z-axis in a coordinate system (paragraph [0044]). Park further discloses a location on the xy-plane of a feature may may be referred to a spatial location, a location on the z-axis of the feature map may be referred to as a channel (paragraph [0045]). Park discloses in paragraph [0056]: the weight list may include an index and data which correspond to each weight having a non-zero value; paragraph [0076]: the input feature list may include an input feature index and an input feature value which correspond to each of input features of the input feature matrix, wherein the index may include a first index and a second index which respectively correspond to a row (first memory address) and a column (second memory address) of the input feature matrix; paragraph [0079]: performing an operation on the input feature index corresponding to the input feature having a non-zero value; paragraph [0081]: generating an output feature value corresponding to the output feature index based on the input feature value and a weight value in operation. In addition, Park discloses in paragraph [0076]: the input feature list may include an input feature index and an input feature value which correspond to each of input features of the input feature matrix, wherein the index may include a first index and a second index which respectively correspond to a row (first memory address) and a column (second memory address) of the input feature matrix; paragraph [0079]: performing an operation on the input feature index corresponding to the input feature having a non-zero value; paragraph [0081]: generating an output feature value corresponding to the output feature index based on the input feature value and a weight value in operation. Thus, Park discloses “one or more processing units configured to… determine a second memory address based on the first memory address and the data stored in the location vector table, a value of the second non-zero weight stored at the second memory address.”
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CHAU T NGUYEN whose telephone number is (571)272-4092. The examiner can normally be reached on Monday-Friday from 8am to 5pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Cesar Paula, can be reached at telephone number 5712724128. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated-interview-request-air-form.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
/CHAU T NGUYEN/Primary Examiner, Art Unit 2145