DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 3, 8, 13, and 17 have been amended.
Claims 2-21 have been examined.
The drawing and claim objections in the previous Office Action have been addressed and are withdrawn.
The § 112 rejections in the previous Office Action have been addressed and are withdrawn.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 2-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over corresponding claims of US Patent No. 12,112,141 (reference patent). Although the claims at issue are not identical, they are not patentably distinct from each other because the claims in the reference patent anticipate the instant claims. For example, claim 1 of the reference patent includes all the limitations of claim 2 of the instant application.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 2, 3, 12, 13, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US Publication No. 2021/001732 by Botimer et al. (previously cited and hereinafter referred to as “Botimer”).
Regarding claims 2, 12, and 20, taking claim 2 as representative, Botimer discloses:
a method for performing a two dimensional (2D) convolution operation, the method comprising: performing a first simultaneous multiplication, by a processor, of each value of a first subset of element values of an input feature map having dimensions n by m, with a first element value from among x * y elements of a convolution kernel, wherein x is a number of rows in the convolution kernel and y is a number of columns in the convolution kernel, and n is a number of rows in the input feature map and m is a number of columns in the input feature map (Botimer discloses, at Figures 6 and 7, and related description, performing a convolution using two matrices, i.e., an input feature map and a weights matrix, and that doing so involves concurrently multiplying each element of a subset of input feature values with a particular weight value. See also ¶ [0031].);
for each remaining value of the x * y elements of the convolution kernel, performing, by the processor, a simultaneous multiplication of the each remaining value with a corresponding subset of element values of the input feature map (Botimer discloses, at Figure 7 and related description, iterating through the weights, i.e., the elements of the convolution kernel.);
for each simultaneous multiplication, storing, by the processor, a result of the each simultaneous multiplication in at least one accumulator connected to the processor (Botimer discloses, at Figure 6 and related description, each multiply and accumulate unit (MAC), includes an adder that receives the product of the multiplier, which discloses storing a result of the multiplication.); and
outputting, by the processor, an output from the at least one accumulator as a first row of an output feature map (OFM) (Botimer discloses, at Figure 7 and related description, outputting OFM values, which discloses outputting, by the processor, an output from the at least one accumulator as a first row of an output feature map.),
wherein the at least one accumulator comprises a first accumulator configured to add the result of the each simultaneous multiplication to an input value, and a second accumulator coupled to an output of the first accumulator and configured to add the result of the each simultaneous multiplication to an input value and output the first row of the OFM (Botimer discloses, at Figure 6 and related description, the adder, i.e., the first accumulator, adds the result of the multiplication to an input value and the MAC includes an accumulator 630, i.e., a second accumulator, coupled to the output of the adder configured to add the result and output OFM values.).
Regarding claims 3 and 13, taking claim 2 as representative, Botimer discloses the elements of claim 2 as discussed above. Botimer also discloses:
outputting the first row of the OFM comprises adding, by the processor, results of the plurality of simultaneous multiplications stored in the at least one accumulator to generate the first row of the OFM (Botimer discloses, at ¶ [0035], accumulating the results of each multiplication to produce corresponding OFM values.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-11, 14-19, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Botimer in view of US Publication No. 2020/0210517 by Baum et al. (previously cited and hereinafter referred to as “Baum”) in view of US Patent No. 5,285,403 by Quisquater et al. (previously cited and hereinafter referred to as “Quisquater”).
Regarding claims 4 and 14, taking claim 4 as representative, Botimer discloses the elements of claim 2 as discussed above. Botimer also discloses:
wherein the convolution kernel is stored in a first storage device of the processor (Botimer discloses, at ¶ [0033], the weights are stored in memory, which discloses a first storage device of the processor.),
wherein the first subset of element values of the input feature map is stored in a second storage device of the processor (Botimer discloses, at ¶ [0033], the input feature map values are stored in memory, which discloses a second storage device of the processor.), and
wherein the first storage device comprises a … memory connected to the processor, and wherein the second storage device comprises a plurality of registers in a processing element (PE) of a multiply-accumulate (MAC) tile of the processor … (Botimer discloses, at ¶ [0033], the weights are stored in various types of memory, which discloses a memory connected to the processor. Botimer also discloses, at ¶ [0039], a plurality of shift buffer elements, which discloses registers in a processing element (PE) of a multiply-accumulate (MAC) tile of the processor.).
Botimer does not explicitly disclose the aforementioned first storage device comprises a cache and the aforementioned registers are 9-bit registers.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
cache memory (Baum discloses, at ¶ [0130], cache.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer to include cache, as disclosed by Baum, because cache memory provides improved performance regarding access to data.
Also, in the same field of endeavor (e.g., arithmetic processing) Quisquater discloses
9-bit registers (Quisquater discloses, at col. 10, lines 44-49, 9-bit registers.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer’s shift elements to utilize 9 bits, as disclosed by Quisquater because the number of bits to use is an arbitrary design choice. Therefore, this modification merely entails a combination of prior art elements (cited above) according to known methods to yield predictable results, which is an exemplary rationale to support a conclusion of obviousness, as per MPEP § 2143.
Regarding claims 5, 15, and 21, taking claim 5 as representative, Botimer discloses the elements of claim 4 as discussed above. Botimer also discloses:
shifting, by the processor, the first subset of element values one register to the left in the plurality of registers of the second storage device c shifting, by the processor, the second subset of element values of the input feature map one register to the left in the plurality of registers of the second storage device, wherein the second subset of element values of the input feature map is then stored to the second storage device.);
based on the shifting, performing a second simultaneous multiplication, by the processor, of each value of a second subset of element values of the input feature map with a second element value from among the x * y elements of the convolution kernel, wherein the first subset of element values of the input feature map comprises values in first to p-th column of a first row of the input feature map, and the second subset of element values of the input feature map comprises values in second to (p+1)-th column of the first row of the input feature map, and wherein the second subset of element values of the input feature map further comprises at least one element value from the first subset of element values of the input feature map (Botimer discloses, at ¶¶ [0033]-[0036], ¶ [0039], and Figure 6, simultaneously multiplying a second subset of IFM values, where the second subset is shifted with respect to a first subset, e.g., the first subset of input values and the second set of are in the first and second columns and second and third columns, respectively, by a second value of the weight values matrix. The value from the second column is present in both the first subset and the second subset.); and
shifting, by the processor, the second subset of element values of the input feature map one register to the left in the plurality of registers of the second storage device, wherein the second subset of element values of the input feature map is then stored to the second storage device (Botimer discloses, at ¶ [0039], a plurality of shift buffer elements to store and shift the input feature map values, which discloses shifting, by the processor, the second subset of element values of the input feature map one register to the left in the plurality of registers of the second storage device, wherein the second subset of element values of the input feature map is then stored to the second storage device.);
wherein the first element value from among the x * y elements of the convolution kernel in the first simultaneous multiplication is different from the second element value from among the x * y elements of the convolution kernel in the second simultaneous multiplication (Botimer discloses, at Figure 6 and¶ [0036], the first and second weight values (elements) are different in the first and second simultaneous multiplications, e.g., (0,0,0) in the first and (0,1,0) in the second.).
Regarding claims 6 and 16, taking claim 6 as representative, Botimer discloses the elements of claim 5 as discussed above. Botimer also discloses:
broadcasting, by the processor, the first element value from among x * y elements of the convolution kernel to the second storage device (Botimer discloses, at ¶ [0035], using the first weight value with multiple multiply and accumulate units in calculations with each of the subset of input values, which discloses the claimed broadcasting to the second storage device.); and
broadcasting, by the processor, the second element value from among the x *y elements of the convolution kernel to the second storage device (Botimer discloses, at ¶ [0035], using the second weight value with multiple multiply and accumulate units in calculations with each of the subset of input values, which discloses the claimed broadcasting to the second storage device.).
Regarding claim 7, Botimer discloses the elements of claim 4 as discussed above. Botimer also discloses:
the convolution kernel having the dimension x by y is a weight matrix, the weight matrix comprising a plurality of weight values, the plurality of weight values of the weight matrix are being stored in a … memory connected to the processor, and wherein the input feature map having the dimension n by m comprises a plurality of activation values (Botimer discloses, at ¶ [0032] et seq., the first matrix is a weight matrix and the second matrix is a matrix of input feature map (activation) values.).
Botimer does not explicitly disclose the aforementioned first storage device comprises a cache.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
cache memory (Baum discloses, at ¶ [0130], cache.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer to include cache, as disclosed by Baum, because cache memory provides improved performance regarding access to data.
Regarding claims 8 and 17, taking claim 8 as representative, Botimer discloses the elements of claim 7 as discussed above. Botimer also discloses:
...an adder tree (Botimer discloses, at ¶ [0035], accumulating products, which discloses an adder tree.).
Botimer does not explicitly disclose wherein the processor comprises a plurality of multiply- accumulate (MAC) tiles, each MAC tile comprising an array of processing elements (PE) comprising a plurality of PE rows and a plurality of PE columns, each PE column of the plurality of PE columns comprising a plurality of PE....
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
a plurality of multiply- accumulate (MAC) tiles, each MAC tile comprising an array of processing element (PE) comprising a plurality of PE rows and a plurality of PE columns, each PE column of the plurality of PE columns comprising a plurality of PEs... (Baum discloses, at Figure 21D, an array of PEs comprising rows and columns.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer to use a PE array, as disclosed by Baum, because doing so improves performance by providing an efficient mechanism to increase parallelism.
Regarding claim 9, Botimer discloses the elements of claim 8 as discussed above. Botimer does not explicitly disclose wherein each MAC tile further comprises cache memory comprising a plurality of parallel activation lanes, each activation lane corresponding to a row of the array of PEs in the each MAC tile.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
wherein each MAC tile further comprises a cache memory comprising a plurality of parallel activation lanes, each activation lane corresponding to a row of the array of PEs in the MAC tile (Baum discloses, at Figure 21D, each row comprises storage, which discloses the claimed cache with parallel activation lanes corresponding to rows of the PEs.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer’s calculations to run on a PE array utilizing cache, as disclosed by Baum, because cache memory provides improved performance regarding access to data.
Regarding claims 10 and 19, taking claim 10 as representative, Botimer discloses the elements of claim 8 as discussed above. Botimer also discloses:
each PE of the plurality of PEs comprises the plurality of registers configured to store the plurality of activation values of the input feature map, a multiplier connected to the plurality of registers and configured to multiply activations from the input feature map by the weight values from the cache memory, and a PE accumulator connected to the multiplier and configured to add an output from the multiplier to a value from the plurality of registers and store the result in the plurality of registers (Botimer discloses, at ¶ [0039], a plurality of shift buffer elements, which discloses registers, that store the IFM values and are connected to multiply and accumulate units to multiply and accumulate the input and weight values.).
Regarding claim 11, Botimer discloses the elements of claim 8 as discussed above. Botimer does not explicitly disclose wherein the array of PEs in the MAC tile comprises eight PE columns, wherein each PE column of the eight PE columns of the MAC tile comprises sixteen PEs.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
wherein the array of PEs in the MAC tile comprises eight PE columns, wherein each PE column of the eight PE columns of the MAC tile comprises sixteen PEs (Baum discloses, at Figure 21B, an 8x8 array. Baum also discloses, at ¶ [0079], using 16 rows.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer’s calculations to run on a PE array, as disclosed by Baum, in order to improve performance by providing a simple mechanism to increase parallelism.
Regarding claim 18, Botimer discloses the elements of claim 17 as discussed above. Botimer does not explicitly disclose
each MAC tile further comprises cache memory comprising a plurality of parallel activation lanes, each activation lane corresponding to a row of the array of PEs in the each MAC tile, wherein the array of PEs in the each MAC tile comprises eight PE columns, wherein each PE column of the eight PE columns of the each MAC tile comprises sixteen PEs.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
wherein each MAC tile further comprises a cache memory comprising a plurality of parallel activation lanes, each activation lane corresponding to a row of the array of PEs in the MAC tile (Baum discloses, at Figure 21D, each row comprises storage, which discloses the claimed cache with parallel activation lanes corresponding to rows of the PEs.); and
wherein the array of PEs in the MAC tile comprises eight PE columns, wherein each PE column of the eight PE columns of the MAC tile comprises sixteen PEs (Baum discloses, at Figure 21B, an 8x8 array. Baum also discloses, at ¶ [0079], using 16 rows.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer’s calculations to run on a PE array utilizing cache, as disclosed by Baum, because cache memory provides improved performance regarding access to data and PE arrays improve performance by providing a simple way to increase parallelism.
Response to Arguments
On pages 11-12 of the response filed April 8, 2026 (“response”), the Applicant argues, “Because the final claim scope of the claims in the present application has not been determined, Applicant reserves the right to file a future terminal disclaimer or to argue that the present claims would be patentably distinct over the claims 2-21 of U.S. Pat. No. 12,112,141.”
The Applicant’s arguments are not a proper response. MPEP § 804.1(B)(1) states, “A complete response to a nonstatutory double patenting (NSDP) rejection is either a reply by applicant showing that the claims subject to the rejection are patentably distinct from the reference claims, or the filing of a terminal disclaimer in accordance…. Such a response is required even when the nonstatutory double patenting rejection is provisional. As filing a terminal disclaimer, or filing a showing that the claims subject to the rejection are patentably distinct from the reference application’s claims, is necessary for further consideration of the rejection of the claims, such a filing should not be held in abeyance. Only compliance with objections or requirements as to form not necessary for further consideration of the claims may be held in abeyance until allowable subject matter is indicated. Replies with an omission should be treated as provided in MPEP § 714.03. Therefore, an application must not be allowed unless the required compliant terminal disclaimer(s) is/are filed and/or the withdrawal of the nonstatutory double patenting rejection(s) is made of record by the examiner.” In the interest of compact prosecution, the Examiner will consider the Applicant’s reply as adequate, rather than requiring a complete reply at this point.
On pages 13-14 of the response the Applicant argues, “the Office action appears to equate the "first simultaneous multiplication" of claim 2 with the "convolution of a first matrix (e.g., weights) and a second matrix (e.g., input feature map)" (paragraph [0032]) of Botimer, and the "at least one accumulator" of claim 2 with the "multiply and accumulate units 605, 610" of FIG. 6 of Botimer. The Office action further appears to equate the adder in the multiply and accumulate unit 605 or 610 of FIG. 6 of Botimer with the "first accumulator" of claim 2, and the buffer 630 in the multiply and accumulate unit 605 or 610 of FIG. 6 of Botimer with the "second accumulator" of claim 2….In Botimer, "[e]ach multiply and accumulate unit 605, 610 can multiply data values of the weight matrix 615 with the corresponding adjacent data values in the input feature map 620 and sum the product with the content of the respective accumulator buffers 630 in the multiply and accumulate units 605, 610 to generate a current data value in the output feature map 625" (paragraph [0032]). However, the buffer 630 (the alleged "second accumulator" of claim 1) in the multiply and accumulate unit 605 or 610 of FIG. 6 of Botimer does not appear to "add the result of the each simultaneous multiplication to an input value and output the first row of the OFM" as claim 2 recites. As such, the cited sections of Botimer do not appear to disclose or even suggest "wherein the at least one accumulator comprises a first accumulator configured to add the result of the each simultaneous multiplication to an input value, and a second accumulator coupled to an output of the first accumulator and configured to add the result of the each simultaneous multiplication to an input value and output the first row of the OFM," as independent claim 2 recites or as independent claims 12 and 20 similarly recite.”
Though fully considered, the Examiner respectfully disagrees. Botimer discloses, at Figure 6, an accumulator 630, which the Examiner maintains teaches the claimed second accumulator. Botimer’s accumulator 630 maps almost exactly to the Applicant’s accumulator 122 of Figure 2, i.e., the claimed second accumulator. The only difference is that accumulator 122 doesn’t show the claimed output, while Botimer’s accumulator 630 does.
Claim 2 recites the second accumulator is coupled to the output of the first accumulator, is configured to add the result of each simultaneous multiplication to an input value, and to output the first row. In Applicant’s Figure 2, accumulator 122 is coupled to the output of accumulator 124, and presumably adds the result of a multiplication value (stored therein) to the input value received from accumulator 124. The result is fed back to accumulator 122. Similarly, Botimer’s accumulator 130, as disclosed in Figure 6 and related description, is coupled to the output of the adder, adds of the value received therefrom, and feeds the result back to the adder. The Examiner maintains that this discloses the claimed limitations. Accordingly, the Applicant’s arguments are deemed unpersuasive.
On page 15 of the response the Applicant argues, “Further, Applicant respectfully submits that, Botimer appears to perform sequential multiplication of values from the input feature map, and not a simultaneous multiplication (FIG. 8, paragraph [0041] of Botimer). For instance, paragraph [0041] of Botimer states "a first multiply and accumulate unit 840 can accumulate the product of the weight value (0,0,0) and the input feature map value (0,0,0), and the second multiply and accumulate unit 845 can accumulate the product of the weight value (0,0,0) and the adjacent input feature map value (0,1,0) during the first cycle." That is, the serial shift buffer 805 is "initialized by shifting in a plurality of input feature map values (0,0,0) through (0,1,2) from memory" which are then sequentially shifted in to two multiply and accumulate units 840 and 845 to perform multiplication. (Botimer at para. [0040]).”
Though fully considered, the Examiner respectfully disagrees. The portion of Botimer cited above by the Applicant explicitly discloses the first MAC unit 840 and the second MAC unit 845 accumulate their respective products “during the first cycle.” This discloses simultaneous multiplication. As do the several explicit mentions in Botimer of performing operations in parallel. See, e.g., ¶ [0031] and ¶ [0052]. The sequential shifting in ¶ [0041] refers to initializing the buffer, and does not indicate that the multiplication and accumulation operations are performed sequentially. Accordingly, the Applicant’s arguments are deemed unpersuasive.
On page 15 of the response the Applicant argues, “Furthermore, because claims 3-11, 13-19, and 21 depend, directly or indirectly, from independent claims 1, 12, or 20, each of the claims 3-11, 13-19, and 21 incorporates all of the terms and features of their respective base claim, in addition to other features, which together further patentably distinguish these claims over the art of record when considered as a whole. Therefore, Applicant requests that the rejections of claims 3-11, 13-19, and 21 also be withdrawn, and that these claims be allowed as well.”
Though fully considered, the Examiner respectfully disagrees. The reasons set forth in the remarks and rejections presented above, including those regarding the independent claims, are applicable to these claims.
Conclusion
THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAWN DOMAN whose telephone number is (571)270-5677. The examiner can normally be reached on Monday through Friday 8:30am-6pm Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jyoti Mehta can be reached on 571-270-3995. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAWN DOMAN/
Primary Examiner, Art Unit 2183
Prosecution Insights