DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 2-21 have been examined.
Priority
Applicant’s claim for the benefit of a prior-filed application under 35 U.S.C. 119(a)-(e) or under 35 U.S.C. 120, 121, 365(c), or 386(c) is acknowledged. The instant application claims priority to U.S. Application 16/900,819 (now US Patent 12,112,141), filed June 12, 2020, which claims priority to U.S. Provisional Application 62/947,423, filed December 12, 2019.
Information Disclosure Statement
The Applicant's submission of the Information Disclosure Statements dated October 7, 2024, December 31, 2024, and November 5, 2025 is acknowledged by the Examiner and the cited references have been considered in the examination of the claims now pending, except as otherwise indicated. Copies of the PTOL-1449s initialed and dated by the Examiner are attached to the instant office action.
On the IDS of October 7, 2024, the Applicant has not complied with the requirements of 37 CFR 1.98(d), which requires providing copies of certain references even if provided previously, unless two conditions are met. In this case, the conditions of 37 CFR 1.98(d)(1) are not met because the IDS does not identify the earlier application in which the references were cited.
Drawings
The drawings are objected to because of the following informalities.
In Figure 2, there are two arrow heads pointing to element 124. One of the arrow heads is not attached to a line. This appears to be in the nature of a typographical error.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the Applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claims 8-11 and 17-19 are objected to because of the following informalities.
Claim 8 recites, at line 3, “array of processing element.” This appears to be a typographical error. Applicant may have intended “array of processing elements.” Claim 17 includes similar language and is similarly objected to.
Claims 9-11, 18, and19 are objected to as depending from objected to base claims and failing to remedy the deficiencies of those claims. Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 2-21 are rejected on the ground of nonstatutory double patenting as being unpatentable over corresponding claims of US Patent No. 12,112,141 (reference patent). Although the claims at issue are not identical, they are not patentably distinct from each other because the claims in the reference patent anticipate the instant claims. For example, claim 1 of the reference patent includes all the limitations of claim 2 of the instant application.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 3 and 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the Applicant regards as the invention.
Claim 3 recites, at line 2, “outputting the values of the at least one accumulator as the first row.” There is insufficient antecedent basis for “the values” in the claim. For purposes of examination, this limitation is interpreted as, “outputting the first row.” Claim 13 includes similar language and is similarly rejected.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
Claims 2, 3, 12, 13, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by US Publication No. 2021/001732 by Botimer et al. (previously cited and hereinafter referred to as “Botimer”).
Regarding claims 2, 12, and 20, taking claim 2 as representative, Botimer discloses:
a method for performing a two dimensional (2D) convolution operation, the method comprising: performing a first simultaneous multiplication, by a processor, of each value of a first subset of element values of an input feature map having dimensions n by m, with a first element value from among x * y elements of a convolution kernel, wherein x is a number of rows in the convolution kernel and y is a number of columns in the convolution kernel, and n is a number of rows in the input feature map and m is a number of columns in the input feature map (Botimer discloses, at Figures 6 and 7, and related description, performing a convolution using two matrices, i.e., an input feature map and a weights matrix, and that doing so involves concurrently multiplying each element of a subset of input feature values with a particular weight value. See also ¶ [0031].);
for each remaining value of the x * y elements of the convolution kernel, performing, by the processor, a simultaneous multiplication of the each remaining value with a corresponding subset of element values of the input feature map (Botimer discloses, at Figure 7 and related description, iterating through the weights, i.e., the elements of the convolution kernel.);
for each simultaneous multiplication, storing, by the processor, a result of the each simultaneous multiplication in at least one accumulator connected to the processor (Botimer discloses, at Figure 6 and related description, each multiply and accumulate unit (MAC), includes an adder that receives the product of the multiplier, which discloses storing a result of the multiplication.); and
outputting, by the processor, an output from the at least one accumulator as a first row of an output feature map (OFM) (Botimer discloses, at Figure 7 and related description, outputting OFM values, which discloses outputting, by the processor, an output from the at least one accumulator as a first row of an output feature map.),
wherein the at least one accumulator comprises a first accumulator configured to add the result of the each simultaneous multiplication to an input value, and a second accumulator coupled to an output of the first accumulator and configured to add the result of the each simultaneous multiplication to an input value and output the first row of the OFM (Botimer discloses, at Figure 6 and related description, the adder, i.e., the first accumulator, adds the result of the multiplication to an input value and the MAC includes an accumulator 630, i.e., a second accumulator, coupled to the output of the adder configured to add the result and output OFM values.).
Regarding claims 3 and 13, taking claim 2 as representative, Botimer discloses the elements of claim 2 as discussed above. Botimer also discloses:
outputting the values of the at least one accumulator as the first row of the OFM comprises adding, by the processor, results of the plurality of simultaneous multiplications stored in the at least one accumulator to generate the first row of the OFM (Botimer discloses, at ¶ [0035], accumulating the results of each multiplication to produce corresponding OFM values.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4-11, 14-19, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Botimer in view of US Publication No. 2020/0210517 by Baum et al. (previously cited and hereinafter referred to as “Baum”) in view of US Patent No. 5,285,403 by Quisquater et al. (previously cited and hereinafter referred to as “Quisquater”).
Regarding claims 4 and 14, taking claim 4 as representative, Botimer discloses the elements of claim 2 as discussed above. Botimer also discloses:
wherein the convolution kernel is stored in a first storage device of the processor (Botimer discloses, at ¶ [0033], the weights are stored in memory, which discloses a first storage device of the processor.),
wherein the first subset of element values of the input feature map is stored in a second storage device of the processor (Botimer discloses, at ¶ [0033], the input feature map values are stored in memory, which discloses a second storage device of the processor.), and
wherein the first storage device comprises a … memory connected to the processor, and wherein the second storage device comprises a plurality of registers in a processing element (PE) of a multiply-accumulate (MAC) tile of the processor … (Botimer discloses, at ¶ [0033], the weights are stored in various types of memory, which discloses a memory connected to the processor. Botimer also discloses, at ¶ [0039], a plurality of shift buffer elements, which discloses registers in a processing element (PE) of a multiply-accumulate (MAC) tile of the processor.).
Botimer does not explicitly disclose the aforementioned first storage device comprises a cache and the aforementioned registers are 9-bit registers.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
cache memory (Baum discloses, at ¶ [0130], cache.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer to include cache, as disclosed by Baum, because cache memory provides improved performance regarding access to data.
Also, in the same field of endeavor (e.g., arithmetic processing) Quisquater discloses
9-bit registers (Quisquater discloses, at col. 10, lines 44-49, 9-bit registers.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer’s shift elements to utilize 9 bits, as disclosed by Quisquater because the number of bits to use is an arbitrary design choice. Therefore, this modification merely entails a combination of prior art elements (cited above) according to known methods to yield predictable results, which is an exemplary rationale to support a conclusion of obviousness, as per MPEP § 2143.
Regarding claims 5, 15, and 21, taking claim 5 as representative, Botimer discloses the elements of claim 4 as discussed above. Botimer also discloses:
shifting, by the processor, the first subset of element values one register to the left in the plurality of registers of the second storage device c shifting, by the processor, the second subset of element values of the input feature map one register to the left in the plurality of registers of the second storage device, wherein the second subset of element values of the input feature map is then stored to the second storage device.);
based on the shifting, performing a second simultaneous multiplication, by the processor, of each value of a second subset of element values of the input feature map with a second element value from among the x * y elements of the convolution kernel, wherein the first subset of element values of the input feature map comprises values in first to p-th column of a first row of the input feature map, and the second subset of element values of the input feature map comprises values in second to (p+1)-th column of the first row of the input feature map, and wherein the second subset of element values of the input feature map further comprises at least one element value from the first subset of element values of the input feature map (Botimer discloses, at ¶¶ [0033]-[0036], ¶ [0039], and Figure 6, simultaneously multiplying a second subset of IFM values, where the second subset is shifted with respect to a first subset, e.g., the first subset of input values and the second set of are in the first and second columns and second and third columns, respectively, by a second value of the weight values matrix. The value from the second column is present in both the first subset and the second subset.); and
shifting, by the processor, the second subset of element values of the input feature map one register to the left in the plurality of registers of the second storage device, wherein the second subset of element values of the input feature map is then stored to the second storage device (Botimer discloses, at ¶ [0039], a plurality of shift buffer elements to store and shift the input feature map values, which discloses shifting, by the processor, the second subset of element values of the input feature map one register to the left in the plurality of registers of the second storage device, wherein the second subset of element values of the input feature map is then stored to the second storage device.);
wherein the first element value from among the x * y elements of the convolution kernel in the first simultaneous multiplication is different from the second element value from among the x * y elements of the convolution kernel in the second simultaneous multiplication (Botimer discloses, at Figure 6 and¶ [0036], the first and second weight values (elements) are different in the first and second simultaneous multiplications, e.g., (0,0,0) in the first and (0,1,0) in the second.).
Regarding claims 6 and 16, taking claim 6 as representative, Botimer discloses the elements of claim 5 as discussed above. Botimer also discloses:
broadcasting, by the processor, the first element value from among x * y elements of the convolution kernel to the second storage device (Botimer discloses, at ¶ [0035], using the first weight value with multiple multiply and accumulate units in calculations with each of the subset of input values, which discloses the claimed broadcasting to the second storage device.); and
broadcasting, by the processor, the second element value from among the x *y elements of the convolution kernel to the second storage device (Botimer discloses, at ¶ [0035], using the second weight value with multiple multiply and accumulate units in calculations with each of the subset of input values, which discloses the claimed broadcasting to the second storage device.).
Regarding claim 7, Botimer discloses the elements of claim 4 as discussed above. Botimer also discloses:
the convolution kernel having the dimension x by y is a weight matrix, the weight matrix comprising a plurality of weight values, the plurality of weight values of the weight matrix are being stored in a … memory connected to the processor, and wherein the input feature map having the dimension n by m comprises a plurality of activation values (Botimer discloses, at ¶ [0032] et seq., the first matrix is a weight matrix and the second matrix is a matrix of input feature map (activation) values.).
Botimer does not explicitly disclose the aforementioned first storage device comprises a cache.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
cache memory (Baum discloses, at ¶ [0130], cache.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer to include cache, as disclosed by Baum, because cache memory provides improved performance regarding access to data.
Regarding claims 8 and 17, taking claim 8 as representative, Botimer discloses the elements of claim 7 as discussed above. Botimer also discloses:
...an adder tree (Botimer discloses, at ¶ [0035], accumulating products, which discloses an adder tree.).
Botimer does not explicitly disclose wherein the processor comprises a plurality of multiply- accumulate (MAC) tiles, each MAC tile comprising an array of processing element (PE) comprising a plurality of PE rows and a plurality of PE columns, each PE column of the plurality of PE columns comprising a plurality of PE....
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
a plurality of multiply- accumulate (MAC) tiles, each MAC tile comprising an array of processing element (PE) comprising a plurality of PE rows and a plurality of PE columns, each PE column of the plurality of PE columns comprising a plurality of PEs... (Baum discloses, at Figure 21D, an array of PEs comprising rows and columns.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer to use a PE array, as disclosed by Baum, because doing so improves performance by providing an efficient mechanism to increase parallelism.
Regarding claim 9, Botimer discloses the elements of claim 8 as discussed above. Botimer does not explicitly disclose wherein each MAC tile further comprises cache memory comprising a plurality of parallel activation lanes, each activation lane corresponding to a row of the array of PEs in the each MAC tile.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
wherein each MAC tile further comprises a cache memory comprising a plurality of parallel activation lanes, each activation lane corresponding to a row of the array of PEs in the MAC tile (Baum discloses, at Figure 21D, each row comprises storage, which discloses the claimed cache with parallel activation lanes corresponding to rows of the PEs.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer’s calculations to run on a PE array utilizing cache, as disclosed by Baum, because cache memory provides improved performance regarding access to data.
Regarding claims 10 and 19, taking claim 10 as representative, Botimer discloses the elements of claim 8 as discussed above. Botimer also discloses:
each PE of the plurality of PEs comprises the plurality of registers configured to store the plurality of activation values of the input feature map, a multiplier connected to the plurality of registers and configured to multiply activations from the input feature map by the weight values from the cache memory, and a PE accumulator connected to the multiplier and configured to add an output from the multiplier to a value from the plurality of registers and store the result in the plurality of registers (Botimer discloses, at ¶ [0039], a plurality of shift buffer elements, which discloses registers, that store the IFM values and are connected to multiply and accumulate units to multiply and accumulate the input and weight values.).
Regarding claim 11, Botimer discloses the elements of claim 8 as discussed above. Botimer does not explicitly disclose wherein the array of PEs in the MAC tile comprises eight PE columns, wherein each PE column of the eight PE columns of the MAC tile comprises sixteen PEs.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
wherein the array of PEs in the MAC tile comprises eight PE columns, wherein each PE column of the eight PE columns of the MAC tile comprises sixteen PEs (Baum discloses, at Figure 21B, an 8x8 array. Baum also discloses, at ¶ [0079], using 16 rows.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer’s calculations to run on a PE array, as disclosed by Baum, in order to improve performance by providing a simple mechanism to increase parallelism.
Regarding claim 18, Botimer discloses the elements of claim 17 as discussed above. Botimer does not explicitly disclose
each MAC tile further comprises cache memory comprising a plurality of parallel activation lanes, each activation lane corresponding to a row of the array of PEs in the each MAC tile, wherein the array of PEs in the each MAC tile comprises eight PE columns, wherein each PE column of the eight PE columns of the each MAC tile comprises sixteen PEs.
However, in the same field of endeavor (e.g., matrix computations) Baum discloses:
wherein each MAC tile further comprises a cache memory comprising a plurality of parallel activation lanes, each activation lane corresponding to a row of the array of PEs in the MAC tile (Baum discloses, at Figure 21D, each row comprises storage, which discloses the claimed cache with parallel activation lanes corresponding to rows of the PEs.); and
wherein the array of PEs in the MAC tile comprises eight PE columns, wherein each PE column of the eight PE columns of the MAC tile comprises sixteen PEs (Baum discloses, at Figure 21B, an 8x8 array. Baum also discloses, at ¶ [0079], using 16 rows.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Botimer’s calculations to run on a PE array utilizing cache, as disclosed by Baum, because cache memory provides improved performance regarding access to data and PE arrays improve performance by providing a simple way to increase parallelism.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to Applicant’s disclosure.
US 8638383 by Jung (Samsung) discloses sequential first and second accumulators.
US 8094058 by Jung (Samsung) discloses sequential first and second accumulators.
US 10713214 by Shalev discloses PEs that each have two sequential accumulators.
US 20120030267 by Rubio discloses a series of accumulators.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAWN DOMAN whose telephone number is (571)270-5677. The examiner can normally be reached on Monday through Friday 8:30am-6pm Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jyoti Mehta can be reached on 571-270-3995. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAWN DOMAN/Primary Examiner, Art Unit 2183