DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 02/06/2026 has been entered.
Accordingly, claims 1-27 and 31-33 are pending in this application. Claims 1, 12, 15, 27 and 33 are currently amended; claims 2 and 19 are original; claims 3-11, 13-14, 16-18, 20-26 and 31-32 are previously presented; claims 28-30 are canceled.
Claim Objections
Claims 1-27 and 31-33 are objected to under 37 C.F.R. 1.71(a) which requires “full, clear, concise, and exact terms” as to enable any person skilled in the art or science to which the invention or discovery appertains, or with which it is most nearly connected, to make and use the same. The following should be corrected.
A. In claim 1 line 14, “grouped adder trees unit” should read “grouped adder tree units” instead for better clarity. Claim 15 recites a similar limitation in line 13 and is objected to for the same reason. Claims 2-14 and 31-32 inherit the same deficiency as claim 1 by reason of dependence. Claims 16-27 and 31-33 inherit the same deficiency as claim 15 by reason of dependence.
B. In claim 33 lines 10-11, “an other adder tree unit that comprises an other multiplier and an other adder tree” should read “another adder tree unit that comprises another multiplier and another adder tree” instead for better clarity.
Claim Interpretation
The broadest reasonable interpretation of a method (or process) claim having contingent limitations requires only those steps that must be performed and does not include steps that are not required to be performed because the condition(s) precedent are not met. For example, assume a method claim requires step A if a first condition happens and step B if a second condition happens. If the claimed invention may be practiced without either the first or second condition happening, then neither step A or B is required by the broadest reasonable interpretation of the claim. If the claimed invention requires the first condition to occur, then the broadest reasonable interpretation of the claim requires step A. If the claimed invention requires both the first and second conditions to occur, then the broadest reasonable interpretation of the claim requires both steps A and B. See MPEP 3111.04 for more information.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 27 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 27 recites “wherein the operator further comprises one or more other adder tree units, including the adder tree unit” in line 2-3. The word “other” means different or additional, therefore,
it is unclear how it is possible for the operator to be comprised of adder tree unit and one or more other adder tree units that also includes the adder tree unit since the one or more other adder tree units are different from the adder tree unit. For purposes of examination, this is interpreted as wherein the operator further comprises one or more other adder tree units.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 15-27 and 33 are rejected under 35 U.S.C. 103 as being unpatentable over Gibson et al. (US 11,625,581 B2), hereinafter Gibson, in view of Lee et al. (US 20190171930 A1), hereinafter Lee.
Regarding claim 33, Gibson teaches a neural network operation method, comprising:
storing an input feature map and a weight (Gibson Fig. 2 and col 6 lines 14-20; input feature map - input data; weight – weights);
mapping the input feature map to an operator and mapping the weight to the operator, (Gibson Fig. 2 and col 6 lines 26-34; operator - convolution engines 240, accumulator 245, and accumulation buffer 250); and
using the operator, performing an operation between the mapped input feature map and the mapped weight (Gibson Figs. 2-3 and col 6 lines 32-57; operation - convolution operation);
wherein the operator comprises an adder tree unit, comprising a multiplier and an adder tree, (Gibson Fig. 3 and col 6 lines 40-43; an adder tree unit – at least one convolution engines 240 and accumulator 245 pair; multiplier – at least one of the plurality of elements of multiply logic 310; adder tree - plurality of elements of addition logic 320), and
a from one weight buffer, to the adder tree unit and another adder tree unit that comprises another multiplier and another adder tree of the operator (Gibson Fig. 2 and col 6 lines 32-34 “weights from the coefficient buffer 230 are provided as a second input into each convolution engine 240a-240n”; one weight buffer - coefficient buffer 230; Fig. 7b and col 17 lines 42-45 “the weight data is received at step 710 and passed to the convolution engines 240a to 240n”; another adder tree unit - at least one of the other 240 and 245 pairs of the n pairs; col 6 lines 26-43; each convolution engine includes a corresponding multiplier and a corresponding adder tree to perform their respective convolution operation).
Gibson does not explicitly teach mapping the input feature map to an operator and mapping the weight to the operator, to provide data parallelism; wherein the operator comprises an adder tree unit, comprising a multiplier and an adder tree, configured to perform the operation by a unit of a reference bit length; and wherein the provision of the data parallelism comprises providing an non-split identical weight, from one weight buffer, to the adder tree unit and another adder tree unit that comprises another multiplier and another adder tree of the operator.
However, on the same field of endeavor, Lee discloses a mapping an input feature map and a same identical non-split weight to a multiplication unit comprising a plurality of sub-multipliers, to provide data parallelism wherein each sub-multiplier is configured to perform an operation by a unit of a reference bit length to the input feature map and the weight (Lee Figs. 8A and 9C and paragraphs [0120-0122 and 0132-0133] unit of a reference bit length – 8-bit by 8-bit multiplication (i.e., the sub-multipliers precision); at least the same weight operand is mapped to 811 and 812 in Fig. 8A; the same first weight is map to all four multipliers in Fig. 9C).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson using Lee and configure the CNN hardware system of Gibson to map the input feature map and the weight to the operator such that non-split identical weights are mapped to the adder tree unit and another adder tree unit and configure the convolution engines 240 and accumulator 245 to perform the convolution operation between the input feature map and the weight by a unit of a reference bit length in order efficiently process the convolution operation by making use of the parallelism of the convolution operation in the neural network by suitably mapping the operand pairs that are capable of being processed in parallel (Lee paragraphs [0122 and 0132-0133]).
Therefore, the combination of Gibson as modified in view of Lee teaches mapping the input feature map to an operator and mapping the weight to the operator, to provide data parallelism; wherein the operator comprises an adder tree unit, comprising a multiplier and an adder tree, configured to perform the operation by a unit of a reference bit length; and wherein the provision of the data parallelism comprises providing an non-split identical weight, from one weight buffer, to the adder tree unit and another adder tree unit that comprises another multiplier and another adder tree of the operator.
Regarding claim 15, Gibson teaches a neural network operation method, comprising:
storing an input feature map and a weight (Gibson Fig. 2 and col 6 lines 14-20; input feature map - input data; weight – weights);
mapping the input feature map to an operator and mapping the weight to the operator, (Gibson Fig. 2 and col 6 lines 29-34; operator - convolution engines 240, accumulator 245, and accumulation buffer 250; plurality of adder tree units - convolution engines 240 and accumulator 245);
using the operator, performing an operation between the mapped input feature map and the mapped weight (Gibson Figs. 2-3 and col 6 lines 34-40; operation - convolution operation);
wherein the operator comprises an adder tree unit, comprising a multiplier and an adder tree, configured to perform the operation (Gibson Fig. 3 and col 6 lines 40-43; multiplier – at least one of the plurality of elements of multiply logic 310; adder tree - plurality of elements of addition logic 320);
in response to a bit length of the input feature map being greater than the reference bit length, dynamically group two or more of a plurality of adder tree units into a single processing entity to perform a multiply-and-accumulate operation between the input feature map and the weight over a plurality of cycles, and mapping the input feature map and the weight to the grouped adder trees unit such that a number of channels of an output feature map is reduced proportionally to the bit length of the input feature map, wherein the single processing entity is configured to sum respective outputs of the two or more adder tree units to generate a single output value for the reduced number of channels, to provide the mixed precision operation (Gibson Figs. 2-3 and col 6 lines 34-40; this is a contingent limitation that is not required to be performed if the bit length of the input feature map is equal to the reference bit length).
Gibson does not explicitly teach mapping the input feature map to an operator and mapping the weight to the operator, to provide one or both of a mixed precision operation and data parallelism; and wherein the operator comprises an adder tree unit, comprising a multiplier and an adder tree, configured to perform the operation by a unit of a reference bit length.
However, on the same field of endeavor, Lee discloses mapping an input feature map and a weight to an operator, to provide one or both of a mixed precision operation and data parallelism, and performing an operation by a unit of a reference bit length using the operator (Lee Figs. 5A-5B, 8A and 9C and paragraphs [0120-0122 and 0132-0133]; unit of a reference bit length – 8-bit (i.e., the precision of the sub-multipliers).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson using Lee and map the input feature map and the weight to the operator, to provide one or both of a mixed precision operation and data parallelism; and configure the operator to perform the operation by the reference bit length in order to increase the utilization of the operator when the inputs to the convolution operation is smaller than the operator precision by providing operands that can be performed in parallel to each of the sub-multipliers for the convolution operation (Lee paragraphs [0107-0108, 0122 and 0132-0133]).
Therefore, the combination of Gibson as modified in view of Lee teaches mapping the input feature map to an operator and mapping the weight to the operator, to provide one or both of a mixed precision operation and data parallelism; and wherein the operator comprises an adder tree unit, comprising a multiplier and an adder tree, configured to perform the operation by a unit of a reference bit length.
Regarding claim 16, Gibson as modified in view of Lee teaches all the limitations of claim 15 as stated above. Further, Gibson as modified in view of Lee teaches wherein the mapping of the input feature map comprises: in response to a bit length of the input feature map being half the reference bit length, increasing a number of channels of the input feature map to be mapped to the operator by a factor of two, to provide the mixed precision operation (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the input feature map is equal to the reference bit length).
Regarding claim 17, Gibson as modified in view of Lee teaches all the limitations of claim 15 as stated above. Further, Gibson as modified in view of Lee teaches wherein the mapping of the weight comprises: in response to a bit length of the weight being half the reference bit length, mapping the weight to the adder tree unit by a unit of a group of two weights, including the weight, that each have half the reference bit length (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length), and
wherein the mapping of the input feature map comprises: in response to a bit length of the input feature map being half the reference bit length, mapping the input feature map to the adder tree unit by a unit of a group of two input feature maps, including the input feature map, that each have half the reference bit length (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the input feature map is equal to the reference bit length).
Regarding claim 18, Gibson as modified in view of Lee teaches all the limitations of claim 17 as stated above. Further, Gibson as modified in view of Lee teaches wherein the performing the operation comprises:
respectively transforming the two input feature maps into first transformed data and second transformed data that each have the reference bit length (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length);
performing a first multiply operation between the first transformed data and a first weight comprised in the group of the two weights, using a first multiplier (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length);
performing a second multiply operation between the second transformed data and a second weight comprised in the group of the two weights, using a second multiplier, where one of the first multiplier and the second multiplier is the multiplier (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length); and
adding an output of the first multiplier, resulting from the performing of the first multiply operation, to an output of the second multiplier, resulting from the performing of the second multiply operation (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length).
Regarding claim 19, Gibson as modified in view of Lee teaches all the limitations of claim 18 as stated above. Further, Gibson as modified in view of Lee teaches wherein the performing the operation further comprises: shifting the output of the first multiplier (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length).
Regarding claim 20, Gibson as modified in view of Lee teaches all the limitations of claim 15 as stated above. Further, Gibson as modified in view of Lee teaches wherein the mapping of the input feature map comprises: in response to the bit length of the input feature map being double the reference bit length, mapping the input feature map and the weight to the operator such that the number of channels of the output feature map is halved, to provide the mixed precision operation (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the input feature map is equal to the reference bit length).
Regarding claim 21, Gibson as modified in view of Lee teaches all the limitations of claim 15 as stated above. Further, Gibson as modified in view of Lee teaches wherein the mapping of the weight comprises: in response to a bit length of the weight being double the reference bit length, grouping two adder tree units together into one group of adder tree units and mapping the weight to the one group of the adder tree units, where one of the two adder tree units is the adder tree unit and another of the two adder tree units is another adder tree unit of the operator (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length).
Regarding claim 22, Gibson as modified in view of Lee teaches all the limitations of claim 21 as stated above. Further, Gibson as modified in view of Lee teaches wherein the mapping of the weight comprises:
mapping a first portion of the weight to the adder tree unit (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length); and
mapping a second portion of the weight to the other adder tree unit of the operator (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length).
Regarding claim 23, Gibson as modified in view of Lee teaches all the limitations of claim 15 as stated above. Further, Gibson as modified in view of Lee teaches wherein the mapping of the input feature map comprises: in response to the bit length of the input feature map being double the reference bit length, grouping two adder tree units together into one group of adder tree units and mapping the input feature map to the one group of the adder tree units, where one of the two adder tree units is the adder tree unit and another of the two adder tree units is another adder tree unit of the operator (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the input feature map is equal to the reference bit length).
Regarding claim 24, Gibson as modified in view of Lee teaches all the limitations of claim 23 as stated above. Further, Gibson as modified in view of Lee teaches wherein the mapping of the input feature map comprises: mapping a first portion of the input feature map to the one group of the adder tree units in a first cycle; and mapping a second portion of the input feature map to the one group of the adder tree units in a second cycle (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the input feature map is equal to the reference bit length).
Regarding claim 25, Gibson as modified in view of Lee teaches all the limitations of claim 15 as stated above. Further, Gibson as modified in view of Lee teaches wherein the performing the operation comprises:
performing a first operation between a first portion of the input feature map and a first portion of the weight in a first cycle, and performing a second operation between a second portion of the input feature map and the first portion of the weight in a second cycle (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the input feature map is equal to the reference bit length); and
performing a third operation between the first portion of the input feature map and a second portion of the weight in the first cycle, and performing a fourth operation between the second portion of the input feature map and the second portion of the weight in the second cycle (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the input feature map is equal to the reference bit length).
Regarding claim 26, Gibson as modified in view of Lee teaches all the limitations of claim 15 as stated above. Further, Gibson as modified in view of Lee teaches
wherein the operator further comprises one or more another adder tree unit, that each comprise a corresponding other multiplier and a corresponding other adder tree (Gibson Fig. 2; one or more other adder tree units – at least one of the other 240 and 245 pairs of the n pairs; col 6 lines 26-43; each convolution engine includes a corresponding multiplier and a corresponding adder tree to perform their respective convolution operation), and
wherein the mapping of the weight comprises:
grouping the adder tree unit and the one or more other adder tree units together into one group of adder tree units by a unit of a weight parallelism size and mapping a same weight to the one group of the adder tree units, to provide the data parallelism (Lee Figs. 8A and 9C and paragraphs [0121-0123 and 0132-0133]). The motivation to combine is the same as claim 15.
Regarding claim 27, Gibson as modified in view of Lee teaches all the limitations of claim 15 as stated above. Further, Gibson as modified in view of Lee teaches wherein the operator further comprises one or more other adder tree units, including the other adder tree unit, each adder tree units comprising a corresponding other multiplier and a corresponding other adder tree (Gibson Fig. 2; one or more other adder tree units – at least one of the other 240 and 245 pairs of the n pairs; col 6 lines 26-43; each convolution engine includes a corresponding multiplier and a corresponding adder tree to perform their respective convolution operation), and
wherein the mapping of the weight comprises: mapping the weight to the operator based on the reference bit length such that a product of a weight parallelism size and a total number of adder tree units, including the adder tree unit and the one or more other adder tree units, is constant, to provide the data parallelism (Lee Fig. 9C and paragraphs [0132-0133]). The motivation to combine is the same as claim 15.
Claims 1-4, 7-27 and 31-32 are rejected under 35 U.S.C. 103 as being unpatentable over Gibson in view of Makino et al. (US 20210011686 A1), hereinafter Makino and Lee.
Regarding claim 1, Gibson teaches a neural network operation device, comprising:
an input feature map buffer configured to store an input feature map (Gibson Fig. 2 and col 6 lines 14-20; input feature map buffer - input buffer 235; input feature map - input data);
a weight buffer configured to store a weight (Gibson Fig. 2 and col 6 lines 14-20; weight buffer - coefficient buffer 230; weight – weights);
an operator comprising a plurality of adder tree units configured to perform an operation between the input feature map and the weight (Gibson Figs. 2-3 and col 6 lines 32-57; operator – convolution engines 240, accumulator 245, and accumulation buffer 250; plurality of adder tree units - convolution engines 240 and accumulator 245; operation - convolution operation); and
a controller (Gibson Fig. 2 and col 5 lines 64-67; controller – command decoder 220, coefficient control buffer controller 225, input buffer controller 215),
wherein each of the adder tree units comprises a multiplier and an adder tree (Gibson Fig. 3 and col 6 lines 40-43; multiplier – at least one of the plurality of elements of multiply logic 310; adder tree - plurality of elements of addition logic 320).
Gibson does not explicitly teach an operator comprising a plurality of adder tree units configured to perform an operation between the input feature map and the weight by a unit of a reference bit length; and a controller configured to map the input feature map and the weight to the operator to provide one or both of a mixed precision operation and data parallelism, wherein the controller is configured to: in response to a bit length of the input feature map being greater than the reference bit length, dynamically group two or more of the plurality of adder tree units into a single processing entity to perform a multiply-and-accumulate operations between the input feature maps and the weight over a plurality of cycles, and map the input feature map and the weight to the grouped adder trees unit such that a number of channels of an output feature map is reduced proportionally to the bit length of the input feature map, wherein the single processing entity is configured to sum respective outputs of the two or more adder tree units to generate a single output value for the reduced number of channels, to provide the mixed precision operation.
However, on the same field of endeavor, Makino discloses a multiplication unit configured to perform an operation between a multiplicand and a multiplier by a unit of a reference bit length; and a controller configured to map the multiplicand and the multiplier to the multiplication unit to provide one or both of a mixed precision operation and data parallelism (Makino Fig. 1 and paragraphs [0066 and 0074-0078]; unit of a reference bit length – half-precision mode/bit length; controller – mode selection section; the mode selection section controls how the multiplicand and the multiplier are mapped and input to the multiplying units; paragraph [0286]). Furthermore, Makino discloses in response to a bit length of an operand (i.e. the multiplicand and/or the multiplier) being greater than the reference bit length, dynamically grouping two multiplying units together into one group (i.e., as a single processing entity) and mapping the multiplicand to the one group of the two multiplying units by providing the higher-order bit portion B10 to the one group in a first clock cycle and the lower-order bit portion B11 to the one group in a second clock cycle, and mapping the operands (the multiplicand and the multiplier) to the one group of two multiplying units to provide the mixed precision operation (Makino Figs. 11-12, 16 and paragraphs [0073, 0077-0078, 0139, 0148, 0157]; bit length being greater than the reference bit length – single-precision and/or double precision; multiplicand – input B). Further, Makino discloses the group is configured to sum respective outputs of the two or more multiplying units to generate a single output value to provide the mixed precision operation (Makino Figs. 11-12, 16 and paragraphs [0153 and 0186]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson in view of Makino and configure the CNN hardware system of Gibson to include a mode selection section for mapping the input feature map and the weight to the operator based on a bit length of the input feature map and the weight and configure the convolution engines 240 and accumulator 245 to perform the convolution operation between the input feature map and the weight by a unit of a reference bit length such that in response to a bit length of the input feature map being double the reference bit length, group two convolution engines and accumulator together into a single processing entity and map the input feature to the group of the two convolution engines and accumulator by providing the higher-order bit portion of the input feature map to the group in a first clock cycle and the lower-order bit portion of the input feature map to the one group in a second first clock cycle and adding the outputs of the group to generate a single convolution output value in order to provide a variable precision CNN hardware system that can efficiently perform convolution operation with high-precision input feature map using lower precision circuitry (Makino paragraph [0010, 0066, 0077-0078]).
Therefore, the combination of Gibson as modified in view of Makino teaches an operator comprising a plurality of adder tree units configured to perform an operation between the input feature map and the weight by a unit of a reference bit length; and a controller configured to map the input feature map and the weight to the operator to provide one or both of a mixed precision operation and data parallelism, wherein the controller is configured to: in response to a bit length of the input feature map being greater than the reference bit length, dynamically group two or more of the plurality of adder tree units into a single processing entity to perform a multiply-and-accumulate operations between the input feature maps and the weight over a plurality of cycles, and map the input feature map and the weight to the grouped adder trees unit, wherein the single processing entity is configured to sum respective outputs of the two or more adder tree units to generate a single output value, to provide the mixed precision operation.
Gibson as currently modified in view of Makino does not explicitly teach wherein the controller is configured to: in response to a bit length of the input feature map being greater than the reference bit length, dynamically group two or more of the plurality of adder tree units into a single processing entity to perform a multiply-and-accumulate operations between the input feature maps and the weight over a plurality of cycles, and map the input feature map and the weight to the grouped adder trees unit such that a number of channels of an output feature map is reduced proportionally to the bit length of the input feature map, wherein the single processing entity is configured to sum respective outputs of the two or more adder tree units to generate a single output value for the reduced number of channels, to provide the mixed precision operation.
However, on the same field of endeavor, Lee discloses a number of channels of an output feature map is reduced proportionally to a bit length of the input feature map. For example, the number of channels of the output feature map is halved when the bit length of the input feature map and a
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson in view of Makino using Lee and generalize the teaching of Lee by mapping the input feature map and the weight to grouped adder trees unit such that a number of channels of an output feature map is reduced proportionally to the bit length of the input feature map (i.e., halved in response to the bit length of the input feature map being double than the reference bit length) so that the convolution operation can still be performed on higher precision input feature maps using lower-precision circuitry (Makino paragraph [0010, 0066]).
Therefore, the combination of Gibson as modified in view of Makino and Lee teaches wherein the controller is configured to: in response to a bit length of the input feature map being greater than the reference bit length, dynamically group two or more of the plurality of adder tree units into a single processing entity to perform a multiply-and-accumulate operations between the input feature maps and the weight over a plurality of cycles, and map the input feature map and the weight to the grouped adder trees unit such that a number of channels of an output feature map is reduced proportionally to the bit length of the input feature map, wherein the single processing entity is configured to sum respective outputs of the two or more adder tree units to generate a single output value for the reduced number of channels, to provide the mixed precision operation.
Regarding claim 2, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above.
Gibson as currently modified in view of Makino and Lee does not explicitly teach wherein the controller is configured to: in response to a bit length of the input feature map being half the reference bit length, increase a number of channels of the input feature map to be mapped to the operator by a factor of two, to provide the mixed precision operation.
However, on the same field of endeavor, Lee discloses increasing a number of channels of an input feature map to be mapped to an operator by a factor of two, to provide a mixed precision operation in response to a bit length of the input feature map being half a reference bit length (Lee Fig. 5B and paragraph [0108]; pixel value
F
1
and
F
0
are from different channels; bit length of the input feature map – 8-bit; reference bit length – 16-bit).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson in view of Makino and Lee using Lee and configure the controller to increase the number of channels of the input feature map to be mapped to the convolution engines and accumulator by a factor of two, to provide a mixed precision operation in response to a bit length of the input feature map being half a reference bit length in order to optimize the utilization and/or double the throughput of the convolution engines (Lee Fig. 5B and paragraph [0103, 0109]).
Therefore, the combination of Gibson as modified in view of Makino and Lee teaches wherein the controller is configured to: in response to a bit length of the input feature map being half the reference bit length, increase a number of channels of the input feature map to be mapped to the operator by a factor of two, to provide the mixed precision operation.
Regarding claim 3, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above.
Gibson as currently modified in view of Makino and Lee does not explicitly teach wherein the controller is configured to: in response to a bit length of the weight being half the reference bit length, map the weight to an adder tree unit by a unit of a group of two weights, including the weight, that each have half the reference bit length; and in response to a bit length of the input feature map being half the reference bit length, map the input feature map to the adder tree unit by a unit of a group of two input feature maps, including the input feature map, that each have half the reference bit length.
However, on the same field of endeavor, Lee discloses in response to a bit length of a weight being half a reference bit length, map the weight to a multiplier unit by a unit of a group of two weights each having half the reference bit length; and in response to a bit length of an input feature map being half the reference bit length, map an input feature map to the multiplier unit by a unit of a group of two input feature maps each having half the reference bit length (Lee Figs. 5a-5b and paragraphs [0107-0109]; group of two group of two weights –
W
00
(
0,0
)
and
W
01
(
0,0
)
; group of two input feature maps –
F
1
(
0,0
)
and
F
0
(0,0); reference bit length – 16-bit; half the reference bit length – 8-bit).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson in view of Makino and Lee using Lee and configure the controller to group the weights and input feature maps in a group of two in response to the bit length of the weights and the input feature maps being half the reference bit length and input the group to the convolution engines in order to optimize the utilization and/or double the throughput of the convolution engines (Lee Fig. 5B and paragraph [0103, 0109]).
Therefore, the combination of Gibson as modified in view of Makino and Lee teaches wherein the controller is configured to: in response to a bit length of the weight being half the reference bit length, map the weight to an adder tree unit by a unit of a group of two weights, including the weight, that each have half the reference bit length; and in response to a bit length of the input feature map being half the reference bit length, map the input feature map to the adder tree unit by a unit of a group of two input feature maps, including the input feature map, that each have half the reference bit length.
Regarding claim 4, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 3 as stated above. Further, Gibson as modified in view of Makino and Lee teaches wherein the adder tree unit comprises:
a multiplier portion, comprising the multiplier configured to perform a multiply operation between the input feature map and the weight (Gibson Fig. 3 and col 6 lines 40-43; multiplier portion comprising the multiplier - plurality of elements of multiply logic 310);
the adder tree that is configured to add outputs of the multiplier portion (Gibson Fig. 3 and col 6 lines 40-43; adder tree - plurality of elements of addition logic 320; claim 14); and
an accumulator configured to accumulate and sum outputs of the adder tree (Gibson Fig. 2 and col 15 lines 34-35; accumulator - accumulators 245; claim 10).
Regarding claim 7, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above. Further, Gibson as modified in view of Makino teaches wherein the controller is configured to: in response to the bit length of the input feature map being double the reference bit length, map the input feature map and the weight to the operator such that the number of channels of the output feature map is halved, to provide the mixed precision operation (Lee Figs. 4a and 5B; 4a only generates a single output while 5B produces two outputs each in different channel; see also claim 1 analysis. The motivation to combine is the same as claim 1).
Regarding claim 8, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above.
Gibson as currently modified in view of Makino and Lee does not explicitly teach wherein the controller is configured to: in response to a bit length of the weight being double the reference bit length, group two adder tree units together into one group of adder tree units and map the weight to the one group of the adder tree units, where one of the two adder tree units is an adder tree unit and another of the two adder tree units is another adder tree unit of the operator.
However, on the same field of endeavor, Makino discloses the controller is configured to: in response to a bit length of the multiplier being double the reference bit length, group two multiplying units together into one group and mapping the multiplier to the one group of the two multiplying units by providing the higher-order bit portion A10 to a first multiplying unit and the lower-order bit portion A11 to a second multiplying unit of the group of two multiplying units (Makino Figs. 11-12 and paragraphs [0077-0078, 0139]; double the reference bit length - single-precision; multiplier – input A).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson in view of Makino and Lee and configure the controller to group two convolution engines and accumulators together into one group and map the weight to the group of the two convolution engines and accumulator by providing the higher-order bit portion of the weight to a first convolution engine and accumulator and the lower-order bit portion of the weight to a second two convolution engine and accumulator in order to provide a variable precision CNN hardware system that can efficiently perform convolution operation with high-precision weights using lower precision circuitry (Makino paragraph [0010, 0066, 0077-0078]).
Therefore, the combination of Gibson as modified in view of Makino and Lee teaches wherein the controller is configured to: in response to a bit length of the weight being double the reference bit length, group two adder tree units together into one group of adder tree units and map the weight to the one group of the adder tree units, where one of the two adder tree units is an adder tree unit and another of the two adder tree units is another adder tree unit of the operator.
Regarding claim 9, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 8 as stated above. Further, Gibson as modified in view of Makino and Lee teaches wherein the controller is configured to: map a first portion of the weight to the adder tree unit, and map a second portion of the weight to the other adder tree unit of the operator (Makino Fig. 12 and paragraph [0139]; see also claim 8 analysis). The motivation to combine is the same as claim 8.
Regarding claim 10, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above. Further, Gibson as modified in view of Makino and Lee does teaches wherein the controller is configured to: in response to the bit length of the input feature map being double the reference bit length, group two adder tree units together into one group of adder tree units and map the input feature map to the one group of the adder tree units where one of the two adder tree units is an adder tree unit and another of the two adder tree units is another adder tree unit of the operator (Makino Figs. 11-12 and paragraphs [0077-0078, 0139, 0148]; double the reference bit length - single-precision; multiplicand – input B). The motivation to combine is the same as claim 1.
Regarding claim 11, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 10 as stated above. Further, Gibson as modified in view of Makino and Lee wherein the controller is configured to: map a first portion of the input feature map to the one group of the adder tree units in a first cycle (Makino Fig. 12 and paragraphs [0138-0139]), and map a second portion of the input feature map to the one group of the adder tree units in a second cycle (Makino Fig. 12 and paragraph [0148]). The motivation to combine is the same as claim 1.
Regarding claim 12, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above. Further, Gibson as modified in view of Makino and Lee teaches wherein the operator further comprises another adder tree unit (Gibson Fig. 2; other adder tree unit – at least one other convolution engine and accumulator pair of the n pairs, for example, 240b and 245b).
Gibson as currently modified in view of Makino does not explicitly teach wherein an adder tree unit is configured to: perform a first operation between a first portion of the input feature map and a first portion of the weight in a first cycle, and perform a second operation between a second portion of the input feature map and the first portion of the weight in a second cycle, and wherein the other adder tree unit is configured to: perform a third operation between the first portion of the input feature map and a second portion of the weight in the first cycle, and perform a fourth operation between the second portion of the input feature map and the second portion of the weight in the second cycle.
However, on the same field of endeavor, Makino discloses a first multiplying unit configured to: perform an operation between a first portion of a multiplicand and a first portion of a multiplier in a first cycle, and perform an operation between a second portion of the multiplicand and the first portion of the multiplier in a second cycle (Makino Fig. 12 and paragraphs [0139,0148]; operation – multiplication; multiplicand – B1; first portion of the multiplicand – B10; multiplier – A1; first portion of the multiplier A10; second portion of the multiplicand – B11; first multiplying unit – 2a). Furthermore, Makino discloses a second multiplying unit configured to: perform an operation between the first portion of the multiplicand and a second portion of multiplier in the first cycle, and perform an operation between the second portion of the multiplicand and the second portion of the multiplier in the second cycle (Makino Fig. 12 and paragraphs [0139, 0148]; operation – multiplication; second portion of the multiplier – A11; second multiplying unit – 2b).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson in view of Makino and Lee and configure 240a and 245a to process a first portion of the input feature map and a first portion of the weight in a first cycle, and process a second portion of the input feature map and the first portion of the weight in a second cycle, and configure 240b and 245b to process the first portion of the input feature map and a second portion of the weight in the first cycle, and process the second portion of the input feature map and the second portion of the weight in the second cycle in order to provide a variable precision CNN hardware system that can efficiently perform convolution operation with high-precision input feature map and/or weight using lower precision circuitry (Makino paragraph [0010, 0066, 0077-0078]).
Therefore, the combination of Gibson as modified in view of Makino teaches wherein an adder tree unit is configured to: perform a first operation between a first portion of the input feature map and a first portion of the weight in a first cycle, and perform a second operation between a second portion of the input feature map and the first portion of the weight in a second cycle, and wherein the other adder tree unit is configured to: perform a third operation between the first portion of the input feature map and a second portion of the weight in the first cycle, and perform a fourth operation between the second portion of the input feature map and the second portion of the weight in the second cycle.
Regarding claim 13, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above. Further, Gibson as modified in view of Makino teaches wherein the operator further comprises one or more other adder tree units, including another adder tree unit, that each comprise a corresponding other multiplier and a corresponding other adder tree (Gibson Fig. 2; one or more other adder tree units – at least one of the other 240 and 245 pairs of the n pairs; col 6 lines 26-43; each convolution engine includes a corresponding multiplier and a corresponding adder tree to perform their respective convolution operation), and
(Gibson Fig. 2; the weight is passed to 240a-240n; Fig. 7b and col 17 lines 42-45 “the weight data is received at step 710 and passed to the convolution engines 240a to 240n”; weight parallelism size - n).
Gibson does not explicitly teach wherein the controller is configured to: group an adder tree unit and the one or more other adder tree units together into one group of adder tree units by a unit of a weight parallelism size and map a same weight to the one group of the adder tree units, to provide the data parallelism.
However, on the same field of endeavor, Makino discloses the controller is configured to group the multiplying units together into one group and mapping the same multiplicand portion to the one group of the adder tree units (Makino Fig. 16 and paragraph [0158]).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson in view of Makino and Lee and generalize the teaching of Makino by grouping the convolution engines 240 and accumulators 240 into one group such that the same weight is mapped to the one group efficiently perform the convolution operation by making full use of the parallelism of a convolution operation in a neural network (Lee Fig. 9C and paragraph [0126 and 0133-0134]). Furthermore, combining multiple elements as one group is obvious to one of ordinary skill in the art (Gibson col 4 lines 8-16). Furthermore, by grouping the convolution engines 240 and accumulators 240 into one group, the CNN hardware may be reduced by using a single coefficient buffer that provides the same weight to the whole group (Gibson Fig. 1).
Therefore, the combination of Gibson as modified in view of Makino and Lee teaches wherein the controller is configured to: group an adder tree unit and the one or more other adder tree units together into one group of adder tree units by a unit of a weight parallelism size and map a same weight to the one group of the adder tree units, to provide the data parallelism.
Regarding claim 14, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above. Further, Gibson as modified in view of Makino and Lee teaches wherein the operator further comprises one or more other adder tree units, including another adder tree unit, that each comprise a corresponding other multiplier and a corresponding other adder tree (Gibson Fig. 2; one or more other adder tree units – at least one of the other 240 and 245 pairs of the n pairs; col 6 lines 26-43; each convolution engine includes a corresponding multiplier and a corresponding adder tree to perform their respective convolution operation), and
wherein the controller is configured to:
map the weight to the operator by matching the weight to the reference bit length such that a product of a weight parallelism size and a total number of the adder tree units including an adder tree unit and the one or more other adder tree units is constant, to provide the data parallelism (Makino Fig. 9 and paragraph [0076]; weight parallelism size – 4; the product of 4xn is a constant value 4n). The motivation to combine is the same as claim 1.
Regarding claim 31, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above. Further, Gibson as modified in view of Makino and Lee teaches wherein each of the plurality of adder tree units further comprises an accumulator configured to accumulate and sum outputs of the adder tree (Gibson Fig. 2 and col 15 lines 34-35; accumulator – at least one of the accumulators 245; claim 10).
Regarding claim 32, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 1 as stated above. Further, Gibson as modified in view of Makino and Lee teaches wherein the controller is configured to map the input feature map and the weight to the operator to provide both the mixed precision operation and the data parallelism (Makino Figs. 9, 12 and 16 and paragraphs [0076-0078, 0286]). The motivation to combine is the same as claim 1.
Regarding claims 15-17 and 20-271, they are directed to a method that is practiced by the device of claims 1-3 and 7-14 respectively. All steps performed by the method of claims 15-17 and 20-27 would be practiced by the device of claims 1-3 and 7-14 respectively. Claims 1-3 and 7-14 analysis applies equally to claims 15-17 and 20-27 respectively.
Regarding claim 18, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 17 as stated above. Further, Gibson as modified in view of Makino and Lee teaches wherein the performing the operation comprises:
respectively transforming the two input feature maps into first transformed data and second transformed data that each have the reference bit length (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length);
performing a first multiply operation between the first transformed data and a first weight comprised in the group of the two weights, using a first multiplier (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length);
performing a second multiply operation between the second transformed data and a second weight comprised in the group of the two weights, using a second multiplier, where one of the first multiplier and the second multiplier is the multiplier (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length); and
adding an output of the first multiplier, resulting from the performing of the first multiply operation, to an output of the second multiplier, resulting from the performing of the second multiply operation (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length).
Regarding claim 19, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 18 as stated above. Further, Gibson as modified in view of Makino and Lee teaches wherein the performing the operation further comprises: shifting the output of the first multiplier (Gibson Fig. 2 and col 6 lines 29-40; this is a contingent limitation that is not required to be performed if the bit length of the weight is equal to the reference bit length and if the bit length of the input feature map is equal to the reference bit length).
Claims 5-6 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Gibson in view of Makino and Lee as applied to claim 4 above, and further in view of Makineni et al. (US 5880985 A), hereinafter Makineni.
Regarding claim 5, Gibson as modified in view of Makino and Lee teaches all the limitations of claim 4 as stated above. Further, Gibson as modified in view of Makino and Lee teaches
wherein the multiplier portion comprises:
a first multiplier configured to perform a first multiply operation between a first (Gibson Fig. 3 and col 6 lines 34-43; first multiplier – one of the multiply logic 310); and
a second multiplier configured to perform a second multiply operation between a second (Gibson Fig. 3 and col 6 lines 34-43; second multiplier – one of the multiply logic 310 different from the first multiplier).
Gibson does not explicitly teach a first transformed data; and second transformed data; wherein the first transformed data and the second transformed data are the two input feature maps that have been respectively transformed to each have the reference bit length.
However, on the same field of endeavor, Makineni discloses a first multiplexer configured to multiplex respective inputs comprised in a group of two inputs into a first transformed data and a second transformed data that each have bit length being twice the bit length of the original inputs; and a first multiplier configured to perform a multiply operation between the first transformed data and a first second operand comprised in a group of two second operands; and a second multiplier configured to perform a multiply operation between the second transformed data and a second second operand comprised in the group of the two second operands (Makineni Figs. 2-3, col 3 lines 46-57 and col 4 lines 12-30; first transformed data – A’; second transformed data - A’’).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson in view of Makino and Lee using Makineni and configure the multiplexers to multiplex respective input feature maps comprised in the group of the two input feature maps into first transformed data and second transformed data that each have the reference bit length in order to extend the bit length of the input feature maps such that the input feature maps matches the precision of the multipliers (Makineni Figs. 2-3 and col 4 lines 12-30; claim 5), then perform the multiplication operations using the first transformed data and second transformed data.
Therefore, the combination of Gibson as modified in view of Makino, Lee and Makineni teaches a first transformed data; and second transformed data; wherein the first transformed data and the second transformed data are the two input feature maps that have been respectively transformed to each have the reference bit length.
Regarding claim 6, Gibson as modified in view of Makino, Lee and Makineni teaches all the limitations of claim 5 as stated above. Further, Gibson as modified in view of Makino, Lee and Makineni teaches wherein the adder tree unit further comprises: an adder configured to add an output of the first multiplier to an output of the second multiplier resulting from the performed second multiply operation (Gibson Fig. 3; adder - addition logic 320).
Gibson does not explicitly teach a shifter configured to shift an output of the first multiplier, resulting from the performed first multiply operation; and an adder configured to add an output of the shifter to an output of the second multiplier.
However, on the same field of endeavor, Makino discloses a shifter configured to shift an output of a multiplier; and an adder configured to add an output of the shifter (Makino Fig. 12 and paragraphs [0141, 0143]; shifter - shifter).
Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention, to modify Gibson in view of Makino, Lee and Makineni and generalize the teaching of Makino by including a shifter to shift an output of the first multiplier and configure the adder to add an output of the shifter to the output of the second multiplier in order to properly align the outputs of the multipliers before being added so that the convolution operation can be calculated correctly. As discussed in col 5 lines 19-23 and col 5 lines 38-42 of Makineni, the outputs of multipliers 51 and 53 are located in different portions of the 24-bit result, therefore in order to correctly add the outputs of the multipliers when performing the convolution operation, the result needs to be aligned correctly before being added to each other.
Therefore, the combination of Gibson as modified in view of Makino, Lee and Makineni teaches wherein the adder tree unit further comprises: a shifter configured to shift an output of the first multiplier, resulting from the performed first multiply operation; and an adder configured to add an output of the shifter to an output of the second multiplier, resulting from the performed second multiply operation.
Regarding claims 18-19, they are directed to a method that is practiced by the device of claim 6. All steps performed by the method of claims 18-19 would be practiced by the device of claim 6. Claim 6 analysis applies equally to claims 18-19.
Response to Arguments
Applicant’s arguments, see remarks page 17, filed 02/06/2026, with respect to the rejection(s) of claim(s) 33 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of amendments made and previously cited prior art references.
Applicant argues the following:
1.) Makino does not teach or suggest “a non-split identical weight”.
Response: Examiner agrees. The multiplicand segments B10/B11 in Makino are divided portions of the multiplicand (weight). However, this feature is disclosed in at least Figs. 8A and 9C and paragraphs [0121-0123 and 0132-0133] of Lee which discloses providing the same full weight value to at least two sub-multipliers of a multiplier in order to efficiently process a convolution operation.
Applicant’s arguments, see remarks page 12-17, filed 02/06/2026, with respect to the rejection(s) of claim(s) 1-27 and 31-32 under 35 U.S.C. 103 have been fully considered but they are not persuasive.
Applicant argues the following:
1.) None of the prior art references cited teach or suggest “dynamic grouping adder tree units for multi-cycle operations with channel reduction to perform a proportional reduction based on the bit length of the input feature map”.
Response: Examiner respectfully disagrees. Paragraph [0073] of Makino discloses “the mode selecting section 460 dynamically control each section of the arithmetic operation device 405 to operate in the designated calculation precision mode.” Further, paragraph [0077] of Makino discloses grouping two multiplying units and two adding units together into a group in response to a bit-length of the multiplier and the multiplicand being greater (double) the reference bit length. Furthermore, Figs. 11-12 and corresponding paragraphs starting at paragraph [0139] discloses the group of two multiplying units and two adding units performing multiply and accumulate operations over a plurality of clock cycles such that during the first clock cycle, multiplication between bits B10 of the multiplicand and bits A10 and A11 of the multiplier are performed, and in the second clock cycle multiplication between bits B11 of the multiplicand and bits A10 and A11 of the multiplier are performed. Further, the results of the multiplication are added together to generate the product of multiplier and the multiplicand. Furthermore, Lee discloses the concept of reducing the number of channels of an output feature map when a bit length of an input feature map and a weight is doubled as disclosed in Figs. 4a and 5B where the channel reduction is proportional based on the bit length of the input feature map (i.e., the number of output feature map channels is reduced by half when the bit length of the input feature map is doubled).
2.) None of the prior art references teach or suggest “wherein the single processing entity is configured to sum respective outputs of the two or more adder tree units to generate a single output value for the reduced number of channels” because Lee fails to teach or suggest the recited summing of the “respective outputs of the two or more adder tree units”, as recited in claim 1.
Response: Examiner respectfully disagrees. This feature is disclosed in at least Figs. 11-12, 16 and paragraphs [0153 and 0186] of Makino which discloses that the single-precision calculation mode, a first product is output in terminals OP1 and OP2, therefore, the adding units 4a and 4b sum the respective outputs of the multiplying units 2a and 2b. For the double-precision calculation mode, the product is output in terminals OP1-OP4, therefore, the adding units 4a-4d sum the respective outputs of the multiplying units 2a-2d. Furthermore, as discussed above, “the reduced number of channels” is disclosed in Lee.
3.) Lee does not describe reducing “a number of channels of an output feature map”. Instead, Lee finds a number of channels in an output feature map based on a number of kernels applied during convolution and on the actual architecture of the neural network. FIG. 4A of Lee merely describes the potential for full utilization of multiplier when both inputs are 16-bit operands. FIG. 5B of Lee likewise describes that four 8-bit operands are provided to sub-multipliers within a 16-bit multiplier to allow for parallel processing of four operand pairs. Lee does not teach that “a number of channels of an output feature map is reduced/halved when a bit length of an input feature map and a weight is doubled,” but rather, Lee describes the performance and efficiency for a 16-bit multiplier when it is provided with 8-bit inputs. That is, Lee does not address feature maps or weights. Lee likewise does not show a reduction or halving. Instead, Lee illustrates the impact of 8-bit inputs into 16-bit multipliers.
Response: Examiner agrees in part. Examiner agrees that Lee discloses the utilization of a 16-bit by 16-bit multipliers comprising four 8-bit by 8-bit sub-multipliers when a bit length of the input feature map is 16-bit and 8-bit. However, examiner respectfully disagrees that Lee not describe reducing a number of channels of an output feature map. It is clear from Fig. 5B and paragraphs [0108-0110] that the 16-bit multiplier generates two output feature map in a different output channel when the feature map is 8-bit and in Fig. 4A only generates one output feature map when the feature map is 16-bit. Therefore, the number of channels of the output feature map is reduced in half when the bit length of the input feature map is doubled.
4.) Makino does not teach or suggest “in response to a bit length of the input feature map being greater than the reference bit length, dynamically group two or more of the plurality of adder tree units into a single processing entity to perform a multiply-and-accumulate operation between the input feature map and the weight over a plurality of cycles, and map the input feature map and the weight to the grouped adder trees unit such that a number of channels of an output feature map is reduced proportionally to the bit length of the input feature map, to provide the mixed precision operation” because Makino’s grouping is not structural. Specifically, Makino groups multipliers for internal precision scaling, not for multi-cycle MAC operations. Makino and Lee both operate completely differently than that from the claimed MAC operations. Further, Makino does not perform these operations “proportionally to the bit length of the input feature map, to provide the mixed precision operation”.
Response: Examiner respectfully disagrees. As discussed above, Lee disclosed the underlined claimed features. Further, it is well-known that a multi-cycle multiplication operation such as disclosed in Makino Figs. 12 and 16 comprises a multiplication operation and partial-product accumulation operation which can be generalized to implement a multi-cycle MAC operation that can be used to perform convolution operations. Furthermore, Makino’s grouping is structural. Compare at least Figs. 12 and 16 where circuit connections are different in the different mode.
5.) There is no motivation to combine Gibson, Makino and Lee because one of ordinary skill in the art would not look to either of these secondary references to improve Gibson.
Response: Examiner respectfully disagrees. At least col 5 lines 15-34 of Gibson discloses that the hardware implementation is a configurable hardware that can be configured to implement a specific CNN including size and values of the weight and the format of the input. Therefore, one of ordinary skill in the art would be motivated to combine the teachings of Gibson, Makino and Lee in order to provide a variable precision CNN hardware system that can efficiently perform convolution operation with high-precision input feature map using lower precision circuitry in a case where the format (bit length) of the input is larger than the bit length of the convolution circuitry (i.e., the convolution engines and accumulator).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Carlo Waje whose telephone number is (571)272-5767. The examiner can normally be reached 9:00-6:00 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached on (571) 272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Carlo Waje/Examiner, Art Unit 2182 (571)272-5767
1 Examiner notes that although the method claims are rejected using the same analysis and the device claims, the method claims includes contingent limitations that are not required by the claims if the precedent condition(s) are not met.