Detailed Action
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is final and is in response to claims filed on 02/12/2026 via amendment. Claims 1-20 are pending for examination. Claims 1-2, 5, 10-11, 13, 16-17, and 19 are currently amended. Claims 3-4, 6-9, 12, 14-15, 18, and 20 are as previously filed.
Response to Arguments
Rejections under 35 U.S.C. 101
Applicant’s arguments, see Remarks 11, filed 02/12/2026, with respect to the rejections under 35 U.S.C. 101 have been fully considered. Applicant merely argues that the Rejections under 35 U.S.C. 101 should be withdrawn because it was agreed upon in an interview. Applicant has not provided any further supporting arguments. Nevertheless, Examiner notes that the amendments overcome the rejection because of the limitations of how the hardware components are connected together and the specific hardware integrates the judicial exceptions into a practical application.
Rejections under 35 U.S.C. 103
Applicant’s arguments regarding the 35 U.S.C. 103 rejections have been fully considered. Applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references. Examiner notes that the claims remain rejected for the reasons discussed below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Claim 16 recites “means for receiving map data”, “means for receiving kernel data”, “means for receiving an indication of an input precision mode”, “means for receiving an indication of an output precision mode”, “means for calculating at least one accumulation of products”, “means for generating an output”, “means for concatenating the output”, and “means for outputting”.
The specification, in paragraph [0212], recites “receiving map data via a first port. In some implementations, the method includes receiving kernel data via a second port. In some implementations, the method includes receiving, via a third port, an indication of an input precision mode that indicates an input word length for the map data and for the kernel data. In some implementations, the method includes receiving, via a fourth port, an indication of an output precision mode that indicates an output word length. In some implementations, the method includes generating, using a plurality of vector-vector (VV) components, a corresponding plurality of VV outputs based on the map data, the kernel data, the input precision mode, the output precision mode, and at least one accumulation of products. In some implementations, the method includes concatenating, using one or more integrated circuits, the corresponding plurality of VV outputs to generate a concatenated VV output. In some implementations, the method includes outputting the concatenated VV output via a fifth port.”
Claim 17 recites “means for generating a plurality of products” and “means for adding the plurality of products”.
The specification, in paragraph [0211], recites “In some implementations, each accumulation of products, of the at least one accumulation of products, is calculated by adding a plurality of products”. The specification, in paragraph [0212], recites “in some implementations, the method includes receiving, via a fourth port, an indication of an output precision mode that indicates an output word length. In some implementations, the method includes generating, using a plurality of vector-vector (VV) components, a corresponding plurality of VV outputs based on the map data, the kernel data, the input precision mode, the output precision mode, and at least one accumulation of products.”
Claim 18 recites “means for rounding”.
The specification, in paragraph [0084], recites “a rounding component 430”.
Claim 19 recites “means for generating a plurality of products”, “means for adding the plurality of products”, and “means for adding the accumulation of products”.
The specification, in paragraph [0211], recites “In some implementations, each accumulation of products, of the at least one accumulation of products, is calculated by adding a plurality of products”. The specification, in paragraph [0212], recites “in some implementations, the method includes receiving, via a fourth port, an indication of an output precision mode that indicates an output word length. In some implementations, the method includes generating, using a plurality of vector-vector (VV) components, a corresponding plurality of VV outputs based on the map data, the kernel data, the input precision mode, the output precision mode, and at least one accumulation of products.”
Claim 20 recites “means for summing the multiple accumulations of products” and “means for rounding”.
The specification, in paragraph [0211], recites “In some implementations, each accumulation of products, of the at least one accumulation of products, is calculated by adding a plurality of products”. The specification, in paragraph [0084], recites “a rounding component 430”.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Redfern et al. (US 20180246855 A1) hereinafter Redfern in view of Culurciello et al. (US 20180341495 A1) hereinafter Culurciello further in view of Szegedy et al. (“Going deeper with convolutions”) hereinafter Szegedy.
With regards to claim 1, Redfern teaches A device, comprising: [a plurality of] vector-vector (VV) components (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM)
that are each configured to generate a VV output based on an input precision mode, an output precision mode, and at least one accumulation of products, (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity)
wherein each VV component of [the plurality of VV components] comprises: an input precision mode port; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
and a plurality of multiply-accumulate (MAC) components, (Redfern [0257]: The exemplary top level floorplan depicted in FIG. 40 (4000) is composed of an array of hierarchical multiply-accumulate modules (4011, 4012, 4013, 4014, 4021, 4022, 4023, 4024, 4031, 4032, 4033, 4034, 4041, 4042, 4043, 4044) a data path for operand pre-processing, and an interface/control region (4001))
each MAC component of the plurality of MAC components coupled with the input precision mode port, (Redfern [0257]: The exemplary top level floorplan depicted in FIG. 40 (4000) is composed of an array of hierarchical multiply-accumulate modules (4011, 4012, 4013, 4014, 4021, 4022, 4023, 4024, 4031, 4032, 4033, 4034, 4041, 4042, 4043, 4044) a data path for operand pre-processing, and an interface/control region (4001)Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
wherein each accumulation of products, of the at least one accumulation of products, is calculated by a respective MAC component, of the plurality of MAC components, by adding a plurality of products based on the input precision mode, (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix)
wherein each product, of the plurality of products, is calculated by a respective MAC component, of the plurality of MAC components, by multiplying a respective [map] word of [map data] and a respective [kernel] word [of kernel data] based on the input precision mode, (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix)
and wherein the output precision mode indicates an output word length for the VV output; (Redfern [0084]-[106]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM... While output data of the RMM may incorporate a variety of data types, the following output data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity)
and an output port configured to output the [concatenated] VV output (Redfern [0251]: As shown in FIG. 38 (3800), dataflow in the RMM is fixed and proceeds from the operand inputs to operand storage. The matrix multiplication operands are next read and multiplied and added with an optional earlier value from the CPM matrix storage. The CPM matrix results are passed through an output processing block to an output buffer which is accessed by the ACL/CPU).
Redfern fails to teach a plurality [of vector-vector (VV) components] and that the inputs are maps and kernels.
However, Culurciello teaches a plurality [of vector-vector (VV) components] (Culurciello [0010]: In one embodiment, an accelerator for processing of a convolutional neural network (CNN) has been developed. The accelerator includes a compute core. The compute core includes a plurality of compute units)
And that the inputs of Redfern are maps and kernels (Culurciello [0036]: In the accelerators 100 and 200, each compute core 120 implements the processing of the CNNs including processing of traces of input map data and traces of the weight values for the kernels that are used in different layers of the CNN).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern with the plurality of components and that the inputs are maps and kernels as taught by Culurciello. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as data could be processed in parallel. Also, the accelerators 100 and 200 can operate in different modes that utilize different forms of parallelism to process CNNs with complex structures more efficiently as taught by Culurciello (Culurciello [0055]).
Redfern in view of Culurciello fails to teach one or more components configured to concatenate a plurality of VV outputs, corresponding to the plurality of VV components, to generate a concatenated VV output, and outputting the concatenated VV output.
However, Szegedy teaches one or more components configured to concatenate a plurality of VV outputs, corresponding to the plurality of VV components, to generate a concatenated VV output; (Szegedy Page 4 Section 4: It also means that the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector)
and outputting the concatenated VV output of Redfern in view of Culurciello (Szegedy Page 4 Section 4: It also means that the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern in view of Culurciello with the concatenated output as taught by Szegedy. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the different layers could be processed in parallel and then concatenated together. Also, one of the main beneficial aspects of this architecture is that it allows for increasing the number of units at each stage significantly without an uncontrolled blow-up in computational complexity as taught by Szegedy (Szegedy Page 5 Section 4).
With regards to claim 2, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 1 above. Redfern further teaches
one or more buses configured to provide the indication of the input precision mode from the input precision mode port to the plurality of VV components (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals).
With regards to claim 3, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 1 above. Redfern further teaches an output precision mode port configured to receive an indication of the output precision mode; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
and one or more buses configured to provide the indication of the output precision mode from the output precision mode port to the plurality of VV components (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals).
With regards to claim 4, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 1 above. Redfern further teaches a coordination mode port configured to receive an indication of a coordination mode [that indicates whether an accumulation of products, of the at least one accumulation of products, is to be summed with one or more other accumulations of products;] (Redfern [0314]: While the function/opcode interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure; Redfern Fig. 1: shows a function port)
and one or more buses configured to provide the indication of the coordination mode from the coordination mode port to the plurality of VV components (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM; Redfern Fig. 1 shows a function bus).
Redfern fails to teach that the coordination mode indicates whether an accumulation of products, of the at least one accumulation of products, is to be summed with one or more other accumulations of products.
However, Culurciello teaches that the coordination mode of Redfern indicates whether an accumulation of products, of the at least one accumulation of products, is to be summed with one or more other accumulations of products (Culurciello [0045]: The gather adder 258 adds a single bias term B that is stored in the weights cache 136 as part of the total sum of all the MACs for at least one vector of map input data in the cooperative mode to produce a single output value M1 for a set of outputs from all of the MACs in a single vMAC. By contrast, in the independent operation mode that that is described in more detail below, the gather adder 258 receives the output of a single MAC from the shift register 252 during each cycle of operation and adds the bias term B to generate a single output. In the independent operating mode, the gather adder 258 generates 16 individual outputs).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern in view of Culurciello further in view of Szegedy with the indication of whether the accumulated products should be summed as taught by Culurciello. One of ordinary skill in the art would be motivated to make this combination for at least the same reasons as stated above. Also, the accelerators 100 and 200 can operate in different modes that utilize different forms of parallelism to process CNNs with complex structures more efficiently as taught by Culurciello (Culurciello [0055]).
With regards to claim 5, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 1 above. Redfern further teaches
wherein each VV component is configured to generate, [based on a coordination mode being a first coordination mode], a corresponding VV output as a rounded sum of multiple accumulations of products output from the plurality of MAC components included in that VV component, (Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix; Redfern [0300]-[0303]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Output rounding enabled for CPM output formatting)
and wherein each VV component is configured to generate, [based on the coordination mode being a second coordination mode], a corresponding VV output as a rounded accumulation of products output [by a single MAC component of the plurality of MAC components included in that VV component] (Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix; Redfern [0300]-[0303]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Output rounding enabled for CPM output formatting).
Redfern fails to teach [wherein each VV component is configured to generate,] based on a coordination mode being a first coordination mode, [a corresponding VV output as a rounded sum of multiple accumulations of products output from the plurality of MAC components included in that VV component,], and [wherein each VV component is configured to generate], based on the coordination mode being a second coordination mode, [a corresponding VV output as a rounded accumulation of products] output by a single MAC component of the plurality of MAC components included in that VV component.
However, Culurciello teaches [wherein each VV component is configured to generate,] based on a coordination mode being a first coordination mode, [a corresponding VV output as a rounded sum of multiple accumulations of products output from the plurality of MAC components included in that VV component,] (Culurciello [0045]: The gather adder 258 adds a single bias term B that is stored in the weights cache 136 as part of the total sum of all the MACs for at least one vector of map input data in the cooperative mode to produce a single output value M1 for a set of outputs from all of the MACs in a single vMAC)
and [wherein each VV component is configured to generate], based on the coordination mode being a second coordination mode, [a corresponding VV output as a rounded accumulation of products] output by a single MAC component of the plurality of MAC components included in that VV component (Culurciello [0045]: By contrast, in the independent operation mode that that is described in more detail below, the gather adder 258 receives the output of a single MAC from the shift register 252 during each cycle of operation and adds the bias term B to generate a single output. In the independent operating mode, the gather adder 258 generates 16 individual outputs).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern in view of Culurciello further in view of Szegedy with the coordination modes, and outputting a single MAC component output as taught by Culurciello. One of ordinary skill in the art would be motivated to make this combination for at least the same reasons as stated above. Also, the accelerators 100 and 200 can operate in different modes that utilize different forms of parallelism to process CNNs with complex structures more efficiently as taught by Culurciello (Culurciello [0055]).
With regards to claim 6, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 5 above. Redfern further teaches wherein each VV component, [of the plurality of VV components], includes an adder component configured to generate an adder component output based on the input precision mode, (Redfern [0262]: The packed SIMD results from the leaf array multiplier modules are combined in two levels of 4:2 SIMD carry save adders to contribute to the dot product summation; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity))
wherein the adder component is configured to generate, [based on the coordination mode being the first coordination mode], the adder component output based on summing the multiple accumulations of products, (Redfern [0262]: The packed SIMD results from the leaf array multiplier modules are combined in two levels of 4:2 SIMD carry save adders to contribute to the dot product summation; Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix)
and wherein the adder component is configured to generate, [based on the coordination mode being the second coordination mode], the adder component output [based on a single accumulation of products output by the single MAC component and without summing the multiple accumulations of products] (Redfern [0262]: The packed SIMD results from the leaf array multiplier modules are combined in two levels of 4:2 SIMD carry save adders to contribute to the dot product summation).
Redfern fails to teach of the plurality of VV components, [wherein the adder component is configured to generate,] based on the coordination mode being the first coordination mode, and [and wherein the adder component is configured to generate,] based on the coordination mode being the second coordination mode, [the adder component output] based on a single accumulation of products output by the single MAC component and without summing the multiple accumulations of products.
However, Culurciello teaches of the plurality of VV components (Culurciello [0010]: In one embodiment, an accelerator for processing of a convolutional neural network (CNN) has been developed. The accelerator includes a compute core. The compute core includes a plurality of compute units)
[wherein the adder component is configured to generate,] based on the coordination mode being the first coordination mode (Culurciello [0045]: The gather adder 258 adds a single bias term B that is stored in the weights cache 136 as part of the total sum of all the MACs for at least one vector of map input data in the cooperative mode to produce a single output value M1 for a set of outputs from all of the MACs in a single vMAC)
[and wherein the adder component is configured to generate,] based on the coordination mode being the second coordination mode, [the adder component output] based on a single accumulation of products output by the single MAC component and without summing the multiple accumulations of products (Culurciello [0045]: By contrast, in the independent operation mode that that is described in more detail below, the gather adder 258 receives the output of a single MAC from the shift register 252 during each cycle of operation and adds the bias term B to generate a single output. In the independent operating mode, the gather adder 258 generates 16 individual outputs).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern in view of Culurciello further in view of Szegedy with the plurality of components, the coordination modes, and the single accumulation of products output by the single MAC component as taught by Culurciello. One of ordinary skill in the art would be motivated to make this combination for at least the same reasons as stated above. Also, the accelerators 100 and 200 can operate in different modes that utilize different forms of parallelism to process CNNs with complex structures more efficiently as taught by Culurciello (Culurciello [0055]).
With regards to claim 7, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 6 above. Redfern further teaches wherein each VV component, [of the plurality of VV components,] includes a rounding component configured to round the adder component output based on the output precision mode, (Redfern [0300]-[0303]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity)... Output rounding enabled for CPM output formatting; Redfern [0416]: the ODF includes sequential data flow through a right shifter, integer rounder)
wherein the rounding component is configured to generate the rounded sum of the multiple accumulations of products [based on the coordination mode being the first coordination mode,] (Redfern [0262]: The packed SIMD results from the leaf array multiplier modules are combined in two levels of 4:2 SIMD carry save adders to contribute to the dot product summation; Redfern [0300]-[0303]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Output rounding enabled for CPM output formatting; Redfern [0416]: the ODF includes sequential data flow through a right shifter, integer rounder)
and wherein the rounding component is configured to generate the rounded [accumulation of products based on the coordination mode being the second coordination mode] (Redfern [0262]: The packed SIMD results from the leaf array multiplier modules are combined in two levels of 4:2 SIMD carry save adders to contribute to the dot product summation; Redfern [0300]-[0303]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Output rounding enabled for CPM output formatting; Redfern [0416]: the ODF includes sequential data flow through a right shifter, integer rounder).
Redfern fails to teach of the plurality of VV components, [wherein the rounding component is configured to generate the rounded sum of the multiple accumulations of products] based on the coordination mode being the first coordination mode, and [and wherein the rounding component is configured to generate the rounded] accumulation of products based on the coordination mode being the second coordination mode.
However, Culurciello teaches of the plurality of VV components (Culurciello [0010]: In one embodiment, an accelerator for processing of a convolutional neural network (CNN) has been developed. The accelerator includes a compute core. The compute core includes a plurality of compute units)
[wherein the rounding component is configured to generate the rounded sum of the multiple accumulations of products] based on the coordination mode being the first coordination mode, (Culurciello [0045]: The gather adder 258 adds a single bias term B that is stored in the weights cache 136 as part of the total sum of all the MACs for at least one vector of map input data in the cooperative mode to produce a single output value M1 for a set of outputs from all of the MACs in a single vMAC)
[and wherein the rounding component is configured to generate the rounded] accumulation of products based on the coordination mode being the second coordination mode (Culurciello [0045]: By contrast, in the independent operation mode that that is described in more detail below, the gather adder 258 receives the output of a single MAC from the shift register 252 during each cycle of operation and adds the bias term B to generate a single output. In the independent operating mode, the gather adder 258 generates 16 individual outputs).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern in view of Culurciello further in view of Szegedy with the plurality of components and the coordination modes as taught by Culurciello. One of ordinary skill in the art would be motivated to make this combination for at least the same reasons as stated above. Also, the accelerators 100 and 200 can operate in different modes that utilize different forms of parallelism to process CNNs with complex structures more efficiently as taught by Culurciello (Culurciello [0055]).
With regards to claim 8, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 1 above. Redfern further teaches wherein the input precision mode is one of a 16-bit signed integer mode or an 8-bit signed integer mode (Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity) (4, 8, 16, 32, 64, 128, 256, etc.); Redfern [0084]-[0090]: Within this context the following operand data types are anticipated to be supported by the RMM... 8-bit signed integer... 16-bit signed integer).
With regards to claim 9, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 1 above. Redfern further teaches wherein the output precision mode is one of a 16-bit signed integer mode or an 8-bit signed integer mode (Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity) (4, 8, 16, 32, 64, 128, 256, etc.); Redfern [0106]-[0112]: While output data of the RMM may incorporate a variety of data types, the following output data types are anticipated to be supported by the RMM... 8-bit signed integer... 16-bit signed integer).
With regards to claim 10, Redfern teaches A method, comprising: receiving [map] data via a first port; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses; Redfern Fig. 1: shows a SRC1 port)
receiving [kernel] data via a second port; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses; Redfern Fig. 1: shows a SRC2 port)
receiving, via a third port, an indication of an input precision mode that indicates an input word length for the [map] data and for the [kernel] data; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity); Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
receiving, via a fourth port, an indication of an output precision mode that indicates an output word length; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity); Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
generating, using [a plurality] of vector-vector (VV) components, a corresponding plurality of VV outputs based on the [map] data, the [kernel] data, the input precision mode, the output precision mode, and at least one accumulation of products; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity)... Right shift count for CPM output formatting)
wherein each VV component of the plurality of VV components comprises: a plurality of multiply-accumulate (MAC) components, (Redfern [0257]: The exemplary top level floorplan depicted in FIG. 40 (4000) is composed of an array of hierarchical multiply-accumulate modules (4011, 4012, 4013, 4014, 4021, 4022, 4023, 4024, 4031, 4032, 4033, 4034, 4041, 4042, 4043, 4044) a data path for operand pre-processing, and an interface/control region (4001))
each MAC component of the plurality of MAC components coupled with the third port, (Redfern [0257]: The exemplary top level floorplan depicted in FIG. 40 (4000) is composed of an array of hierarchical multiply-accumulate modules (4011, 4012, 4013, 4014, 4021, 4022, 4023, 4024, 4031, 4032, 4033, 4034, 4041, 4042, 4043, 4044) a data path for operand pre-processing, and an interface/control region (4001)Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
wherein each accumulation of products, of the at least one accumulation of products, is calculated by a respective MAC component, of the plurality of MAC components by adding a plurality of products based on the input precision mode, (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix)
wherein each product, of the plurality of products, is calculated by a respective MAC component, of the plurality of MAC components, by multiplying a respective [map] word of the [map data] and a respective [kernel] word of the [kernel data] based on the input precision mode; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix)
and outputting the [concatenated] VV output via a fifth port (Redfern [0251]: As shown in FIG. 38 (3800), dataflow in the RMM is fixed and proceeds from the operand inputs to operand storage. The matrix multiplication operands are next read and multiplied and added with an optional earlier value from the CPM matrix storage. The CPM matrix results are passed through an output processing block to an output buffer which is accessed by the ACL/CPU).
Redfern fails to teach a plurality [of vector-vector (VV) components] and that the inputs are maps and kernels.
However, Culurciello teaches a plurality [of vector-vector (VV) components] (Culurciello [0010]: In one embodiment, an accelerator for processing of a convolutional neural network (CNN) has been developed. The accelerator includes a compute core. The compute core includes a plurality of compute units)
And that the inputs of Redfern are maps and kernels (Culurciello [0036]: In the accelerators 100 and 200, each compute core 120 implements the processing of the CNNs including processing of traces of input map data and traces of the weight values for the kernels that are used in different layers of the CNN).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern with the plurality of components and that the inputs are maps and kernels as taught by Culurciello. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as data could be processed in parallel. Also, the accelerators 100 and 200 can operate in different modes that utilize different forms of parallelism to process CNNs with complex structures more efficiently as taught by Culurciello (Culurciello [0055]).
Redfern in view of Culurciello fails to teach concatenating, using one or more integrated circuits, the corresponding plurality of VV outputs to generate a concatenated VV output; and outputting the concatenated VV output.
However, Szegedy teaches concatenating, using one or more integrated circuits, the corresponding plurality of VV outputs to generate a concatenated VV output; (Szegedy Page 4 Section 4: It also means that the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector)
and outputting the concatenated VV output of Redfern in view of Culurciello (Szegedy Page 4 Section 4: It also means that the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern in view of Culurciello with the concatenated output as taught by Szegedy. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the different layers could be processed in parallel and then concatenated together. Also, one of the main beneficial aspects of this architecture is that it allows for increasing the number of units at each stage significantly without an uncontrolled blow-up in computational complexity as taught by Szegedy (Szegedy Page 5 Section 4).
With regards to claim 11, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 10 above. Redfern further teaches and wherein generating a VV output, of the corresponding plurality of VV outputs, comprises: adding the plurality of products, based on the input precision mode, to generate the [single] accumulation of products (Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity)).
Redfern fails to teach wherein the at least one accumulation of products is a single accumulation of products and [and adding the plurality of products, based on the input precision mode, to generate the] single [accumulation of products].
However, Culurciello teaches wherein the at least one accumulation of products is a single accumulation of products; (Culurciello [0045]: By contrast, in the independent operation mode that that is described in more detail below, the gather adder 258 receives the output of a single MAC from the shift register 252 during each cycle of operation and adds the bias term B to generate a single output. In the independent operating mode, the gather adder 258 generates 16 individual outputs)
[and adding the plurality of products, based on the input precision mode, to generate the] single [accumulation of products] (Culurciello [0045]: By contrast, in the independent operation mode that that is described in more detail below, the gather adder 258 receives the output of a single MAC from the shift register 252 during each cycle of operation and adds the bias term B to generate a single output. In the independent operating mode, the gather adder 258 generates 16 individual outputs).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern in view of Culurciello further in view of Szegedy with the single accumulation of products as taught by Culurciello. One of ordinary skill in the art would be motivated to make this combination for at least the same reasons as stated above. Also, the accelerators 100 and 200 can operate in different modes that utilize different forms of parallelism to process CNNs with complex structures more efficiently as taught by Culurciello (Culurciello [0055]).
With regards to claim 12, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 11 above. Redfern further teaches wherein generating the VV output comprises rounding the single accumulation of products or rounding a biased value generated based on the single accumulation of products (Redfern [0300]-[0303]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Output rounding enabled for CPM output formatting; Redfern [0416]: the ODF includes sequential data flow through a right shifter, integer rounder).
With regards to claim 13, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 10 above. Redfern further teaches wherein the at least one accumulation of products is multiple accumulations of products; (Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix)
and wherein generating a VV output, of the corresponding plurality of VV outputs, comprises: adding the plurality of products, based on the input precision mode, to generate an accumulation of products; (Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity))
and adding the accumulation of products and one or more other accumulations of products to generate the multiple accumulations of products (Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity)).
With regards to claim 14, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 13 above. Redfern further teaches wherein generating the VV output comprises summing the multiple accumulations of products to generate a sum of the multiple accumulations of products (Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity)).
With regards to claim 15, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 14 above. Redfern further teaches wherein generating the VV output comprises rounding the sum of the multiple accumulations of products (Redfern [0300]-[0303]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Output rounding enabled for CPM output formatting; Redfern [0416]: the ODF includes sequential data flow through a right shifter, integer rounder).
With regards to claim 16, Redfern teaches An apparatus, comprising: means for receiving [map] data; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses; Redfern Fig. 1: shows a SRC1 port)
means for receiving [kernel] data; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses; Redfern Fig. 1: shows a SRC2 port)
means for receiving an indication of an input precision mode that indicates an input word length for the [map] data and for the [kernel] data; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity); Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
means for receiving an indication of an output precision mode that indicates an output word length; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity); Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
means for calculating at least one accumulation of products using a [plurality] of vector-vector (V) components, (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0302]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity)... Right shift count for CPM output formatting)
wherein each VV component of the [plurality] of VV components comprises: an input precision mode port; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
and a plurality of multiply-accumulate (MAC) components, (Redfern [0257]: The exemplary top level floorplan depicted in FIG. 40 (4000) is composed of an array of hierarchical multiply-accumulate modules (4011, 4012, 4013, 4014, 4021, 4022, 4023, 4024, 4031, 4032, 4033, 4034, 4041, 4042, 4043, 4044) a data path for operand pre-processing, and an interface/control region (4001))
each MAC component of the plurality of MAC components coupled with the input precision mode port, (Redfern [0257]: The exemplary top level floorplan depicted in FIG. 40 (4000) is composed of an array of hierarchical multiply-accumulate modules (4011, 4012, 4013, 4014, 4021, 4022, 4023, 4024, 4031, 4032, 4033, 4034, 4041, 4042, 4043, 4044) a data path for operand pre-processing, and an interface/control region (4001)Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern Fig. 1: shows a control port for a control bus; Redfern Fig. 55: shows the control bus with many ports for the different signals)
wherein each accumulation of products, of the at least one accumulation of products, is calculated by a respective MAC component, of the plurality of MAC components by adding a plurality of products based on the input precision mode, (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix)
wherein each product, of the plurality of products, is calculated by a respective MAC component, of the plurality of MAC components, by multiplying a respective [map] word of the [map data] and a respective [kernel] word of the [kernel data] based on the input precision mode; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity; Redfern [0140]: A general depiction of this multiplication/accumulation process is depicted in FIG. 9 (0900), wherein rows (0911, 0912, 0913) of the AMM matrix/vector (0910) are multiplied by columns (0921, 0922, 0923) of the BMM matrix (0920) and summed to individual elements of the CPM matrix)
and means for outputting the [concatenated] output. (Redfern [0251]: As shown in FIG. 38 (3800), dataflow in the RMM is fixed and proceeds from the operand inputs to operand storage. The matrix multiplication operands are next read and multiplied and added with an optional earlier value from the CPM matrix storage. The CPM matrix results are passed through an output processing block to an output buffer which is accessed by the ACL/CPU).
Redfern fails to teach a plurality [of vector-vector (VV) components] and that the inputs are maps and kernels.
However, Culurciello teaches a plurality [of vector-vector (VV) components] (Culurciello [0010]: In one embodiment, an accelerator for processing of a convolutional neural network (CNN) has been developed. The accelerator includes a compute core. The compute core includes a plurality of compute units)
and that the inputs of Redfern are maps and kernels (Culurciello [0036]: In the accelerators 100 and 200, each compute core 120 implements the processing of the CNNs including processing of traces of input map data and traces of the weight values for the kernels that are used in different layers of the CNN).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern with the plurality of vector-vector (VV) components and that the inputs are maps and kernels as taught by Culurciello. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as data could be processed in parallel. Also, the accelerators 100 and 200 can operate in different modes that utilize different forms of parallelism to process CNNs with complex structures more efficiently as taught by Culurciello (Culurciello [0055]).
Redfern in view of Culurciello fails to teach means for concatenating the output with one or more other outputs to generate a concatenated output and outputting the concatenated output.
However, Szegedy teaches means for concatenating the output with one or more other outputs to generate a concatenated output; (Szegedy Page 4 Section 4: It also means that the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector)
and outputting the concatenated output of Redfern in view of Culurciello (Szegedy Page 4 Section 4: It also means that the suggested architecture is a combination of all those layers with their output filter banks concatenated into a single output vector).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern in view of Culurciello with the concatenated output as taught by Szegedy. One of ordinary skill in the art would be motivated to make this combination because it would increase the efficiency of the system as the different layers could be processed in parallel and then concatenated together. Also, one of the main beneficial aspects of this architecture is that it allows for increasing the number of units at each stage significantly without an uncontrolled blow-up in computational complexity as taught by Szegedy (Szegedy Page 5 Section 4).
With regards to claim 17, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 16 above. Redfern further teaches wherein the means for calculating the at least one accumulation of products comprises: means for adding the plurality of products, based on the input precision mode, to generate the at least one accumulation of products [as a single accumulation of products] (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity)).
Redfern fails to teach [and means for adding the plurality of products, based on the input precision mode, to generate the at least one accumulation of products] as a single accumulation of products.
However, Culurciello teaches [and means for adding the plurality of products, based on the input precision mode, to generate the at least one accumulation of products] as a single accumulation of products (Culurciello [0045]: By contrast, in the independent operation mode that that is described in more detail below, the gather adder 258 receives the output of a single MAC from the shift register 252 during each cycle of operation and adds the bias term B to generate a single output. In the independent operating mode, the gather adder 258 generates 16 individual outputs).
Therefore, it would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings Redfern in view of Culurciello further in view of Szegedy with the single accumulation as taught by Culurciello. One of ordinary skill in the art would be motivated to make this combination for at least the same reasons as above. Also, the accelerators 100 and 200 can operate in different modes that utilize different forms of parallelism to process CNNs with complex structures more efficiently as taught by Culurciello (Culurciello [0055]).
With regards to claim 18, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 17 above. Redfern further teaches wherein the means for generating the output comprises means for rounding the single accumulation of products or a biased value generated based on the single accumulation of products (Redfern [0300]-[0303]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Output rounding enabled for CPM output formatting; Redfern [0416]: the ODF includes sequential data flow through a right shifter, integer rounder).
With regards to claim 19, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 16 above. Redfern further teaches wherein the means for calculating the at least one accumulation of products comprises: means for adding the plurality of products, based on the input precision mode, to generate an accumulation of products; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity))
and means for adding the accumulation of products and one or more other accumulations of products to generate the at least one accumulation of products as multiple accumulations of products (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity); Redfern [0262]: The packed SIMD results from the leaf array multiplier modules are combined in two levels of 4:2 SIMD carry save adders to contribute to the dot product summation).
With regards to claim 20, Redfern in view of Culurciello further in view of Szegedy teaches all of the limitations of claim 19 above. Redfern further teaches wherein the means for generating the output comprises: means for summing the multiple accumulations of products to generate a sum of the multiple accumulations of products; (Redfern [0084]: The present disclosure anticipates that the multiplication/accumulation operations executed by the RMM may operate on a variety of data types as present on the various external and internal data busses and that the mode/control and/or function/opcode information provided to the RMM may control the interpretation of data retrieved from the EMB and operated on by the RMM. Within this context the following operand data types are anticipated to be supported by the RMM; Redfern [0300]-[0301]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure... Scalar bit width and signed/unsigned mode (number of bits in each individual AMM/BMM/CPM vector/matrix entity); Redfern [0262]: The packed SIMD results from the leaf array multiplier modules are combined in two levels of 4:2 SIMD carry save adders to contribute to the dot product summation)
and means for rounding the sum of the multiple accumulations of products (Redfern [0300]-[0303]: While the mode/control interface configuration between the ACL/CPU and RMM may take many forms, the following list of mode/control functions is anticipated within the scope of the present disclosure… Output rounding enabled for CPM output formatting; Redfern [0416]: the ODF includes sequential data flow through a right shifter, integer rounder).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jakob O Gudas whose telephone number is (571)272-0695. The examiner can normally be reached Monday-Thursday: 7:30AM-5:00PM Friday: 7:30AM-4:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James Trujillo can be reached at (571) 272-3677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.O.G./Examiner, Art Unit 2151
/NICHOLAS KLICOS/Primary Examiner, Art Unit 2118
Prosecution Insights