Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 1-20 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea (mental process) without significantly more.
Regarding claim 1, in Step 1 of the 101 analysis set forth in MPEP 2106, the claim recites a processing unit that can perform hybrid and mixed precision computations. A processing unit is considered a machine and is one of the four statutory categories of invention.
In Step 2a Pong 1 of the 101 analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a machine that, under the broadest reasonable interpretation, covers a mental process but for recitation of generic computer components:
“perform tensor operations of a given layer of a neural network in one of a first number representation and a second number representation” (a person can mentally perform a tensor operation by simply evaluating the tensors and making judgments as tensor operations (MPEP 2106)).
“convert between the first number representation and the second representation” (A person can mentally convert between two number representations by evaluating the relationship between the two number representation, and making a judgement of the conversion (MPEP 2106)).
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In Step 2a Prong 2 of the 101 analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
iii. “A neural network processing unit, comprising: an operation circuit” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
iv. “a conversion circuit coupled to at least one of an input port and an output port of the operation circuit” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
v. “wherein the first number representation is one of a fixed-point number representation and a floating-point number representation, and the second number representation is the other one of the fixed-point number representation and the floating-point number representation.” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
Since the claim does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In Step 2b of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements (iii), (iv), and (v) recite generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 2, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 2 recites “the conversion circuit, according to operating parameters for the given layer of the neural network, is configurable to be coupled to one or both of the input port and the output port of the operation circuit.” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 3, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 3 recites “the conversion circuit, according to operating parameters for the given layer of the neural network, is configurable to be enabled or bypassed for one or both input conversion and output conversion” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 4, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Additionally, new limitations that fall within Step 2A Prong 1 of the 101 analysis recite, under the broadest reasonably interpretation, the following abstract idea:
“operative to perform hybrid-precision computing on a first input operand and a second input operand of the given layer” (This is a mental process because a person can mentally evaluate the input, and make a judgement of the computation (MPEP 2106)).
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In Step 2a Prong 2 of the 101 analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
ii. “the neural network processing unit…the first input operand and the second input operand having different number representations” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
Since the claim does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In Step 2b of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements (ii) recite generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 5, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Additionally, new limitations that fall within Step 2A Prong 1 of the 101 analysis recite, under the broadest reasonably interpretation, the following abstract idea:
“operative to perform mixed-precision computing” (This is a mental process because a person can mentally evaluate the data, and make a judgement of the computation result (MPEP 2106)).
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In Step 2a Prong 2 of the 101 analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
ii. “the neural network processing unit…which computation in a first layer of the neural network is performed in the first number representation and computation in a second layer of the neural network is performed the second number representation.” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
Since the claim does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In Step 2b of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements (ii) recite generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 6, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 6 recites “the neural network processing unit is time-shared among multiple layers of the neural network by operating on one layer at a time.” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 7, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Additionally, new limitations that fall within Step 2A Prong 1 of the 101 analysis recite, under the broadest reasonably interpretation, the following abstract idea:
“determine…a scaling factor for conversion between the first number representation and the second number representation” (This is a mental process because a person can mentally evaluate the data, and make a judgement of the scaling factor (MPEP 2106)).
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In Step 2a Prong 2 of the 101 analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
“a buffer memory…for the convert circuit…during operations of the given layer of the neural operations of the given layer of the neural network.” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
“buffer non-converted input” (Adding insignificant extra-solution activity (mere data storage) to the judicial exception (MPEP 2106.05(g)).
Since the claim does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In Step 2b of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements (ii) recite generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of significantly more. Additional elements (iii) recite adding insignificant extra solution activity to the judicial exception, which is also not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 8, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 8 recites “a buffer coupled between the converter circuit and the operation circuit” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 9, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Additionally, new limitations that fall within Step 2A Prong 1 of the 101 analysis recite, under the broadest reasonably interpretation, the following abstract idea:
“compute a layer of the neural network in fixed-point” (This is a mental process because a person can mentally evaluate some data, and make a judgement of operation result (MPEP 2106)).
compute a layer of the neural network in floating-point” (This is a mental process because a person can mentally evaluate some data, and make a judgement of operation result (MPEP 2106)).
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In Step 2a Prong 2 of the 101 analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
“the operation circuit includes a fixed-point circuit to…and a fixed-point circuit” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
Since the claim does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In Step 2b of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements (iii) recite generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 10, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 10 recites “the neural network processing unit is coupled to one or more processors that are operative to perform operations of one or more layers of the neural network in the first number representation.” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 11, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 11 recites “a plurality of operation circuits including one or more fixed-point circuits and floating-point circuits, different ones of the operation circuits operative to compute different layers of the neural network; and one or more of the conversion circuits coupled to the operation circuits.” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 12, it is dependent upon claim 1, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 12 recites “the operation circuit further comprises one or more of: an adder, a subtractor, a multiplier, a function evaluator, and a multiply-and-accumulate (MAC) circuit.” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 13, in Step 1 of the 101 analysis set forth in MPEP 2106, the claim recites a processing unit comprising different circuits to perform hybrid and mixed precision computations in different layers of a neural network. A processing unit is considered a machine and is one of the four statutory categories of invention.
In Step 2a Pong 1 of the 101 analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a machine that, under the broadest reasonable interpretation, covers a mental process but for recitation of generic computer components:
“select to enable or bypass the conversion circuit for input conversion of an input operand according to operating parameters for a given layer of the neural network” (a person can mentally perform a selection by simply evaluating the parameters for a layer of a neural network and making a judgment of choosing to enable or bypass (MPEP 2106)).
“perform tensor operations on the input operand in the second number representation to generate an output operand in the second number representation;” (A person can mentally perform tensor operations by evaluating input data, and making a judgement of operations (MPEP 2106)).
“select to enable or bypass the conversion circuit for output conversion of an output operand according to the operating parameters” (a person can mentally perform a selection by simply evaluating the parameters for a layer of a neural network and making a judgment of choosing to enable or bypass (MPEP 2106)).
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In Step 2a Prong 2 of the 101 analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
“A neural network processing unit comprising: an operation circuit; and a conversion circuit” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
“wherein the input conversion, when enabled, converts from a first number representation to a second number representation” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
“the output conversion, when enabled, converts from the second number representation to the first number representation” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
“the first number representation is one of a fixed-point number representation and a floating-point number representation, and the second number representation is the other one of the fixed-point number representation and the floating-point number representation” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
Since the claim does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In Step 2b of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements (iv), (v), (vi), and (vii) recite generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 14, it is dependent upon claim 13, and thereby incorporates the limitations of, and corresponding analysis applied to claim 13. Additionally, new limitations that fall within Step 2A Prong 1 of the 101 analysis recite, under the broadest reasonably interpretation, the following abstract idea:
“perform, for another given layer of the neural network, additional tensor operations on another input operand in the first number representation to generate another output operand in the first number representation” (This is a mental process because a person can mentally perform an additional tensor operation in a different layer of a neural network by evaluating the tensor data, and making a judgement of the operation result (MPEP 2106)).
Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible.
Regarding claim 15, it is dependent upon claim 13, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 15 recites “the neural network processing unit is time-shared among multiple layers of the neural network by operating on one layer at a time” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 16, in Step 1 of the 101 analysis set forth in MPEP 2106, the claim recites a system consists of one or more floating-point and fixed-point circuits that perform tensor operation in mixed precision in different layers of a neural network. A system is considered a machine and is one of the four statutory categories of invention.
In Step 2a Pong 1 of the 101 analysis set forth in the MPEP 2106, the examiner has determined that the following limitations recite a machine that, under the broadest reasonable interpretation, covers a mental process but for recitation of generic computer components:
“perform floating-point tensor operations for one or more layers of the neural network” (a person can mentally perform a floating-point tensor operation by simply evaluating some data and making a judgment of the operation result (MPEP 2106)).
“perform fixed-point tensor operations for other one or more layers of the neural network” (a person can mentally perform a fixed-point tensor operation by simply evaluating some data and making a judgment of the operation result (MPEP 2106)).
“convert between a floating-point number representation and a fixed-point number representation.” (a person can mentally perform conversion between two number representation by simply evaluating the data and making a judgment of the conversion result (MPEP 2106)).
If claim limitations, under their broadest reasonable interpretation, covers performance of the limitations as a mental process but for the recitation of generic computer components, then it falls within the mental process grouping of abstract ideas. Accordingly, the claim “recites” an abstract idea.
In Step 2a Prong 2 of the 101 analysis set forth in MPEP 2106, the examiner has determined that the following additional elements do not integrate this judicial exception into a practical application:
“A system comprising: one or more floating-point circuits…one or more fixed-point circuits…one or more conversion circuits coupled to at least one of the floating-point circuits and the fixed-point circuits” (Generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h))).
Since the claim does not contain any other additional elements that are indicative of integration into a practical application, the claim is “directed” to an abstract idea.
In Step 2b of the 101 analysis set forth in the 2019 PEG, the examiner has determined that the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
As discussed above, additional elements (iv) recite generally linking the use of the judicial exception to a particular technological environment or field of use, which is not indicative of significantly more. Considering the additional elements individually and in combination, and the claim as a whole, the additional elements do not provide significantly more than the abstract idea. Therefore, the claim is not patent eligible.
Regarding claim 17, it is dependent upon claim 16, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 17 recites “the one or more floating-point circuits and the one or more fixed-point circuits are coupled to one another in a series according to a predetermined order” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 18, it is dependent upon claim 16, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 18 recites “output ports of one of the floating-point circuits and one of the fixed-point circuits are coupled, in parallel, to a multiplexer” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 19, it is dependent upon claim 16, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 19 recites “the one or more conversion circuits includes a floating-point to fixed-point converter that is coupled to an input port of a fixed-point circuit or an output port of a floating-point circuit.” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Regarding claim 20, it is dependent upon claim 16, and thereby incorporates the limitations of, and corresponding analysis applied to claim 1. Further, claim 20 recites “the one or more conversion circuits includes a fixed-point to floating-point converter that is coupled to an input port of a floating-point circuit or an output port of a fixed-point circuit.” ((in step 2A prong 2, and step 2B this recites generally linking the use of the judicial exception to a particular technological environment or field of use (MPEP 2106.05(h)). Since the claim does not recite additional elements that either integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception, the claim is not patent eligible).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7-8, 10, and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. Pub. No: US2020/0160222 A1(hereafter ZHANG) and in view of Meng et al. Pub. No: US2020/0242468 A1(hereafter MENG).
Regarding claim 1, ZHANG teaches the invention substantially as claimed, including:
A neural network processing unit, comprising: an operation circuit([Abstract]The computation device is configured to perform a machine learning computation, and includes an operation unit, a controller unit, and a conversion unit). to perform tensor operations ([0113] and the data may be n-dimensional data, where n is an integer greater than or equal to one. For example, if n=1, the data is a one-dimensional data (that is, a vector). For another example, if n=2, the data is a two-dimensional data (that is, a matrix). If n=3 or more, the data is a multi-dimensional tensor.) of a given layer of a neural network ([0056] FIG. 4 is a flow chart of a forward operation of a single-layer artificial neural network according to an example of the present disclosure) in one of a first number representation and a second number representation ([0008] The conversion unit may be configured to convert the first input data into second input data according to the opcode and the opcode field of the data conversion instruction. [0009] The operation unit may be configured to perform operations on the second input data according to the plurality of operation instructions to obtain a computation result of the computation instruction). …to convert between the first number representation and the second number representation, wherein the first number representation is one of a fixed-point number representation and a floating-point number representation, and the second number representation is the other one of the fixed-point number representation and the floating-point number representation ([0131] The conversion unit 13 may be configured to convert the first input data into the second input data according to the decimal point position and the identifier of data type conversion. [0132] Specifically, identifiers of data type conversion are in one-to-one correspondence with conversion manners of data type. Table 3 is a table illustrating correspondence relation between the identifier of data type conversion and the conversion manner of data type. [0133] As illustrated in Table 3, if the identifier of data type conversion is 00, the conversion manner of data type is converting the fixed-point data into fixed-point data. If the identifier of data type conversion is 01, the conversion manner of data type is converting the floating point data into floating point data. If the identifier of data type conversion is 10, the conversion manner of data type is converting the fixed-point data into floating point data. If the identifier of data type conversion is 11, the conversion manner of data type is converting the floating point data into fixed-point data).
While ZHANG teaches a processing unit that has an operation circuit and a conversion unit that perform tensor operations in one or more layers of a neural network in fixed-point and floating-point number representation, ZHANG does not explicitly disclose that
and a conversion circuit coupled to at least one of an input port and an output port of the operation circuit
However, in analogous art, MENG teaches:
and a conversion circuit coupled to at least one of an input port and an output port of the operation circuit ([0051] The controller unit 11 is connected to the operation unit 12 and the conversion unit 13 (the conversion unit may be set separately, or may be integrated in the controller unit or the operation unit. Figure 1 below shows the conversion unit next to the output of the operation unit, and the controller unit connected to the input of the operation unit).
PNG
media_image1.png
644
820
media_image1.png
Greyscale
It would have been obvious to a person having ordinary skill in the art before the filing date of the invention to have combined MENG’s teaching of a conversion unit coupled at the output and input port of the operation unit, with ZHANG’s teaching a processing unit that has an operation circuit perform tensor operations in one or more layers of a neural network in fixed-point and floating-point number representation, and a conversion unit that can convert between these two number representations, to realize, with a reasonable expectation of success, a system that has an operation unit that performs tensor operation in one or more layers of a neural network, and a conversion unit that converts between fixed-point and floating-point, as in ZHANG, and having the conversion unit coupled to the input and output port of the operation unit, as in MENG. A person of ordinary skill would have been motivated to make this combination to result in a processing unit that is flexible with data representation and is optimized for efficient data conversion and tensor operations at different processing stages.
Regarding claim 2, MENG further teaches:
wherein the conversion circuit, according to operating parameters for the given layer of the neural network, is configurable to be coupled to one or both of the input port and the output port of the operation circuit ([0051] The controller unit 11 is connected to the operation unit 12 and the conversion unit 13 (the conversion unit may be set separately, or may be integrated in the controller unit or the operation unit (i.e., the conversion unit can be coupled at the output port as shown in Figure 1 below, or integrated with the operation unit at the input port ).
PNG
media_image1.png
644
820
media_image1.png
Greyscale
Regarding claim 3, ZHANG further teaches:
wherein the conversion circuit, according to operating parameters for the given layer of the neural network, is configurable to be enabled or bypassed for one or both input conversion and output conversion ([0007] The controller unit may be configured to acquire first input data and a computation instruction, to parse the computation instruction to obtain at least one of a data conversion instruction and at least one operation instruction, where the data conversion instruction may include an opcode field and an opcode. The opcode may be configured to indicate information of a function of the data conversion instruction. The opcode field may include information of a decimal point position, a flag bit indicating a data type of the first input data, and an identifier of data type conversion. The controller unit may be further configured to transfer the opcode and the opcode field of the data conversion instruction and the first input data to the conversion unit, and to send a plurality of operation instructions to the operation unit. [0008] The conversion unit may be configured to convert the first input data into second input data according to the opcode and the opcode field of the data conversion instruction (i.e., according to the opcode and opcode field, the conversion could be bypassed or enabled)).
Regarding claim 4, ZHANG further teaches:
wherein the neural network processing unit is operative to perform hybrid-precision computing on a first input operand and a second input operand of the given layer, the first input operand and the second input operand having different number representations ([0093] The operation unit 12 may be configured to perform operations on the second input data according to the multiple operation instructions to obtain a computation result of the computation instruction. [0131] The conversion unit 13 may be configured to convert the first input data into the second input data according to the decimal point position and the identifier of data type conversion).
Regarding claim 5, MENG further teaches:
wherein the neural network processing unit is operative to perform mixed-precision computing in which computation in a first layer of the neural network is performed in the first number representation and computation in a second layer of the neural network is performed the second number representation ([0160] In an example, before the operation unit 12 of the computation device performs operations on data of an ith layer of a multi-layer neural network model, the controller unit 11 of the computation device acquires a configuration command, which may include a decimal point position and a data type of data involved in the operations (i.e., the data type could be floating-point or fixed-point depending on the command received). The controller unit 11 parses the configuration instruction to obtain the decimal point position and the data type of the data involved in the operations.
Regarding claim 7, ZHANG further teaches:
further comprising: a buffer memory to buffer non-converted input for the converter circuit to determine, during operations of the given layer of the neural network ([0170] In an example, the operation unit 12 may be provided with a separate cache. As illustrated in FIG. 3G, the operation unit 12 may include a neuron cache unit 63 configured to buffer input neuron vector data and output neuron weight data of the secondary processing circuits 102. [0171] As illustrated in FIG. 3H, the operation unit 12 may further include a weight cache unit 64 configured to buffer weight data required by the secondary processing circuit 102 in the operation process); a scaling factor for conversion between the first number representation and the second number representation ([0069] Examples of the present disclosure provide a data type. The data type may include an adjustment factor. The adjustment factor may be configured to indicate a value range and precision of the data type. [0070] The adjustment factor may include a first scaling factor. Optionally, the adjustment factor may further include a second scaling factor. The first scaling factor may be configured to indicate the precision of the data type, and the second scaling factor may be configured to adjust the value range of the data type. [0075] Scaling factors may be applied to any format of data (such as floating point data and discrete data), so as to adjust the size and precision of the data).
Regarding claim 8, ZHANG further teaches:
a buffer coupled between the converter circuit and the operation circuit (
PNG
media_image2.png
740
958
media_image2.png
Greyscale
As shown in the figure 3A above, the control unit has a Instruction Cache unit, and a storage queue unit, and the control unit is between the Conversion unit and the Operation unit. [0105] In an example, the controller unit 11 may include an instruction cache unit 110, an instruction processing unit 111, and a storage queue unit 113. [0106] The instruction cache unit 110 may be configured to store the computation instruction associated with artificial neural network operations. [0107] The instruction processing unit 111 may be configured to parse the computation instruction to obtain the data conversion instruction and the multiple operation instructions, and to parse the data conversion instruction to obtain the opcode and the opcode field of the data conversion instruction).
Regarding claim 10, Zhang further teaches that:
wherein the neural network processing unit is coupled to one or more processors that are operative to perform operations of one or more layers of the neural network in the first number representation ([0279] In an example, a main processor and a coprocessor are included in a system on chip (SOC), and the main processor may include the above-mentioned computation device. The coprocessor obtains the decimal point position of input data of the same type of the each layer in the multi-layer network model according to the above-identified method, and sends the decimal point position of the input data of the same type of the each layer in the multi-layer network model to the computation device. Alternatively, if the computation device needs to use the decimal point position of the input data of the same type of the each layer in the multi-layer network model, the decimal point position of the input data of the same type of the each layer in the multi-layer network model is obtained from the above-mentioned coprocessor).
Regarding claim 12, ZHANG further teaches that:
wherein the operation circuit further comprises one or more of: an adder, a subtractor, a multiplier, a function evaluator, and a multiply-and-accumulate (MAC) circuit ([0006] The operation unit may include a primary processing circuit and a plurality of secondary processing circuits. [0127] In an example, as illustrated in FIG. 3D, the primary processing circuit 101 illustrated in FIGS. 3A-3C may further include one or any combination of an activation processing circuit 1011 and an addition processing circuit 1012. [0129] The addition processing circuit 1012 may be configured to perform an addition operation or an accumulation operation. [0261] For example, the first input data may include the input data I1 and the input data I2, and the corresponding decimal point positions are P1 and P2, and P1>P2. If the operation type indicated by the above-mentioned operation instruction is an addition operation or a subtraction operation. [0293] Further, the operation instructions may be configured to perform neural network operations. The operation instruction may include a matrix operation instruction, a vector operation instruction, and a scalar operation instruction. [0294] Furthermore, the matrix operation instruction performs matrix operations in neural networks, which include operations of matrix multiplying vector, vector multiplying matrix, and matrix multiplying scalar, outer products, matrix adding matrix, matrix subtracting matrix. [0176] In an example, the operation unit 12 may include, but not limited to, one or more multipliers of a first part, one or more adders of a second part (more specifically, the adders of the second part may also constitute an addition tree), an activation function unit of a third part, and/or a vector processing unit of the fourth part.
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over ZHANG, in view of MENG, as applied to claim 1 above, and in further view of Dawwd, “Time Sharing Based Parallel Implementation of CNN on Low Cost FPGA” (hereafter DAWWD).
Regarding claim 6, while ZHANG teaches the neural network processing unit that can perform tensor operations with floating point and fixed point data representations, the combination of ZHANG and MENG does not explicitly disclose:
the neural network processing unit is time-shared among multiple layers of the neural network by operating on one layer at a time
However, in analogous art, DAWWD teaches:
the neural network processing unit is time-shared among multiple layers of the neural network by operating on one layer at a time (Figure 4 below shows the time-sharing operation. (Section IV, A. Time Sharing Processing Flow) Therefore, in the proposed architecture of CNN, a repetitively utilized neuron circuits are performed using time-sharing operation. Each group of receptive field vectors are processed in a parallel manner spending a time slice that is enough to complete their respective computations, then another group which may be located in successive layers (except the feedforward layer) takes its role, and then return to the fist layer and so on, until the all neurons’ outputs in the last simple layer are calculated. The CNN architecture shown in Fig. 2 can share the time according to the above approach. Fig. 4 shows the time sharing operation of this architecture).
PNG
media_image3.png
490
1000
media_image3.png
Greyscale
It would have been obvious to a person having ordinary skill in the art before the filing date of the invention to have combined DAWWD’s teaching of operations being time-shared among the layers of a neural network and operates on one layer at a time, with the combination of ZHANG and MENG’s teaching of a processing unit that has an operation circuit perform tensor operations in one or more layers of a neural network in fixed-point and floating-point number representation, and a conversion unit coupled to the input and output port of the operation circuit that can convert between these two number representations, to realize, with a reasonable expectation of success, a system that has an operation unit that performs tensor operation in one or more layers of a neural network, and a conversion unit that converts between fixed-point and floating-point, as in the combination of ZHANG and MENG, and operations are time-shared among the layers by operating on one layer at a time, as in DAWWD. A person of ordinary skill would have been motivated to make this combination to reduce the resources required for implementation (DAWWD[Abstract]), and enable more efficient tensor operation while minimizing hardware requirement.
Claim(s) 9 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over ZHANG, in view of MENG, as applied to claim 1 above, and in further view of Kawabe et al. Pub. Num: US2020/0192633 A1(hereafter KAWABE).
Regarding claim 9, while ZHANG teaches a processing unit that have multiple secondary operation circuits that can perform tensor operations in different layers of a neural network, and conversion units that convert between floating point and fixed point data, the combination of ZHANG and MENG does not explicitly disclose:
wherein the operation circuit includes a fixed-point circuit to compute a layer of the neural network in fixed-point and a floating-point circuit to compute another layer of the neural network in floating-point
However, in analogous art, KAWABE teaches:
a fixed-point circuit to compute a layer of the neural network in fixed-point and a floating-point circuit to compute another layer of the neural network in floating-point (KAWABE figure 1 shows a fixed-point operator 1 that computes in fixed point and a floating-point operator 2 that computes in floating point. [0025] The arithmetic processing device 100 includes a fixed-point operator 1 that executes an operation on a fixed-point number, a floating-point operator 2 that executes an operation on a floating-point number, a first converter 3, a selector 4, a statistical information acquirer 5, an update information generator 6, and a second converter 7. The fixed-point operator 1 includes, for example, a 16-bit multiply-accumulate operator and outputs an operation result DT1 (of, for example, 40 bits) to the selector 4. [0026] The floating-point operator 2 includes, for example, a 32-bit multiplier, a 32-bit divider, or the like and outputs a result DT2 (of, for example, 32 bits) of executing an operation on a floating-point number to the first converter 3. For example, the 32-bit floating-point number includes an 8-bit decimal part and may represent 256 decimal point positions. [0021] For example, a deep neural network includes a layer for executing an operation using a fixed-point number and a layer for executing an operation using a floating-point number. In the case where the layer for executing the operation using the fixed-point number is coupled to and succeeding the layer for executing the operation using the floating-point number, a result of executing the operation on the floating-point number is converted to a fixed-point number and the fixed-point number after the conversion is input to the succeeding layer).
PNG
media_image4.png
730
930
media_image4.png
Greyscale
It would have been obvious to a person having ordinary skill in the art before the filing date of the invention to have combined KAWABE’s teaching of a fixed-point operator that operates with fixed point data representation in one layer of a neural network and a floating-point operator that operates with floating point data representation in another layer of a neural network, with the combination of ZHANG and MENG’s teaching of a processing unit that has an operation unit and multiple secondary processing unit that perform tensor operations in one or more layers of a neural network in fixed-point and floating-point number representation, to realize, with a reasonable expectation of success, a system that has an operation unit that performs tensor operation in one or more layers of a neural network using its secondary processing units, as in the combination of ZHANG and MENG, and uses fixed-point operators to compute operations in fixed point data representation in one layer and floating-point operators to compute operations in fixed point data representation in different layer of the neural network, as in KAWABE. A person of ordinary skill would have been motivated to make this combination to result in an enhanced computation efficiency in different layers of a neural network when working with floating point and fixed point data representations, since some layer may benefit from the speed and power efficiency of fixed-point operations and some layer may benefits from the accuracy of floating-point operations.
Regarding claim 11, ZHANG further teaches:
and one or more of the conversion circuits coupled to the operation circuits ([0089] The controller unit 11 may be further configured to parse the computation instruction to obtain at least one of a data conversion instruction (i.e., one or more conversions) and at least one operation instruction, where the data conversion instruction may include an opcode field and an opcode. The opcode may be configured to indicate information of a function of the data conversion instruction. The opcode field may include information of a decimal point position, a flag bit indicating a data type of the first input data, and an identifier of data type conversion).
While the combination of ZHANG and MENG teaches a processing unit with multiple secondary processing unit to perform tensor operations in different data representations in different layers of a neural network, they do not explicitly disclose:
a plurality of operation circuits including one or more fixed-point circuits and floating- point circuits, different ones of the operation circuits operative to compute different layers of the neural network.
However, in analogous art, KAWABE teaches:
…fixed-point circuits and floating- point circuits, different ones of the operation circuits operative to compute different layers of the neural network ([0021] For example, a deep neural network includes a layer for executing an operation using a fixed-point number and a layer for executing an operation using a floating-point number. In the case where the layer for executing the operation using the fixed-point number is coupled to and succeeding the layer for executing the operation using the floating-point number, a result of executing the operation on the floating-point number is converted to a fixed-point number and the fixed-point number after the conversion is input to the succeeding layer [0025] The arithmetic processing device 100 includes a fixed-point operator 1 that executes an operation on a fixed-point number, a floating-point operator 2 that executes an operation on a floating-point number, a first converter 3, a selector 4, a statistical information acquirer 5, an update information generator 6, and a second converter 7. The fixed-point operator 1 includes, for example, a 16-bit multiply-accumulate operator and outputs an operation result DT1 (of, for example, 40 bits) to the selector 4. [0026] The floating-point operator 2 includes, for example, a 32-bit multiplier, a 32-bit divider, or the like and outputs a result DT2 (of, for example, 32 bits) of executing an operation on a floating-point number to the first converter 3. For example, the 32-bit floating-point number includes an 8-bit decimal part and may represent 256 decimal point positions).
It would have been obvious to a person having ordinary skill in the art before the filing date of the invention to have combined KAWABE’s teaching of a fixed-point operator that operates with fixed point data representation in one layer of a neural network and a floating-point operator that operates with floating point data representation in another layer of a neural network, with the combination of ZHANG and MENG’s teaching of a processing unit that has an operation unit and multiple secondary processing unit that perform tensor operations in one or more layers of a neural network in fixed-point and floating-point number representation, to realize, with a reasonable expectation of success, a system that has an operation unit with multiple secondary processing units that performs tensor operation in one or more layers of a neural network, as in the combination of ZHANG and MENG, and uses fixed-point circuits for operations in fixed point data representation in one layer and floating-point circuits for operations in fixed point data representation in different layer of the neural network, as in KAWABE. A person of ordinary skill would have been motivated to make this combination to result in an enhanced computation efficiency and flexibility when working with floating point and fixed point operations in different layers of a neural network, since some layer may benefit from the speed and power efficiency of fixed-point operations and some layer may benefits from the accuracy of floating-point operations.
Claim(s) 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over ZHANG, and in further view of MENG.
Regarding claim 13, ZHANG teaches the claimed invention substantially, including:
A neural network processing unit comprising: an operation circuit ([Abstract]The present disclosure provides a computation device. The computation device is configured to perform a machine learning computation, and includes an operation unit, a controller unit, and a conversion unit) and a conversion circuit, the neural network processing unit operative to: select to enable or bypass the conversion circuit for input conversion of an input operand according to operating parameters for a given layer of the neural network, wherein the input conversion, when enabled, converts from a first number representation to a second number representation ([0160] In an example, before the operation unit 12 of the computation device performs operations on data of an ith layer of a multi-layer neural network model, the controller unit 11 of the computation device acquires a configuration command, which may include a decimal point position and a data type of data involved in the operations. The controller unit 11 parses the configuration instruction to obtain the decimal point position and the data type of the data involved in the operations…. If the controller unit 11 has obtained the input data, it is determined whether the data type of the input data is consistent with that of the data involved in the operations. If it is determined that the data type of the input data is inconsistent with that of the data involved in the operations, the controller unit 11 sends the input data, the decimal point position, and the data type of the data involved in the operations to the conversion unit 13. The conversion unit 13 performs data type conversion on the input data according to the decimal point position and the data type of the data involved in the operations, such that the data type of the input data is consistent with that of the data involved in the operations. And then, the input data converted is transferred to the operation unit 12, and the primary processing circuit 101 and the secondary processing circuits 102 of the operation unit 12 perform operations on the input data converted. If it is determined that the data type of the input data is consistent with that of the data involved in the operations, the controller unit 11 transfers the input data to the operation unit 12, and the primary processing circuit 101 and the secondary processing circuits 102 of the operation unit 12 directly perform operations on the input data without performing data type conversion. [0012] The primary processing circuit may be configured to perform pre-processing on the second input data and to send data and the plurality of operation instructions between the plurality of secondary processing circuits and the primary processing circuit. [0013] The plurality of secondary processing circuits may be configured to perform an intermediate operation to obtain a plurality of intermediate results according to the second input data and the plurality of operation instructions sent from the primary processing circuit, and to transfer the plurality of intermediate results to the primary processing circuit); perform tensor operations on the input operand in the second number representation to generate an output operand in the second number representation ([0113] and the data may be n-dimensional data, where n is an integer greater than or equal to one. For example, if n=1, the data is a one-dimensional data (that is, a vector). For another example, if n=2, the data is a two-dimensional data (that is, a matrix). If n=3 or more, the data is a multi-dimensional tensor. [0008] The conversion unit may be configured to convert the first input data into second input data according to the opcode and the opcode field of the data conversion instruction. [0009] The operation unit may be configured to perform operations on the second input data according to the plurality of operation instructions to obtain a computation result of the computation instruction); wherein the first number representation is one of a fixed-point number representation and a floating-point number representation, and the second number representation is the other one of the fixed-point number representation and the floating-point number representation ([0133] As illustrated in Table 3, if the identifier of data type conversion is 00, the conversion manner of data type is converting the fixed-point data into fixed-point data. If the identifier of data type conversion is 01, the conversion manner of data type is converting the floating point data into floating point data. If the identifier of data type conversion is 10, the conversion manner of data type is converting the fixed-point data into floating point data. If the identifier of data type conversion is 11, the conversion manner of data type is converting the floating point data into fixed-point data).
ZHANG does not explicitly disclose:
and select to enable or bypass the conversion circuit for output conversion of an output operand according to the operating parameters, wherein the output conversion, when enabled, converts from the second number representation to the first number representation.
However, in analogous art, MENG teaches:
and select to enable or bypass the conversion circuit for output conversion of an output operand according to the operating parameters, wherein the output conversion, when enabled, converts from the second number representation to the first number representation ([0077] The operation unit is configured to perform fixed point data operations on the some fixed point input neuron data and the some fixed point weight data to obtain some fixed point forward output results, and send the some fixed point forward output results to the conversion unit. [0078] The conversion unit 13 is configured to convert the some fixed point forward output results between fixed point data and floating point data to obtain a first set of some floating point forward operation results, and send the first set of some floating point forward operation results to the operation unit).
It would have been obvious to a person having ordinary skill in the art before the filing date of the invention to have combined MENG’s teaching of selectively enabling and bypassing the conversion unit for output conversion based on the conversion parameter, with ZHANG’s teaching of a processing unit that has an operation unit that perform tensor operations, and conversion units that selectively enables or bypasses input data conversion according to the conversion parameter, to realize, with a reasonable expectation of success, a system that has an operation unit that can perform tensor operations, and selectively enables or bypasses input data conversion as in ZHANG, and selectively enables or bypasses output data conversion, as in MENG. A person of ordinary skill would have been motivated to make this combination to result in a more flexible and efficient computation process in a multi-layered neural network.
Regarding claim 14, ZHANG further teaches that:
wherein the neural network processing unit is operative to: perform, for another given layer of the neural network, additional tensor operations on another input operand in the first number representation to generate another output operand in the first number representation ([0160] In an example, before the operation unit 12 of the computation device performs operations on data of an i.sup.th layer of a multi-layer neural network model, the controller unit 11 of the computation device acquires a configuration command, which may include a decimal point position and a data type of data involved in the operations. The controller unit 11 parses the configuration instruction to obtain the decimal point position and the data type of the data involved in the operations. [0133] As illustrated in Table 3, if the identifier of data type conversion is 00, the conversion manner of data type is converting the fixed-point data into fixed-point data. If the identifier of data type conversion is 01, the conversion manner of data type is converting the floating point data into floating point data. If the identifier of data type conversion is 10, the conversion manner of data type is converting the fixed-point data into floating point data. If the identifier of data type conversion is 11, the conversion manner of data type is converting the floating point data into fixed-point data. [0113] and the data may be n-dimensional data, where n is an integer greater than or equal to one. For example, if n=1, the data is a one-dimensional data (that is, a vector). For another example, if n=2, the data is a two-dimensional data (that is, a matrix). If n=3 or more, the data is a multi-dimensional tensor).
Claim(s) 15 is rejected under 35 U.S.C. 103 as being unpatentable over ZHANG, in view of MENG, as applied to claim 13 above, and in further view of DAWWD.
Regarding claim 15, while ZHANG teaches the neural network processing unit that has operation circuit that can perform tensor operations with floating point and fixed point data representations and conversion circuits for selectively enable or bypass conversion of input data according to the conversion parameter, the combination of ZHANG and MENG does not explicitly disclose:
wherein the neural network processing unit is time-shared among multiple layers of the neural network by operating on one layer at a time.
However, in analogous art, DAWWD teaches:
wherein the neural network processing unit is time-shared among multiple layers of the neural network by operating on one layer at a time (Figure 4 below shows the time-sharing operation. (Section IV, A. Time Sharing Processing Flow) Therefore, in the proposed architecture of CNN, a repetitively utilized neuron circuits are performed using time-sharing operation. Each group of receptive field vectors are processed in a parallel manner spending a time slice that is enough to complete their respective computations, then another group which may be located in successive layers (except the feedforward layer) takes its role, and then return to the fist layer and so on, until the all neurons’ outputs in the last simple layer are calculated. The CNN architecture shown in Fig. 2 can share the time according to the above approach. Fig. 4 shows the time sharing operation of this architecture).
PNG
media_image3.png
490
1000
media_image3.png
Greyscale
It would have been obvious to a person having ordinary skill in the art before the filing date of the invention to have combined DAWWD’s teaching of operations being time-shared among the layers of a neural network and operates on one layer at a time, with the combination of ZHANG and MENG’s teaching of a processing unit that has an operation circuit perform tensor operations in one or more layers of a neural network in fixed-point and floating-point number representation, and a conversion unit that selectively bypass or enables conversion on input and output data according to the conversion parameter, to realize, with a reasonable expectation of success, a system that has an operation unit that performs tensor operation in one or more layers of a neural network, and a conversion unit that converts input and output data between fixed-point and floating-point according to the conversion parameter, as in the combination of ZHANG and MENG, and operations are time-shared among the layers by operating on one layer at a time, as in DAWWD. A person of ordinary skill would have been motivated to make this combination to reduce the resources required for implementation (DAWWD[Abstract]), and enable more efficient tensor operation while minimizing hardware requirement.
Claim(s) 16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over MENG, and in further view of KAWABE.
Regarding claim 16, MENG teaches the invention substantially as claimed, including:
A system comprising: one or more … circuits to perform floating-point tensor operations for one or more layers of the neural network ([0097] the primary processing circuit 101 is configured to preprocess data (including one or more of input neuron data, weight data, and input neuron gradients, where the data may be fixed point or floating point) and transfer data and operation instructions to the plurality of secondary processing circuits. [0098] The plurality of secondary processing circuits 102 are configured to perform intermediate operations in parallel according to data (fixed point or floating point) and operation instructions transferred by the primary processing circuit to obtain a plurality of intermediate results, and transfer the plurality of intermediate results to the primary processing circuit. [0077] The operation unit is configured to perform fixed point data operations on the some fixed point input neuron data and the some fixed point weight data to obtain some fixed point forward output results, and send the some fixed point forward output results to the conversion unit. [0131] The data may be n-dimensional data, where n is an integer greater than or equal to 1. For instance, when n=1, the data is one-dimensional data which is a vector, when n=2, the data is two-dimensional data which is a matrix, and when n=3 or above 3, the data is multi-dimensional tensor), one or more … circuits to perform fixed-point tensor operations for other one or more layers of the neural network ([0079] The operation unit is configured to perform operations (floating point operations) on the some input neuron data and the some weight data to obtain a second set of some floating point forward operation results, and combine the first set of some floating point forward operation results and the second set of some floating point forward operation results to obtain forward output results of the i.sup.th layer. [0131] The data may be n-dimensional data, where n is an integer greater than or equal to 1. For instance, when n=1, the data is one-dimensional data which is a vector, when n=2, the data is two-dimensional data which is a matrix, and when n=3 or above 3, the data is multi-dimensional tensor); and one or more conversion circuits coupled to at least one of the floating-point circuits and the fixed-point circuits to convert between a floating-point number representation and a fixed- point number representation ([0078] The conversion unit 13 is configured to convert the some fixed point forward output results between fixed point data and floating point data to obtain a first set of some floating point forward operation results, and send the first set of some floating point forward operation results to the operation unit).
While MENG teaches a system that has operating circuits that perform tensor operation in floating point and fixed point in different layers of a neural network, and conversion circuits that are coupled to the operation circuits to convert between floating point and fixed point, it does not explicitly disclose:
one or more floating point circuits to perform floating-point tensor operations for one or more layers of the neural network
one or more fixed point circuits to perform fixed-point tensor operations for other one or more layers of the neural network
However, in analogous art, KAWABE teaches:
one or more fixed-point circuits to perform fixed-point tensor operations for other one or more layers of the neural network ([0025] The arithmetic processing device 100 includes a fixed-point operator 1 that executes an operation on a fixed-point number, a floating-point operator 2 that executes an operation on a floating-point number, a first converter 3, a selector 4, a statistical information acquirer 5, an update information generator 6, and a second converter 7. The fixed-point operator 1 includes, for example, a 16-bit multiply-accumulate operator and outputs an operation result DT1 (of, for example, 40 bits) to the selector 4). one or more floating point circuits to perform floating-point tensor operations for one or more layers of the neural network ([0026] The floating-point operator 2 includes, for example, a 32-bit multiplier, a 32-bit divider, or the like and outputs a result DT2 (of, for example, 32 bits) of executing an operation on a floating-point number to the first converter 3. For example, the 32-bit floating-point number includes an 8-bit decimal part and may represent 256 decimal point positions).
It would have been obvious to a person having ordinary skill in the art before the filing date of the invention to have combined KAWABE’s teaching of a fixed-point operator that operates with fixed point data representation in one layer of a neural network and a floating-point operator that operates with floating point data representation in another layer of a neural network, with MENG’s teaching of a system comprises of one more processing unit to perform tensor operations in different layers of a neural network and conversion circuits coupled to the operation circuits, to realize, with a reasonable expectation of success, a system that has multiple circuits performs tensor operation in one or more layers of a neural network and conversion units coupled to at least one of the processing unit, as in MENG, and uses fixed-point circuits for operations using fixed point data representation in one layer and floating-point circuits for operations using fixed point data representation, as in KAWABE. A person of ordinary skill would have been motivated to make this combination to result in an enhanced computation efficiency and flexibility when working with floating point and fixed point operations in different layers of a neural network, since some layer may benefit from the speed and power efficiency of fixed-point operations and some layer may benefits from the accuracy of floating-point operations.
Regarding claim 18, the combination of MENG and KAWABE teaches the limitations of claim 16. KAWABE further teaches that:
wherein output ports of one of the floating-point circuits and one of the fixed-point circuits are coupled, in parallel, to a multiplexer (KAWABE Figure 1 shows the floating-point operator and fixed-point operator coupled in parallel, and connected to a Selector 4. [0028] The selector 4 selects any of the operation results DT1 and DT3 based on a selection signal SEL and outputs the selected operation result DT1 or DT3 as an operation result DT4. The figure below shows the fixed-point operator and floating-pointer is connected by a selector).
PNG
media_image5.png
730
930
media_image5.png
Greyscale
Regarding claim 19, the combination of MENG and KAWABE teaches the limitation of claim 16. KAWABE further teaches that:
wherein the one or more conversion circuits includes a floating- point to fixed-point converter that is coupled to an input port of a fixed-point circuit or an output port of a floating-point circuit (KAWABE figure 1 shows that the first converter, a floating-point to fixed-point converter, is attached to the output of the floating point operator output. [0027] The first converter 3 converts the 32-bit operation result DT2 (floating-point number) obtained by the floating-point operator 2 to, for example, an operation result DT3 (fixed-point number) of 32 bits that are an example of a second bit width).
PNG
media_image6.png
730
930
media_image6.png
Greyscale
Regarding claim 20, MENG further teaches that:
wherein the one or more conversion circuits includes a fixed- point to floating-point converter that is coupled to an input port of a floating-point circuit or an output port of a fixed-point circuit ([0075] If artificial neural network operations have operations of multiple layers, input neurons and output neurons of the multi-layer operations do not refer to neurons in an input layer and in an output layer of the entire neural network. For any two adjacent layers in the network, neurons in a lower layer of the network forward operations are the input neurons, and neurons in an upper layer of the network forward operations are the output neurons. [0051] The controller unit 11 is connected to the operation unit 12 and the conversion unit 13 (the conversion unit may be set separately, or may be integrated in the controller unit or the operation unit). [0078] The conversion unit 13 is configured to convert the some fixed point forward output results between fixed point data and floating point data to obtain a first set of some floating point forward operation results, and send the first set of some floating point forward operation results to the operation unit).
Claim(s)17 is rejected under 35 U.S.C. 103 as being unpatentable over MENG, in view of KAWABE, as applied to claim 16, and in view of Shimokawa et al. Patent. No: 5613143(hereafter SHIMOKAWA).
Regarding claim 17, while MENG teaches a system that comprises of multiple processing unit to perform tensor operations, the combination of MENG and KAWABE does not explicitly disclose:
wherein the one or more floating-point circuits and the one or more fixed-point circuits are coupled to one another in a series according to a predetermined order
However, in analogous art, SHIMOKAWA teaches:
wherein the one or more floating-point circuits and the one or more fixed-point circuits are coupled to one another in a series according to a predetermined order (SHIMOKAWA Figure 3 shows fixed-point operation circuit A0 and floating-point operation circuit B0 are connected in series. (188) In the controller according to the present invention, since the internal data bus B is commonly connected to the bit operation circuit 80, the fixed-point operation circuit A0, the floating-point operation circuit B0, and the register circuit 90, a pipeline arrangement can be obtained. Even in a comparison instruction (e.g., two numbers are compared, and a comparison result is stored in a bit register) wherein a bit operation and a fixed- or floating-point operation are simultaneously performed, the mutual connection can be systematically performed without any time delay. Even if the type of operation is changed, no overhead is imposed.]).
PNG
media_image7.png
586
896
media_image7.png
Greyscale
It would have been obvious to a person having ordinary skill in the art before the filing date of the invention to have combined SHIMOKAWA’s teaching of a floating-point operation circuit and a fixed-point operation circuit connected in series, with the combination of MENG and KAWABE’s teaching of a system comprises of one more floating point and fixed point circuits to perform tensor operations in different layers of a neural network, to realize, with a reasonable expectation of success, a system that has multiple floating-point and fixed-point circuits performs tensor operation in one or more layers of a neural network, as in the combination of MENG and KAWABE, and the floating-point operation circuit and fixed-point operation circuit are connected in series, as in SHIMOKAWA. A person of ordinary skill would have been motivated to make this combination to result in operation without causing any delay in switching between the floating point and the fixed point (SHIMOKAWA (188)).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QI LIU whose telephone number is (571)272-2083. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached on (571)272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/QI LIU/ Examiner, Art Unit 2144
/MICHAEL W AYERS/Primary Examiner, Art Unit 2195
Prosecution Insights