DETAILED ACTION
Response to Amendment
The amendment filed on 24 March 2026 has been entered.
Claims 1-7 are pending.
Claims 1-7 are amended.
Claims 8-12 are new.
Claims 1-12 will be pending.
Response to Arguments
Applicant’s remarks, regarding the rejections of claims under 35 USC 103, have been fully considered.
Applicant notes Claim 1 has been amended to recite:
“a command fetch unit that separately reads, from a memory, commands for operating the convolution operation circuit and commands for operating the quantization operation circuit.”.
Applicant submits that neither Lee nor Rouhani provides any clear motivation or instruction for applying the technology of Lee to NN accelerators with different purposes. For at least the foregoing reasons, Applicant submits that Claim 1 is patentably distinct from the cited art.
Applicant’s arguments have been considered, but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant submits Claims 8-12 are allowable over the cited references.
Examiner notes Applicant’s arguments, as outlined, are directed to new claim limitations for which Examiner has not yet made a prima facie case for, rendering Applicant’s arguments moot.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 7 is objected to because of the following informalities: “command” in line 6 should be “commands”. Appropriate correction is required.
Claim 7 is objected to because of the following informalities: “command” in line 8 should be “commands”. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 6-7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 6 recites the limitation "the command" in line 15. There is insufficient antecedent basis for this limitation in the claim. For examination purposes, the term "the command" has been construed to be “a command”. Claim 7, which is dependent on claim 6, is similarly rejected.
Allowable Subject Matter
The following is a statement of reasons for the indication of allowable subject matter.
The prior art does not anticipate, or render obvious as a whole, the claim limitations of Claim 5 as disclosed in Applicant’s claims.
Presented with the additional limitations of Claim 5, the prior art fails to make a prima facie case
of obviousness for limitations,
“further comprising: a semaphore for controlling data flow via the first memory and the second memory; wherein: the convolution operation circuit, when operated based on the commands for operating the convolution operation circuit, performs an operation on the semaphore; and the quantization operation circuit, when operated based on the commands for operating the quantization operation circuit, performs an operation on the semaphore”,
because the neural network circuits shown to be found in the prior arts, made of record (see Non-Final Office Action, mailed 27 October 2025) cannot be reconciled with the above limitations and an obviousness determination would require impermissible hindsight.
In summary, the references made of record, fail to disclose the required claimed technical features recited by the Claim 5 limitations as a whole. The claim, definite, and enabled by the Specification, is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Similarly, Claim 7, definite, and enabled by the Specification, is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. Claims 11-12, which depend directly or indirectly on Claim 5, would be considered allowable.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2 are rejected under 35 U.S.C. 103 as being unpatentable over Rouhani et al. (U.S. Pre-Grant Publication No. 20200193274, hereinafter ‘Rouhani'), in view of Zhou et al. (U.S. Pre-Grant Publication No. 20190251424, hereinafter 'Zhou').
Regarding claim 1, Rouhani teaches A neural network circuit comprising: a convolution operation circuit that performs a convolution operation on input data; a quantization operation circuit that performs a quantization operation on convolution operation output data from the convolution operation circuit ([0031] Neural network operations are used in many artificial intelligence operations. Often, the bulk of the processing a convolution operation circuit that performs a convolution operation on input data operations performed in implementing a neural network is in performing Matrix×Matrix or Matrix×Vector multiplications or convolution operations. Such operations are compute- and memory-bandwidth intensive, where the size of a matrix may be, for example, 1000×1000 elements (e.g., 1000×1000 numbers, each including a sign, mantissa, and exponent) or larger and there are many matrices used. As discussed herein, BFP techniques can be applied to such operations to reduce the demands for computation as well as memory bandwidth in a given system, whether it is an FPGA, CPU or another hardware platform. As used herein, the use of the term “element” herein refers to a member of such a matrix or vector.; [0050] In one example, the neural network accelerator 180 on convolution operation output data from the convolution operation circuit receives and returns normal-precision values 150 from the neural network module 130. As illustrated in FIG. 1, the subgraph accelerator 186 can perform a bulk of its operations using quantized floating-point and an interface between the subgraph accelerator 186 and the neural network module 130 can use full-precision values for communicating information between the modules. The normal-precision values can be represented in 16-, 32-, 64-bit, or other suitable floating-point format. For example, a portion of values representing the neural network can be received, including edge weights, activation values, or other suitable parameters for quantization. The a quantization operation circuit that performs a quantization operation normal-precision values 150 are provided to a normal-precision floating-point to quantized floating-point converter 152, which converts the normal-precision value into quantized values. Quantized floating-point operations 154 can then be performed on the quantized values. The quantized values can then be converted back to a normal-floating-point format using a quantized floating-point to normal-floating-point converter which produces normal-precision floating-point values. As a specific example, the subgraph accelerator 186 can be used to accelerate a given layer of a neural network, and the vector-vector, matrix-vector, matrix-matrix, and convolution operations can be performed using quantized floating-point operations and less compute-intensive operations (such as adding a bias value or calculating an activation function) can be performed using normal-floating-point operations.); and
Rouhani fails to teach a command fetch unit that separately reads, from a memory, commands for operating the convolution operation circuit and commands for operating the quantization operation circuit.
Zhou teaches a command fetch unit that separately reads, from a memory, commands for operating the convolution operation circuit and commands for operating the quantization operation circuit ([0036] The plurality of functional operation modules are connectable by way of combinations to form a plurality of operation paths; wherein commands for operating the convolution operation circuit and commands for operating the quantization operation circuit each of the operation paths is used to implement a computing logic; the computing logic includes at least one of: a convolution processing, a deconvolution processing, a pooling processing, a quantizing processing and a fully connected processing.; [0063] The a command fetch unit that separately reads, from a memory controller is configured to read, from the memory, a current single instruction corresponding to a current layer operation as required by the operation apparatus, and parse the module selecting parameter and the module operating parameter included in the current single instruction, so as to determine a operation path corresponding to the current single instruction; the controller is further configured to send a control signal to the operation apparatus, so that the operation apparatus is connected with the operation path corresponding to the current single instruction. Specifically, the controller may parse the single instruction carrying the module selecting parameter and the module operating parameter by using an instruction decoder, thereby generating a control signal, so that the operation apparatus forms a operation path corresponding to the single instruction.).
Rouhani and Zhou are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Rouhani, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Zhou to Rouhani before the effective filing date of the claimed invention in order to improve utilization of the hardware resources (cf. Zhou, [0005] In view of this, the objectives of the present disclosure are to provide an operation apparatus, an operation execution device and an operation execution method, which can make improvement with respect to the technical problem that the hardware resources for implementing the neural network are apt to waste, and can effectively improve utilization of the hardware resources.).
Regarding claim 2, Rouhani, as modified by Zhou, teaches The neural network circuit of claim 1.
Zhou teaches wherein the command fetch unit has: a first fetch unit that reads, the commands for operating the convolution operation circuit and supplies the commands for operating the convolution operation circuit to the convolution operation circuit ([0036] The plurality of functional operation modules are connectable by way of combinations to form a plurality of operation paths; wherein each of the operation paths is used to implement a computing logic; the computing logic includes at least one of: a convolution processing, a deconvolution processing, a pooling processing, a quantizing processing and a fully connected processing.; [0063] The a first fetch unit that reads, the commands for operating the convolution operation circuit controller is configured to read, from the memory, a current single instruction corresponding to a current layer operation as required by the operation apparatus, and parse the module selecting parameter and the module operating parameter included in the current single instruction, so as to determine a operation path corresponding to the current single instruction; the controller is further and supplies the commands for operating the convolution operation circuit to the convolution operation circuit configured to send a control signal to the operation apparatus, so that the operation apparatus is connected with the operation path corresponding to the current single instruction. Specifically, the controller may parse the single instruction carrying the module selecting parameter and the module operating parameter by using an instruction decoder, thereby generating a control signal, so that the operation apparatus forms a operation path corresponding to the single instruction.); and
a second fetch unit that reads, the commands for operating the quantization operation circuit and supplies the commands for operating the quantization operation circuit to the quantization operation circuit ([0036] The plurality of functional operation modules are connectable by way of combinations to form a plurality of operation paths; wherein each of the operation paths is used to implement a computing logic; the computing logic includes at least one of: a convolution processing, a deconvolution processing, a pooling processing, a quantizing processing and a fully connected processing.; [0063] The a second fetch unit that reads, the commands for operating the quantization operation circuit controller is configured to read, from the memory, a current single instruction corresponding to a current layer operation as required by the operation apparatus, and parse the module selecting parameter and the module operating parameter included in the current single instruction, so as to determine a operation path corresponding to the current single instruction; the controller is further and supplies the commands for operating the quantization operation circuit to the quantization operation circuit configured to send a control signal to the operation apparatus, so that the operation apparatus is connected with the operation path corresponding to the current single instruction. Specifically, the controller may parse the single instruction carrying the module selecting parameter and the module operating parameter by using an instruction decoder, thereby generating a control signal, so that the operation apparatus forms a operation path corresponding to the single instruction.).
Rouhani and Zhou are combinable for the same rationale as set forth above with respect to claim 1.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Rouhani, Zhou, and further in view of Henry et al. (U.S. Pre-Grant Publication No. 20180157970, hereinafter 'Henry').
Regarding claim 3, Rouhani, as modified by Zhou, teaches The neural network circuit of claim 2.
Rouhani, as modified by Zhou, fails to teach wherein the first fetch unit has: a command pointer that holds memory addresses, in the memory, at which the commands for operating the convolution operation circuit are stored; and a command counter that holds a command count of the commands for operating the convolution operation circuit that are stored.
Henry teaches wherein the first fetch unit has: a command pointer that holds memory addresses, in the memory, at which the commands for operating the convolution operation circuit are stored; and a command counter that holds a command count of the commands for operating the convolution operation circuit that are stored ([0181] Referring now to FIG. 24, a block diagram illustrating an example of data structures used by the NNU 121 of FIG. 1 for operating the convolution operation circuit to perform a convolution operation are shown. The block diagram includes a convolution kernel 2402, a data array 2404, and the data RAM 122 and weight RAM 124 of FIG. 1.; [0058] The instruction fetch unit 101 controls the fetching of architectural instructions 103 from system memory (not shown) into the instruction cache 102. The instruction fetch unit 101 provides a at which the commands for operating the convolution operation circuit are stored fetch address to the instruction cache 102 that specifies a memory address at which the processor 100 fetches a cache line of architectural instruction bytes into the instruction cache 102. The fetch address is based on the current value of the a command pointer that holds memory addresses, in the memory instruction pointer (not shown), or program counter, of the processor 100. Normally, the a command counter that holds a command count of the commands that are stored program counter is incremented sequentially by the size of an instruction unless a control instruction is encountered in the instruction stream, such as a branch, call or return instruction, or an exception condition occurs, such as an interrupt, trap, exception or fault, in which case the program counter is updated with a non-sequential address, such as a branch target address, return address or exception vector. Generally speaking, the program counter is updated in response to the execution of instructions by the execution units 112/121. The program counter may also be updated in response to detection of an exception condition such as the instruction translator 104 encountering an instruction 103 that is not defined by the instruction set architecture of the processor 100.)
Rouhani, Zhou, and Henry are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Rouhani and Zhou, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Henry to Rouhani before the effective filing date of the claimed invention in order to require fewer multipliers and a smaller, simpler and faster adder circuit (e.g., a 2-input adder) in the neuron than an adder that would be required to add all, or even a subset of, the products associated with all the connection inputs (cf. Henry, [0073] However, rather than performing all the multiplies associated with all the connection inputs and then adding all the products together as in a conventional manner, advantageously each neuron is configured to perform, in a given clock cycle, the weight multiply operation associated with one of the connection inputs and then add (accumulate) the product with the accumulated value of the products associated with connection inputs processed in previous clock cycles up to that point. Assuming there are M connections to the neuron, after all M products have been accumulated (which takes approximately M clock cycles), the neuron performs the activation function on the accumulated value to generate the output, or result. This has the advantage of requiring fewer multipliers and a smaller, simpler and faster adder circuit (e.g., a 2-input adder) in the neuron than an adder that would be required to add all, or even a subset of, the products associated with all the connection inputs.).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Rouhani, Zhou, and further in view of Lee et al. (U.S. Pre-Grant Publication No. 20140025930, hereinafter 'Lee').
Regarding claim 4, Rouhani, as modified by Zhou, teaches The neural network circuit of claim 1.
Lee teaches further comprising: a first memory in which the input data is stored; and a second memory in which the convolution operation output data is stored; wherein: quantization operation output data from the quantization operation circuit is stored in the first memory; and the quantization operation output data stored in the first memory is input as the input data to the convolution operation circuit ([0068] The out-of-order execution data units 453 may include conventionally understood arithmetic and logic units (ALUs), multipliers, dividers, branches, load and store units, and/or floating point units.; [0097] The outbound processor core continuously performs a normal operation (S230), in response to preparation for a task movement output from the inbound processor core, the outbound processor core wherein: quantization operation output data from the quantization operation circuit is stored in the first memory stores data necessary for storage (or to store) in a first memory in which the input data is stored a corresponding memory and transmits data necessary for transmission to the inbound processor core (S250).; [0098] The data necessary for transmission are all transmitted from the outbound processor core to the inbound processor core, the outbound processor core is powered-down (S260). The memory may be the level 1-data cache 487 or another level of memory. Data stored in the memory may include a start address of a task to be performed next.; [0099] The the quantization operation output data stored in the first memory is input as the input data to the convolution operation circuit inbound processor core receives data transmitted from the outbound processor core and a second memory in which the convolution operation output data is stored stores the received data in a corresponding memory (S270), and performs a normal operation (S280).).
Rouhani, Zhou, and Lee are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Rouhani and Zhou, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Lee to Rouhani before the effective filing date of the claimed invention in order to improve performance of a system on chip (cf. Lee, [0002] The present inventive concept relates to multi-core processors, and more particularly, to multi-core processors including a plurality of processor cores sharing a level 1 (L1) cache, and devices having same.; [0003] To improve performance of a system on chip (SoC), certain circuits and/or methods that effectively increase the operating frequency of a central processing unit (CPU) within the SoC has been proposed. One approach to increasing the operating frequency of the CPU increases a number of pipeline stages.; [0004] One technique referred to as dynamic frequency and voltage scaling (DVFS) has been successfully used to reduce power consumption in computational systems, particularly those associated with mobile devices. However, under certain workload conditions, the application of DVFS to a CPU has proved inefficient.).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Rouhani, Zhou, and further in view of Moreau et al. (NPL: "A Hardware–Software Blueprint for Flexible Deep Learning Specialization", hereinafter 'Moreau').
Regarding claim 6, Rouhani teaches A neural network circuit control method for a neural network circuit comprising a convolution operation circuit that performs a convolution operation on input data, a quantization operation circuit that performs a quantization operation on convolution operation output data from the convolution operation circuit ([0031] Neural network operations are used in many artificial intelligence operations. Often, the bulk of the processing a convolution operation circuit that performs a convolution operation on input data operations performed in implementing a neural network is in performing Matrix×Matrix or Matrix×Vector multiplications or convolution operations. Such operations are compute- and memory-bandwidth intensive, where the size of a matrix may be, for example, 1000×1000 elements (e.g., 1000×1000 numbers, each including a sign, mantissa, and exponent) or larger and there are many matrices used. As discussed herein, BFP techniques can be applied to such operations to reduce the demands for computation as well as memory bandwidth in a given system, whether it is an FPGA, CPU or another hardware platform. As used herein, the use of the term “element” herein refers to a member of such a matrix or vector.; [0050] In one example, the neural network accelerator 180 on convolution operation output data from the convolution operation circuit receives and returns normal-precision values 150 from the neural network module 130. As illustrated in FIG. 1, the subgraph accelerator 186 can perform a bulk of its operations using quantized floating-point and an interface between the subgraph accelerator 186 and the neural network module 130 can use full-precision values for communicating information between the modules. The normal-precision values can be represented in 16-, 32-, 64-bit, or other suitable floating-point format. For example, a portion of values representing the neural network can be received, including edge weights, activation values, or other suitable parameters for quantization. The a quantization operation circuit that performs a quantization operation normal-precision values 150 are provided to a normal-precision floating-point to quantized floating-point converter 152, which converts the normal-precision value into quantized values. Quantized floating-point operations 154 can then be performed on the quantized values. The quantized values can then be converted back to a normal-floating-point format using a quantized floating-point to normal-floating-point converter which produces normal-precision floating-point values. As a specific example, the subgraph accelerator 186 can be used to accelerate a given layer of a neural network, and the vector-vector, matrix-vector, matrix-matrix, and convolution operations can be performed using quantized floating-point operations and less compute-intensive operations (such as adding a bias value or calculating an activation function) can be performed using normal-floating-point operations.), and
Rouhani fails to teach a command fetch unit that reads, from a memory, commands for operating the convolution operation circuit and commands for operating the quantization operation circuit wherein the neural network circuit control method includes: a step of making the command fetch unit separately read the commands for operating the convolution operation circuit and the commands for operating the quantization operation circuit from the memory and separately supply the commands to the convolution operation circuit and the quantization operation circuit; and a step of making the convolution operation circuit or the quantization operation circuit operate in parallel based on the command that was supplied.
Zhou teaches a command fetch unit that reads, from a memory, commands for operating the convolution operation circuit and commands for operating the quantization operation circuit
wherein the neural network circuit control method includes: a step of making the command fetch unit separately read the commands for operating the convolution operation circuit and the commands for operating the quantization operation circuit from the memory and separately supply the commands to the convolution operation circuit and the quantization operation circuit ([0036] The plurality of functional operation modules are connectable by way of combinations to form a plurality of operation paths; wherein for operating the convolution operation circuit and the commands for operating the quantization operation circuit each of the operation paths is used to implement a computing logic; the computing logic includes at least one of: a convolution processing, a deconvolution processing, a pooling processing, a quantizing processing and a fully connected processing.; [0063] The a step of making the command fetch unit separately read the commands controller is configured to read, from the memory from the memory, a current single instruction corresponding to a current layer operation as required by the operation apparatus, and parse the module selecting parameter and the module operating parameter included in the current single instruction, so as to determine a operation path corresponding to the current single instruction; the controller is further and separately supply the commands to the convolution operation circuit and the quantization operation circuit configured to send a control signal to the operation apparatus, so that the operation apparatus is connected with the operation path corresponding to the current single instruction. Specifically, the controller may parse the single instruction carrying the module selecting parameter and the module operating parameter by using an instruction decoder, thereby generating a control signal, so that the operation apparatus forms a operation path corresponding to the single instruction.); and
Rouhani and Zhou are combinable for the same rationale as set forth above with respect to claim 1.
Moreau teaches a step of making the convolution operation circuit or the quantization operation circuit operate in parallel based on the command that was supplied-- ([Exposing task-level pipeline parallelism:, pg. 11] Task level operate in parallel pipeline parallelism (TLPP) is a vital VTA feature because it enables simultaneous use of compute and memory resources to maximize their utilization. TLPP is based on the paradigm of access-execute decoupling.8 To extract TLPP, we partition tasks into two mutually exclusive execution contexts, so that concurrent load, compute, and store operations do not interfere with one another.; [Task level ISA:, pg. 11] The based on the command that was supplied fetch module loads task instructions from DRAM and dispatches them according to their type to the corresponding command queues connected to load, compute, and store modules. The load module loads input, weight, and bias tensor tiles from DRAM into on-chip memories. The compute module loads a microcoded kernel from DRAM into on-chip memory. The compute module executes the microcoded kernel to perform either a dense linear algebra computation via the GEMM core or a pairwise arithmetic operation via the Tensor ALU.; [Compute module:, pg. 11] Two functional units perform operations on the register file, i.e., the or the quantization operation circuit tensor ALU and the a step of making the convolution operation circuit GEMM core. The tensor ALU performs element-wise tensor operations, such as addition, activation, normalization, and pooling tasks. The GEMM core performs high-arithmetic-intensity matrix multiplication over input and weight tensors to implement common DL operators including 2-D convolutions or fully connected layers.).
Rouhani, Zhou, and Moreau are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Rouhani and Zhou, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Moreau to Rouhani before the effective filing date of the claimed invention in order to implement a programmable DL architecture for evolving workloads (cf. Moreau, [Abstract, pg. 8] This article describes the Versatile Tensor Accelerator (VTA), a programmable DL architecture designed to be extensible in the face of evolving workloads. VTA achieves “flexible specialization” via a parameterizable architecture, two-level Instruction Set Architecture (ISA), and a Justin Time (JIT) compiler.).
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Rouhani, Zhou, and further in view of Qureshi et al. (NPL: "NeuroMAX: A High Throughput, Multi-Threaded, Log-Based Accelerator for Convolutional Neural Networks", hereinafter 'Qureshi').
Regarding claim 8, Rouhani, as modified by Zhou, teaches The neural network circuit of claim 1.
Rouhani, as modified by Zhou, fails to teach wherein: the convolution operation circuit has a state controller for controlling the convolution operation circuit based on the commands for the convolution operation circuit, and the quantization operation circuit has a state controller for controlling the quantization operation circuit based on the commands for operating the quantization operation circuit.
Qureshi teaches wherein: the convolution operation circuit has a state controller for controlling the convolution operation circuit based on the commands for the convolution operation circuit, and the quantization operation circuit has a state controller for controlling the quantization operation circuit based on the commands for operating the quantization operation circuit ([4.1 Top-Level, pg. 3] The the convolution operation circuit has a state controller for controlling the convolution operation circuit based on the commands for the convolution operation circuit state controller modifies the configurable adders and determines the dataflow to be used for convolution operation. The linear convolution outputs are sent to the post processing block which performs ReLU operation and the quantization operation circuit has a state controller for controlling the quantization operation circuit based on the commands for operating the quantization operation circuit quantizes the results back into log values using pre-computed log table. These output log values are loaded into the output SRAMs and sent back to the on-chip DDR memory to be used for processing the next layer.).
Rouhani, Zhou, and Qureshi are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Rouhani and Zhou, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Qureshi to Rouhani before the effective filing date of the claimed invention in order to provide a 200% increase in peak throughput per PE count while only incurring a 6% increase in area overhead compared to a single, linear multiplier PE core with same output bit precision (cf. Qureshi, [Abstract, pg. 1] Convolutional neural networks (CNNs) require high throughput hardware accelerators for real time applications owing to their huge computational cost. Most traditional CNN accelerators rely on single core, linear processing elements (PEs) in conjunction with 1D dataflows for accelerating convolution operations. This limits the maximum achievable ratio of peak throughput per PE count to unity. Most of the past works optimize their dataflows to attain close to a 100% hardware utilization to reach this ratio. In this paper, we introduce a high throughput, multi-threaded, log-based PE core. The designed core provides a 200% increase in peak throughput per PE count while only incurring a 6% increase in area overhead compared to a single, linear multiplier PE core with same output bit precision. We also present a 2D weight broadcast dataflow which exploits the multi-threaded nature of the PE cores to achieve a high hardware utilization per layer for various CNNs. The entire architecture, which we refer to as NeuroMAX, is implemented on Xilinx Zynq 7020 SoC at 200 MHz processing clock. Detailed analysis is performed on throughput, hardware utilization, area and power breakdown, and latency to show performance improvement compared to previous FPGA and ASIC designs.).
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Rouhani, Zhou, Lee, and further in view of Kang et al. (U.S. Pre-Grant Publication No. 20200242456, hereinafter 'Kang').
Regarding claim 9, Rouhani, as modified by Zhou and Lee, teaches The neural network circuit of claim 4.
Rouhani, as modified by Zhou and Lee, fails to teach wherein: the first memory, the convolution operation circuit, the second memory, and the quantization operation circuit are formed in a loop.
Kang teaches wherein: the first memory, the convolution operation circuit, the second memory, and the quantization operation circuit are formed in a loop ([0049] FIG. 3 illustrates a are formed in a loop block diagram of a neural network accelerator of FIG. 1. Referring to FIG. 3, the neural network accelerator 100 may include the convolution operation circuit an operator 110, the quantization operation circuit a quantizer 120, a compressor 130, and a decompressor 140. The operator 110 may include a multiplier 111 and an accumulator 112.; [0039] An example is illustrated in FIG. 1 as the the first memory first memory 200 and the the second memory second memory 300 are independent of each other, but the inventive concept is not limited thereto. The first memory 200 and the second memory 300 may be implemented with one memory.; [0045] As described above, the accelerator system 1000 may tile an input feature map and filter data based on the channel loop tiling. In the case where the input feature map and the filter data are tiled, even though the size of the input feature map and the filter data is large, the input feature map and the filter data may be accommodated in the on-chip memory of the neural network accelerator 100. In the case where the output feature map OFM is generated based on the channel loop tiling, the partial sum PS may be generated from the neural network accelerator 100, and the generated partial sum PS may be stored in an external memory. The partial sum PS stored in the external memory may be provided to the neural network accelerator 100 as the previous partial sum PPS for the purpose of generating the output feature map OFM. As such, there may be required a space of the external memory for storing the partial sum PS, and there may occur communication overhead according to the exchange of the partial sum PS between the neural network accelerator 100 and the external memory.).
Rouhani, Zhou, Lee, and Kang are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Rouhani, Zhou, and Lee, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Kang to Rouhani before the effective filing date of the claimed invention in order to improve the accuracy of calculations (cf. Kang, [0005] Embodiments of the inventive concept provide a neural network accelerator for improving the accuracy of calculation which decreases due to an error according to channel loop tiling and an operating method thereof.).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Rouhani, Zhou, Lee, and further in view of Henry and Miyazaki et al. (U.S. Pre-Grant Publication No. 20050082572, hereinafter 'Miyazaki').
Regarding claim 10, Rouhani, as modified by Zhou and Lee, teaches The neural network circuit of claim 4.
Rouhani, as modified by Zhou and Lee, fails to teach wherein: the commands for the convolution operation circuit and the commands for the quantization operation circuit are read from the memory by the command fetch unit via an external bus connected to an external host CPU.
Henry teaches wherein: the commands for the convolution operation circuit and the commands for the quantization operation circuit are read from the memory by the command fetch unit via an external bus connected to an external host CPU ([0056] Referring now to FIG. 1, a block diagram illustrating a processor 100 that includes a neural network unit (NNU) 121 is shown. The processor 100 includes by the command fetch unit an instruction fetch unit 101, an instruction cache 102, and instruction translator 104, a rename unit 106, reservation stations 108, media registers 118, general purpose registers (GPR) 116, execution units 112 other than the NNU 121, and a memory subsystem 114.; [0065] The the commands for the convolution operation circuit execution units 112 include one or more load/store units (not shown) that load data are read from the memory from the memory subsystem 114 and store data to the memory subsystem 114. Preferably, the memory subsystem 114 includes a memory management unit (not shown), which may include, e.g., translation lookaside buffers and a tablewalk unit, a level-1 data cache (and the instruction cache 102), a level-2 unified cache, and a via an external bus connected to an external host CPU bus interface unit that interfaces the processor 100 to system memory.).
Rouhani, Zhou, Lee, and Henry are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Rouhani, Zhou, and Lee, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Henry to Rouhani before the effective filing date of the claimed invention in order to require fewer multipliers and a smaller, simpler and faster adder circuit (e.g., a 2-input adder) in the neuron than an adder that would be required to add all, or even a subset of, the products associated with all the connection inputs (cf. Henry, [0073] However, rather than performing all the multiplies associated with all the connection inputs and then adding all the products together as in a conventional manner, advantageously each neuron is configured to perform, in a given clock cycle, the weight multiply operation associated with one of the connection inputs and then add (accumulate) the product with the accumulated value of the products associated with connection inputs processed in previous clock cycles up to that point. Assuming there are M connections to the neuron, after all M products have been accumulated (which takes approximately M clock cycles), the neuron performs the activation function on the accumulated value to generate the output, or result. This has the advantage of requiring fewer multipliers and a smaller, simpler and faster adder circuit (e.g., a 2-input adder) in the neuron than an adder that would be required to add all, or even a subset of, the products associated with all the connection inputs.).
Miyazaki teaches wherein: the commands for the convolution operation circuit and the commands for the quantization operation circuit are read from the memory by the command fetch unit via an external bus connected to an external host CPU ([0052] FIG. 2 illustrates a semiconductor integrated circuit according to the present invention. The semiconductor integrated circuit 1 illustrated in the figure is constituted as a microcomputer or system LSI. The semiconductor integrated circuit 1 is obtained by forming the following over one semiconductor chip by, for example, a publicly known CMOS integrated circuit fabrication technique: a central processing unit (CPU) 2, RAM 3, ROM 4, a bus controller (BSC) 5, and a peripheral circuit 6. However, its constitution is not limited to this. The CPU 2 performs computation and control processing by by the command fetch unit fetching instructions and decoding the fetched instructions. The are read from the memory RAM 3 is used as a work area or area for temporarily storing data when the CPU 2 performs computation and control processing. The ROM 4 holds operating programs for the CPU 2 and parameter data, and information stored in the ROM 4 is used by the CPU 2, the peripheral circuit 6, or the like. The peripheral circuit 6 includes an A-D converter, a D-A converter, a timer/counter, an input/output port circuit, the commands for the quantization operation circuit accelerators, typified by digital signal processor (DSP) and ECC circuit, for the CPU 2, other custom logic circuits, and the like. The bus controller 5 carries out control of via an external bus connected to an external host CPU external bus access required for data fetch and instruction fetch by the CPU 2, and the like operations.).
Rouhani, Zhou, Lee, Henry, and Miyazaki are considered to be analogous to the claimed invention because they are in the same field of machine learning. In view of the teachings of Rouhani, Zhou, Lee, and Henry, it would have been obvious for a person of ordinary skill in the art to apply the teachings of Miyazaki to Rouhani before the effective filing date of the claimed invention in order to improve the speed and reduce the power consumption of a nonvolatile memory of the type that is effectively applied to a microcomputer, a system LSI, and the like (cf. Miyazaki, [0001] The present invention relates to a semiconductor integrated circuit of the type having a nonvolatile memory. More particularly, it relates to a technique which leads to an improvement in the enhancement of the speed and a reduction of the power consumption of a nonvolatile memory of the type that is effectively applied to a microcomputer, a system LSI, and the like.).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MAGGIE MAIDO whose telephone number is (703) 756-1953. The examiner can normally be reached M-Th: 6am - 4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached on (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MM/Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights