DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 07/27/2026 have been fully considered but they are not persuasive.
Regarding applicant’s remarks directed to the rejection of claims under 35 USC § 103, the arguments are directed to newly amended limitations that were not previously examined by the examiner. Therefore, applicants arguments are rendered moot. The examiner refers to the rejection under 35 USC § 103 in the current office action for more details.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-3 and 5-8 are rejected under 35 U.S.C. 103 as being unpatentable over US Pub No. US20190303757A1 Wang et al. (“Wang”) in view of Y. Guan et al., "Crane: Mitigating Accelerator Under-utilization Caused by Sparsity Irregularities in CNNs," in IEEE Transactions on Computers, vol. 69, no. 7, pp. 931-943, 1 July 2020, doi: 10.1109/TC.2020.2981080. (“Guan”) in further view of US Pub no. US20200059513A1 Sampathkumar et al. (“Sampathkumar”)
In regards to claim 1,
Wang teaches A method, comprising: performing an output channel (OC) reordering operation upon multiple OCs of a convolutional layer of a neural network, wherein the OC recording operation comprises: identifying a number of non-zero weights of each OC among the multiple OCs; sorting the multiple OCs according to the number of non-zero weights of said each OC among the multiple OCs,
(Wang, “[0033] FIG. 4 illustrates skipped weights in FC computations according to one embodiment. Referring to FIG. 4, the buffer loader 140 reads an input activation 410 which includes multiple input channels (e.g., C1, C2, C3 and C4). The data in each input channel is to be multiplied by corresponding weights (e.g., a corresponding column of the two-dimensional weights 420). In this example, after reading the input activation 410, the buffer loader 140 identifies that the data in the input channels (e.g., C1 and C4) are zeros (labeled in FIG. 4 as “Z”), and marks the corresponding weights (e.g., the two columns W1 and W4) as “skipped weights” (labeled as “S”) without loading W1 and W4. In this example, the data in the input channels C2 and C3 are non-zeros (labeled as “N”) [identifying a number of non-zero weights of each OC among the multiple OCs], so the buffer loader 140 loads the corresponding weights W2 and W3 from the DRAM 150 into the buffer 130. Thus, the buffer loader 140 skips reading (and loading) the weights W1 and W4. Skipping the read access to W1 and W4 reduces memory bus traffic.”
PNG
media_image1.png
472
607
media_image1.png
Greyscale
)
(Wang, “[0034] After weights W2 and W3 are loaded into the buffer 130, the dispatcher 120 identifies zero weights (labeled in FIG. 4 as “Z”) and non-zero weights (labeled as “N”) in W2 and W3. The dispatcher 120 is to skip dispatching the zero weights to the PEs 110. The dispatcher 120 dispatches the non-zero weights in W2 and W3, together with the input data in the corresponding input channels C2 and C3 to the PEs 110 for MAC operations [sorting the multiple OCs according to the number of non-zero weights of said each OC among the multiple OCs]. By skipping the MAC operations for zero weights that are loaded into the buffer 130, the workload of PEs 110 can be reduced….”)
Wang teaches in order to generate a sorted result, wherein the sorted result orders the multiple OCs according to the numbers of non-zero weights [from a highest workload to a lowest workload]; and grouping the multiple OCs to multiple processing element (PE) pairs according to the sorted result,
(Wang, “[0036] The method 500 begins with the accelerator at step 510 groups processing elements (PEs) into PE groups [grouping the multiple OCs to multiple processing element (PE) pairs according to the sorted result; wherein a PE group can merely be two PEs]. Each PE group is to perform CNN computations by applying multi-dimensional weights on an input activation. The accelerator includes a dispatcher which, at step 520, dispatches input data in the input activation and non-zero weights in the multi-dimensional weights to the PE groups according to a control mask. The control mask specifies positions of zero weights in the multi-dimensional weights. The PE groups share the same control mask specifying the same positions of the zero weights. The PE groups at step 530 generate output data of respective output channels in an output activation [in order to generate a sorted result, wherein the sorted result orders the multiple OCs according to the numbers of non-zero weights…; and grouping the multiple OCs to multiple processing element (PE) pairs according to the sorted result; wherein the weights were dispatched according to non-zero weights by the dispatcher to the PE groups].”)
However, Wang does not explicitly teach receiving, at a memory in a neural network processor, first weights and second weights of the convolutional layer of the neural network, the first weights associated with a first OC of the convolutional layer, the second weights associated with a second OC of the convolutional layer, [wherein a first PE pair comprises a first PE and a second PE, and the first OC and the second OC are grouped to the first PE pair according to the sorted result;] computing, by the first PE in the neural network processor, a partial sum (PSUM) of an output activation of the first OC based on non-zero weights in the first weights; computing, by the second PE in the neural network processor, a PSUM of an output activation of the second OC based on non-zero weights in the second weights; and allocating, by a controller in the neural network processor, one or more non-zero weights of the first weights to the second PE for computing the PSUM of the output activation of the first OC to balance a workload between the first PE and the second PE when a first number of the non-zero weights in the first weights is larger than a second number of the non-zero weights in the second weights.
Guan teaches receiving, at a memory in a neural network processor, first weights and second weights of the convolutional layer of the neural network,
(Guan, Section 4.1, “The overall architecture of Crane is depicted in Fig. 7. The whole accelerator employs a DRAM as the external memory, and the DMA is responsible for data movement operations between on-chip RAMs and off-chip DRAM. Processing Unit consists of N PEs to perform CNN inference in parallel with the Input Sharing data-flow. The weights for convolution are stored in Weight RAM [receiving, at a memory in a neural network processor, first weights and second weights of the convolutional layer of the neural network], and input activation patches are broadcast into input buffers residing in each PE. Both input activation and weight matrices are stored in an encoded format. As a result, only non-zero activation data and weights can be fetched by DMA and stored on-chip for convolution processing. Output RAM stores the results generated by all PEs, and transfers these results to Output Unit for after-convolution processing, which includes activation function, pooling, and encoding. After that, these output results are written back to DRAM.
PNG
media_image2.png
234
392
media_image2.png
Greyscale
”)
Guan teaches the first weights associated with a first OC of the convolutional layer, the second weights associated with a second OC of the convolutional layer;
(Guan, Section 2.1, Fig. 2b, Weight Sparsity. We call the fraction of zero-values in weight matrices as weight sparsity. Weight sparsity mainly comes from weight pruning techniques proposed in prior works [7], [8]. During weight pruning, if the absolute values of weights are lower than a threshold, these weights are pruned (i.e., set to zeros) from the convolutional layer. Then, the whole CNN gets re-trained to recover accuracy. Weight sparsity is demonstrated in Fig. 2b [the first weights associated with a OC of the convolutional layer, the second weights associated with a second OC of the convolutional layer; see fig. 2b for first and second output channel (Out1 and Out2) with corresponding sets of weights associated].
PNG
media_image3.png
231
184
media_image3.png
Greyscale
”)
Guan teaches computing, by the first PE in the neural network processor, a partial sum (PSUM) of an output activation of the first OC based on non-zero weights in the first weights; computing, by the second PE in the neural network processor, a PSUM of an output activation of the second OC based on non-zero weights in the second weights;
(Guan, Section 3.2, “Fig. 5 illustrates the Weight Sharing data-flow. It broadcasts weights to all of the PEs during CNN inference. As shown in Fig. 5, each PE generates the results for a tile of output activation data (Rout/N×Cout in each channel). Thus, the input activation matrices are partitioned for parallel processing among PEs. For each round, CHin weight kernels (K×K each) are broadcast to all PEs. Each PE is responsible for generating Rout/N×Cout output data [computing, by the first/second processing element (PE) in the neural network processor; see Fig. 5 for first and second PE (PE_1… PE_N); see also fig. 6 for support]. Once the current set of weight kernels is processed, the next set corresponding to the following output channel is broadcast. The arrows show the order of weight set fetching and output generating.
PNG
media_image4.png
147
392
media_image4.png
Greyscale
Due to the sliding style of a convolution window, Weight Sharing data-flow has to duplicate some input activation data or exchange partial sums [a partial sum (PSUM) of an output activation of the first/second OC based on non-zero weights in the first/second weights; wherein the computing based on non-zero weights can be given by applying the weight sparsity in (Guan, Section 2.1)] across adjacent output tiles to deal with data dependency at the edges of tiles (also called “halos” [15], [31]).”)
Guan teaches and allocating, by a controller in the neural network processor, one or more non-zero weights of the first weights to the second PE for computing the PSUM of the output activation of the first OC to balance a workload between the first PE and the second PE when a first number of the non-zero weights in the first weights is larger than a second number of the non-zero weights in the second weights.
(Guan, Section 3.1, “To address the under-utilization problem, we employ a dynamic load-balancing scheme, which is similar to the work stealing strategy [30] applied to schedule multi-thread computer programs. When a PE completes its own workloads, it can “steal” weight kernels from another PE to help [when a first number of the non-zero weights in the first weights is larger than a second number of the non-zero weights in the second weights].
As illustrated in Fig. 4b, when PE1 completes results for the two output channels assigned to it, PE2 is still busy working on output channel 3. As a result, PE1 can fetch the weight kernels corresponding to output channel 4. In this way, PE1 “steals” a portion of workloads from PE2, and generates the results for output channel 4 [allocating,… , one or more non-zero weights of the first weights to the second PE for computing the PSUM of the output activation of the first OC to balance a workload between the first PE and the second PE]. By applying this dynamic load-balancing scheme, it takes only two cycles to finish the processing of this input patch. As shown in Fig. 4b, both PE1 and PE2 work at full utilization, and the overall performance is improved by 50 percent.”)
(Guan, Section 4.1, “The key modules of Crane, which enable load-balancing, are Workload Scheduler, Weight RAM Controller, and Output RAM Controller. Workload Scheduler monitors the run-time processing status of each PE, and produces schedule information to Weight RAM Controller and Output RAM Controller. These two controllers use the schedule information to control the weight fetching and output storing, respectively. These two control modules manage the workload stealing across different PEs [by a controller in the neural network processor]. Processing Controller is the global controller to generate detailed control signals for other modules.”)
Wang and Guan are both considered to be analogous to the claimed invention because they are in the same field of neural network acceleration. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang to incorporate the teachings of Guan in order to provide a load-balancing method in order to address sparsity irregularities, improve model performance and reduce energy consumption (Guan, Abstract, “Convolutional neural networks (CNNs) have achieved great success in numerous AI applications. To improve inference efficiency of CNNs, researchers have proposed various pruning techniques to reduce both computation intensity and storage overhead. These pruning techniques result in multi-level sparsity irregularities in CNNs. Together with that in activation matrices, which is induced by employment of ReLU activation function, all these sparsity irregularities cause a serious problem of computation resource under-utilization in sparse CNN accelerators. To mitigate this problem, we propose a method of load-balancing based on a workload stealing technique. We demonstrate that this method can be applied to two major inference data-flows, which cover all state-of-the-art sparse CNN accelerators. Based on this method, we present an accelerator, called Crane, which addresses all kinds of sparsity irregularities in CNNs. We perform a fair comparison between Crane and state-of-the-art prior approaches. Experimental results show that Crane improves performance by 27% ~ 88% and reduces energy consumption by 16% ~ 48%, respectively, compared to the counterparts.”)
However, Wang and Guan does not explicitly teach [wherein the sorted result orders the multiple OCs according to the numbers of non-zero weights] from a highest workload to a lowest workload… wherein grouping the multiple OCs to the multiple PE pairs according to the sorted result comprises: grouping a highest-workload OC and a lowest-workload OC into a same PE pair; grouping a second-highest-workload OC and a second-lowest-workload OC into another PE pair; and iteratively grouping remaining OCs according to the sorted result;
Sampathkumar teaches from a highest workload to a lowest workload… wherein grouping the multiple OCs to the multiple PE pairs according to the sorted result comprises: grouping a highest-workload OC and a lowest-workload OC into a same PE pair; grouping a second-highest-workload OC and a second-lowest-workload OC into another PE pair; and iteratively grouping remaining OCs according to the sorted result;
(Sampathkumar, “[0030] FIGS. 5A and 5B illustrate example resource utilization graphs for a plurality of hosts. Resource utilization graph 500A illustrates a scenario in which the resource utilization difference between host 1 (e.g., the host machine with the highest resource utilization) and host 2 (e.g., the host machine with the lowest resource utilization) [from a highest workload to a lowest workload… grouping a highest-workload OC and a lowest-workload OC into a same PE pair; grouping a second-highest-workload OC and a second-lowest-workload OC into another PE pair; and iteratively grouping remaining OCs according to the sorted result] exceeds a difference threshold. As illustrated, workloads 501, 502, 503, and 504 are executing on host 1; workloads 505, 506, and 507 are executing on host 2; and workloads 508, 509, and 510 are executing on host 3. [0031] According to embodiments, resource scheduler 110 determines that the resource utilization difference between hosts 1 and 2 exceeds the threshold difference. Based on the determination that the resource utilization difference between hosts 1 and 2 exceeds the threshold difference, resource scheduler 110 examines resource utilization for each workload executing on host 1 to identify candidate workloads for migration to host 2. As illustrated in post-migration resource utilization graph 500B, resource scheduler 100 identified workload 504 as the workload to be migrated from host 1 to host 2. The post-migration resource utilization difference between host 1 and host 2, as illustrated, is less than the threshold difference. It should be recognized that resource scheduler 110 did not select workload 502 for migration from host 1 to host 2 because such a migration would have caused host 2 to be overloaded and host 1 to be underloaded to an extent the resource utilization difference between hosts 1 and 2 would have exceeded the threshold difference.”
PNG
media_image5.png
629
426
media_image5.png
Greyscale
)
Sampathkumar is considered to be analogous to the claimed invention because they are in the same field of load balancing with pairwise comparison. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang and Guan to incorporate the teachings of Sampathkumar in order to provide a load balancing technique that is able to identify load balancing opportunities with outliers (highest and lowest workloads) (Sampathkumar, “[0003] Resource schedulers generally use standard deviation of resource utilization among multiple hosts in the cluster as a trigger for performing load balancing. In some cases, however, the standard deviation approach may not be sufficient to identify all of the load balancing opportunities. For example, where there are a small number of outliers in the cluster (e.g., hosts having a very high resource utilization relative to the average), load balancing opportunities for such outliers may be missed because the standard deviation may still be below the threshold required to trigger load balancing. As a result, the high resource utilization state in the outliers will continue to exist so long as the standard deviation is below the threshold.”)
In regards to claim 2,
Wang and Guan and Sampathkumar teaches The method of claim 1,
Guan teaches wherein the first weights and the second weights correspond to a same set of input channels (ICs) of the convolutional layer.
(Guan, Section 2.1, Fig. 2b, Weight Sparsity. We call the fraction of zero-values in weight matrices as weight sparsity. Weight sparsity mainly comes from weight pruning techniques proposed in prior works [7], [8]. During weight pruning, if the absolute values of weights are lower than a threshold, these weights are pruned (i.e., set to zeros) from the convolutional layer. Then, the whole CNN gets re-trained to recover accuracy. Weight sparsity is demonstrated in Fig. 2b [wherein the first weights and the second weights correspond to a same set of input channels (ICs) of the convolutional layer; see fig. 2b for first and second input channel (In1 and In2) with corresponding sets of weights associated].
PNG
media_image3.png
231
184
media_image3.png
Greyscale
”)
In regards to claim 3,
Wang and Guan and Sampathkumar teaches The method of claim 1,
Guan teaches further comprising: determining, by the controller in the neural network processor, whether a workload imbalance exists between the first PE and the second PE based on the first number of the non-zero weights in the first weights and the second number of the non-zero weights in the second weights.
(Guan, Section 3.1, “To address the under-utilization problem, we employ a dynamic load-balancing scheme, which is similar to the work stealing strategy [30] applied to schedule multi-thread computer programs. When a PE completes its own workloads, it can “steal” weight kernels from another PE to help [determining, …, whether a workload imbalance exists between the first PE and the second PE based on the first number of the non-zero weights in the first weights and the second number of the non-zero weights in the second weights].
As illustrated in Fig. 4b, when PE1 completes results for the two output channels assigned to it, PE2 is still busy working on output channel 3. As a result, PE1 can fetch the weight kernels corresponding to output channel 4. In this way, PE1 “steals” a portion of workloads from PE2, and generates the results for output channel 4. By applying this dynamic load-balancing scheme, it takes only two cycles to finish the processing of this input patch. As shown in Fig. 4b, both PE1 and PE2 work at full utilization, and the overall performance is improved by 50 percent.”)
(Guan, Section 4.1, “The key modules of Crane, which enable load-balancing, are Workload Scheduler, Weight RAM Controller, and Output RAM Controller. Workload Scheduler monitors the run-time processing status of each PE, and produces schedule information to Weight RAM Controller and Output RAM Controller. These two controllers use the schedule information to control the weight fetching and output storing, respectively. These two control modules manage the workload stealing across different PEs [by the controller in the neural network processor]. Processing Controller is the global controller to generate detailed control signals for other modules.”)
In regards to claim 5,
Wang and Guan and Sampathkumar teaches The method of claim 1,
Guan teaches wherein a workload of every two sets of weights in the memory corresponding to every two neighboring active OCs in a sequence of OCs of the convolutional layer is balanced by the controller in the neural network processor between a pair of PEs in the neural network processor according to an amount of non-zero weights in each set of the weights.
(Guan, Section 3.1, “To address the under-utilization problem, we employ a dynamic load-balancing scheme, which is similar to the work stealing strategy [30] applied to schedule multi-thread computer programs. When a PE completes its own workloads, it can “steal” weight kernels from another PE to help [wherein a workload of every two sets of weights in the memory corresponding to every two neighboring active OCs in a sequence of OCs of the convolutional layer is balanced … between a pair of PEs in the neural network processor according to an amount of non-zero weights in each set of the weights].
As illustrated in Fig. 4b, when PE1 completes results for the two output channels assigned to it, PE2 is still busy working on output channel 3. As a result, PE1 can fetch the weight kernels corresponding to output channel 4. In this way, PE1 “steals” a portion of workloads from PE2, and generates the results for output channel 4. By applying this dynamic load-balancing scheme, it takes only two cycles to finish the processing of this input patch. As shown in Fig. 4b, both PE1 and PE2 work at full utilization, and the overall performance is improved by 50 percent.”)
(Guan, Section 4.1, “The key modules of Crane, which enable load-balancing, are Workload Scheduler, Weight RAM Controller, and Output RAM Controller. Workload Scheduler monitors the run-time processing status of each PE, and produces schedule information to Weight RAM Controller and Output RAM Controller. These two controllers use the schedule information to control the weight fetching and output storing, respectively. These two control modules manage the workload stealing across different PEs [by the controller in the neural network processor]. Processing Controller is the global controller to generate detailed control signals for other modules.”)
In regards to claim 6,
Wang and Guan and Sampathkumar teaches The method of claim 1,
Guan teaches wherein the first OC and the second OC are not adjacent to each other in a sequence of OCs of the convolutional layer.
(Guan, Section 3.2, “The idea for load-balancing is similar. In Fig. 6b, PE1 finishes generating the output results assigned to it after Cycle 1, while PE2 and PE3 are still working on their own workloads. Since PE3 has the most workloads left after Cycle 1, PE1 can fetch the third input patch of Tile3 to generate corresponding output data [wherein the first OC and the second OC are not adjacent to each other in a sequence of OCs of the convolutional layer; wherein Out1-Tile1 is not adjacent to Out1-Tile3]. Fig. 6b shows that the workloads of processing the third element in Tile3 are “stolen” by PE1, and workloads are balanced among PEs.”
PNG
media_image6.png
272
557
media_image6.png
Greyscale
)
In regards to claim 7,
Wang and Guan and Sampathkumar teaches The method of claim 1,
Guan teaches further comprising: balancing a workload of P sets of weights corresponding to P number of OCs of the convolutional layer in the neural network among P number of PEs, P being a number larger than 2.
(Guan, Section 3.1, “As shown in Fig. 4a, PE1 is responsible for processing output channel 1 and 2, and PE2 is responsible for output channel 3 and 4 [P number of OCs of the convolutional layer in the neural network among P number of PEs; wherein Fig. 4 shows 2 sets of weights broadcasted to 2 PE wherein each PE corresponds to 2 output channels]. The first input patch is fetched and broadcast to two PEs. According to the activation sparsity and weight sparsity, the effectual workloads (non-zero computation) and computing process of each PE are shown in the table in Fig. 4a. It takes one cycle and three cycles for PE1 and PE2 to process this input patch, respectively. Thus, PE1 is idle for two cycles to wait for PE2.
To address the under-utilization problem, we employ a dynamic load-balancing scheme [balancing a workload of P sets of weights; wherein balancing a workload is performed by the dynamic load-balancing scheme that moves the P sets of weights], which is similar to the work stealing strategy [30] applied to schedule multi-thread computer programs. When a PE completes its own workloads, it can “steal” weight kernels from another PE to help.
As illustrated in Fig. 4b, when PE1 completes results for the two output channels assigned to it, PE2 is still busy working on output channel 3. As a result, PE1 can fetch the weight kernels corresponding to output channel 4. In this way, PE1 “steals” a portion of workloads from PE2, and generates the results for output channel 4. By applying this dynamic load-balancing scheme, it takes only two cycles to finish the processing of this input patch. As shown in Fig. 4b, both PE1 and PE2 work at full utilization, and the overall performance is improved by 50 percent.”
PNG
media_image7.png
411
413
media_image7.png
Greyscale
)
In regards to claim 8,
Wang and Guan and Sampathkumar teaches The method of claim 1,
Guan teaches wherein zero-weights among the first weights and the second weights are skipped for processing during the computing by the first PE and the second PE.
(Guan, Section 4.2, “The architecture of a PE in Crane is illustrated in Fig. 8. It consists of an Input Buffer, Weight Regs, an Index Selector, parallel multipliers and an adder tree. Input buffer stores the input activation patch broadcast to all PEs, including both non-zero values and indices. Non-zero weights and their indices are fetched from Weight RAM to Weight Regs. Each non-zero activation or weight is paired with an index that indicates the location of this value in the original format.
PNG
media_image8.png
237
401
media_image8.png
Greyscale
To improve performance and save energy, PEs need to skip ineffectual multiplications. Index Selector leverages input indices and weight indices to find the indices of effectual multiplications (both input activation and weight are non-zero values) [wherein zero-weights among the first weights and the second weights are skipped for processing during the computing by the first PE and the second PE]. The selected indices are used to select non-zero “activation-weight” pairs from Input Buffer and Weight Regs. All the effectual “activation-weight” pairs are fed into the multipliers to perform multiplications in parallel. The products are summed up together by an adder tree to minimize latency. The accumulation result is fed into the last-level adder or Output RAM depending on whether the computation for current output has finished or not. Since Index Selector needs to traverse indices of input data during index selection, it can easily judge if current accumulation result should be fed into Output RAM.”)
Claim(s) 4 is rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Guan in further view of Sampathkumar in further view of I. -C. Wu, P. -T. Huang, C. -Y. Lo and W. Hwang, "An Energy-Efficient Accelerator with Relative- Indexing Memory for Sparse Compressed Convolutional Neural Network," 2019 IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), Hsinchu, Taiwan, 2019, pp. 42-45. (“Wu”)
In regards to claim 4,
Wang and Guan and Sampathkumar teaches The method of claim 1,
Wu teaches wherein the first weights and the second weights are in a compressed form.
(Wu, “Deep Compression [2] is utilized to reduce the required memory capacity and bandwidth by compressing weight [wherein the first weights and the second weights are in a compressed form], and composed of three stages, including pruning, quantization and Huffman coding. First, pruning is widely used in neural networks to reduce model size and avoid over-fitting [1] . Removing small-value weights is the most important step to reduce a large amount of connections and partial neurons. The weights below a threshold value are set to zero, and then the remaining connections would be retrained to compensate for the accuracy. Second, each weight is represented in fewer bits to be further compressed by quantization. Weights are shared by k-means clustering, and then retrain the model. The final step is Huffman coding, which is a kind of optimal prefix code and widely used for lossless data compression. The Huffman coding is utilized to reduce the model size since the non-uniform distribution.”)
Wu is considered to be analogous to the claimed invention because they are in the same field of accelerating sparse convolutional neural networks. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Wang and Guan and Sampathkumar to incorporate the teachings of Wu in order to provide weight compression to reduce required memory capacity/bandwidth (Wu, Abstract, “Deep convolutional neural networks (CNNs) are widely used in image recognition and feature classification. However, deep CNNs are hard to be fully deployed for edge devices due to both computation-intensive and memory-intensive workloads. The energy efficiency of CNNs is dominated by off-chip memory accesses and convolution computation. In this paper, an energy-efficient accelerator is proposed for sparse compressed CNNs by reducing DRAM accesses and eliminating zero-operand computation. Weight compression is utilized for sparse compressed CNNs to reduce the required memory capacity/bandwidth and a large portion of connections. Thus, ReLU function produces zero-valued activations. Additionally, the workloads are distributed based on channels to increase the degree of task parallelism, and all-row- to-all-row non-zero element multiplication is adopted for skipping redundant computation. The simulation results over the dense accelerator show that the proposed accelerator achieves 1.79x speedup and reduces 23.51%, 69.53%, 88.67% on-chip memory size, energy, and DRAM accesses of VGG-16.”)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US Pub No. US20190205358A1 Meta teaches Sparsity-aware hardware accelerators
US Pub No. US20220261649A1 Gao et al. teaches balancing workloads related to the operation by reordering the compressed matrix based on the workloads
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASMINE THAI whose telephone number is (703)756-5904. The examiner can normally be reached M-F 8-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Michael Huntley can be reached at (303) 297-4307. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.T.T./Examiner, Art Unit 2129
/MICHAEL J HUNTLEY/Supervisory Patent Examiner, Art Unit 2129
Prosecution Insights