DETAILED ACTION
Claims 1-20 are pending.
The office acknowledges the following papers:
IDS filed on 9/2/2025.
Allowable Subject Matter
Claims 6 and 19-20 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
As allowable subject matter has been indicated, applicant's reply must either comply with all formal requirements or specifically traverse each requirement not complied with. See 37 CFR 1.111(b) and MPEP § 707.07(a).
Priority
The effective filing date for the subject matter defined in the pending claims in this application is 8/5/2024.
Drawings
The Examiner contends that the drawings submitted on 03/27/2025 are acceptable for examination proceedings.
Specification
The disclosure is objected to because of the following informalities:
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. The Applicant’s cooperation is requested in correcting any errors of which the Applicant may become aware.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4 and 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Ruan (U.S. 2024/0412051), in view of Henry et al. (U.S. 2018/0165575).
As per claim 1:
Ruan disclosed an artificial intelligence (AI) accelerator comprising:
a vector processing circuit (VPC) (Ruan: Figure 8 element 800) configured to, according to commands, perform a rearrangement on input data into sub-blocks (Ruan: Figure 8 elements 821 and 853, paragraphs 88-89)(The activation buffer fetches input tensors and performs a transpose operation on the fetched data (i.e. rearranges the input data). Additionally, the activation buffer divides the input tensor into subunits for fetching at different addresses.) and configured to perform vector processing for a convolution computation (Ruan: Figure 8 element 861, paragraph 94 and 97-98)(The dense tensor compute circuit performs a dot product calculation (i.e. convolution) on the input matrices.).
Ruan failed to teach a vector processing circuit (VPC) configured to configured to generate addresses for the rearranged sub-blocks.
However, Henry combined with Ruan disclosed a vector processing circuit (VPC) configured to configured to generate addresses for the rearranged sub-blocks (Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 elements 851-853, paragraphs 88-90)(Henry disclosed a sequencer to generate memory addresses to input and weight RAMs for reading/writing purposes. The combination implements the sequencer within Ruan to generate memory addresses on stored data to be processed by the dense tensor compute circuit.).
Ruan disclosed an activation buffer that divides input data into subunits that are fetched from different memory addresses, but doesn’t disclose generating memory addresses to fetch input data and weight data from their corresponding buffers. One of ordinary skill in the art would have been motivated by this lack of teaching in Ruan to find the Henry reference that generates memory addresses to input and weight RAMs. Thus, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date to implement the sequencer of Henry within the hardware accelerator of Ruan to generate memory addresses to fetch data for processing by the dense tensor compute circuit.
As per claim 2:
Ruan and Henry disclosed the Al accelerator of claim 1, wherein the commands comprise at least one of a first command to perform the rearrangement on the input data into the sub-blocks (Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 elements 821 and 853, paragraphs 88-89)(Ruan disclosed the activation buffer fetches input tensors and performs a transpose operation on the fetched data (i.e. rearranges the input data). Additionally, Ruan disclosed the activation buffer divides the input tensor into subunits for fetching at different addresses. Henry disclosed a sequencer that fetches NNU instruction from program memory. The combination implements the transpose operation and divide operation as program instructions fetched from memory.) and a second command to perform the convolution computation on the rearranged sub-blocks (Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 element 861, paragraph 94 and 97-98)(Ruan disclosed the dense tensor compute circuit performs a dot product calculation (i.e. convolution) on the input matrices. Henry disclosed a sequencer that fetches NNU instruction from program memory. The combination implements the dot product calculation as program instructions fetched from memory.).
As per claim 3:
Ruan and Henry disclosed the AI accelerator of claim 1, wherein the VPC comprises at least one of:
a command register configured to store a command for the vector processing transmitted from a central processing unit (CPU) core;
an address controller configured to generate an address to access a buffer in which the input data and weight data are stored (Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 elements 851-853, paragraphs 88-90)(Henry disclosed a sequencer to generate memory addresses to input and weight RAMs for reading/writing purposes. The combination implements the sequencer within Ruan to generate memory addresses on stored data to be processed by the dense tensor compute circuit.);
a controller configured to generate control signals to control the VPC by decoding the command stored in the command register;
an interconnect exchange (IX) buffer comprising a data buffer that stores the input data (Ruan: Figure 8 elements 853, paragraphs 88-89) and a weight buffer that stores the weight data (Ruan: Figure 8 element 851, paragraph 90);
a data aligner configured to select, from the input data and the weight data stored in the IX buffer, at least some input data and at least some weight data used for the convolution computation and configured to rearrange positions of the at least some input data and the at least some weight data according to computation units; and
a vector computation circuit comprising the computation units for real-time processing of a speech signal and configured to perform the convolution computation on the rearranged sub-blocks.
As per claim 4:
Ruan and Henry disclosed the AI accelerator of claim 3, wherein the data aligner comprises at least one of:
to perform the rearrangement on the at least some input data according to a first command or to compute two vector operands according to a second command, a first data aligner configured to perform the rearrangement on the at least some input data corresponding to a first vector operand among the two vector operands (Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 elements 821 and 853, paragraphs 88-89)(Ruan disclosed the activation buffer fetches input tensors and performs a transpose operation on the fetched data (i.e. rearranges the input data). Additionally, Ruan disclosed the activation buffer divides the input tensor into subunits for fetching at different addresses. Henry disclosed a sequencer that fetches NNU instruction from program memory. The combination implements the transpose operation and divide operation as program instructions fetched from memory.); and
to compute the two vector operands according to the second command, a second data aligner configured to perform the rearrangement on at least one of the at least some weight data and the at least some input data, which corresponds to a second vector operand among the two vector operands.
As per claim 7:
Ruan and Henry disclosed the AI accelerator of claim 3, wherein the computation units comprise at least one of:
128 16-bit floating-point multipliers;
128 32-bit floating-point adders to obtain a sum of outputs of the 128 16-bit floating-point multipliers;
an accumulator to obtain an accumulated sum of multiplication and accumulation (MAC) computation results (Ruan: Figure 8 element 861, paragraphs 94 and 97-98)(The dense tensor compute circuit includes an accumulator for accumulating the products.); and
128 rectified linear units (ReLUs) or 128 Leaky ReLUs.
As per claim 8:
Ruan and Henry disclosed the AI accelerator of claim 1, further comprising:
a floating-point calculating circuit configured to perform a high-precision computation used in executing an application program (Ruan: Figure 8 element 861, paragraph 98)(The dense tensor compute circuits performs MAC operations on floating-point inputs.).
Claims 5 and 9-17 are rejected under 35 U.S.C. 103 as being unpatentable over Ruan (U.S. 2024/0412051), in view of Henry et al. (U.S. 2018/0165575), in view of Official Notice.
As per claim 5:
Ruan and Henry disclosed the AI accelerator of claim 4, wherein the first data aligner comprises at least one of:
according to the second command that uses the two vector operands, a first input register configured to store second 128-word data among two pieces of 128-word data to rearrange the first vector operand according to inputs of the computation units (Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 elements 821 and 853, paragraphs 88-89)(Ruan disclosed the activation buffer fetches input tensors and performs a transpose operation on the fetched data (i.e. rearranges the input data). Additionally, Ruan disclosed the activation buffer divides the input tensor into subunits for fetching at different addresses. Henry disclosed a sequencer that fetches NNU instruction from program memory. The combination implements the transpose operation and divide operation as program instructions fetched from memory. Official notice is given that large transpose/shuffle operations can be implemented using large registers (e.g. 128-word) to for the advantage of temporarily storing the data prior to a transpose/shuffle operation. Thus, it would have been obvious to one of ordinary skill in the art to implement registers within the activation buffer to support transpose operations. In addition, according to “In re Rose” (105 USPQ 237 (CCPA 1955)), changes in size or range doesn’t give patentability over prior art.);
a second input register configured to store 128-word data to rearrange the at least some input data according to the first command, and according to the second command, configured to store first 128-word data among the two pieces of 128-word data to rearrange the first vector operand according to the inputs of the computation units;
for a mask generation, a mask generation circuit configured to generate first mask data used as a write enable write enable control signal in a word unit with respect to an output memory or second mask data;
a shifter configured to align pieces of data stored in the second input register according to the first command and configured to align pieces of data stored in each of the first input register and the second input register according to the second command;
a masking circuit configured to generate data used for a multiplication and accumulation (MAC) computation through masking between the pieces of data aligned by the shifter and the second mask data and configured to record 'O' in a position of data that is not used for the MAC computation; and
a mask register configured to store the first mask data.
As per claim 9:
Claim 9 essentially recites the same limitations of claim 2. Claim 9 additionally recites the following limitations:
a memory configured to store input data of an artificial neural network model for a convolution computation (Ruan: Figure 8 elements 807 and 811, paragraph 87);
a central processing unit (CPU) core configured to generate commands for the convolution computation (Ruan: Figure 8 element 800, paragraph 86)(Official notice is given that CPUs can fetch accelerator instructions and offload them to accelerators for increased performance. Thus, it would have been obvious to one of ordinary skill in the art to implement a CPU that offloads accelerator instructions to the hardware accelerator of Ruan.);
a negative AND (NAND) controller configured to communicate with an external memory that stores weight data of the artificial neural network model for the convolution computation (Ruan: Figure 8 elements 807 and 811, paragraph 87)(Official notice is given that memories can be implemented as NAND flash memories that are accessed by NAND memory controllers for the advantage of decreased costs and high storage density. Thus, it would have been obvious to one of ordinary skill in the art to implement the external memory as a NAND memory that is accessible via a NAND memory controller.).
As per claim 10:
Ruan and Henry disclosed the SoC of claim 9, wherein the memory is configured to further store at least one of information used by the CPU core to generate the commands for the AI accelerator and data to be transmitted to the AI accelerator (Ruan: Figure 8 elements 800 and 807, paragraph 86)(In view of the above official notice, the CPU fetches accelerator instructions from memory, including the external memory, and offloads them to the hardware accelerator of Ruan.).
As per claim 11:
Ruan and Henry disclosed the SoC of claim 9, wherein the CPU core is configured to generate and transmit the commands that perform vector processing for the convolution computation performed by the AI accelerator (Ruan: Figure 8 elements 800 and 807, paragraph 86)(In view of the above official notice, the CPU fetches accelerator instructions from memory, including the external memory, and offloads them to the hardware accelerator of Ruan.).
As per claim 12:
Ruan and Henry disclosed the SoC of claim 9, wherein the NAND controller is configured to read the weight data stored in the external memory and transmit the weight data to a weight buffer of the AI accelerator (Ruan: Figure 8 elements 807, 811, and 851-853, paragraphs 87-90)(In view of the above official notice, the external memory is implemented as a NAND flash memory accessed by a NAND memory controller. The added NAND memory controller transfers externally stored inputs and weights to the hardware accelerator for storage in the activation and weight buffers for processing.).
As per claim 13:
Ruan and Henry disclosed the SoC of claim 9, wherein the external memory comprises a non-volatile memory comprising a NAND flash memory, wherein the NAND flash memory is connected to the SoC and configured to store at least one of the weight data and an instruction of the artificial neural network model (Ruan: Figure 8 elements 807 and 811, paragraph 87)(In view of the above official notice, the external memory is implemented as a NAND flash memory (i.e. non-volatile memory) accessed by a NAND memory controller.).
As per claim 14:
The additional limitation(s) of claim 14 basically recite the additional limitation(s) of claim 3. Therefore, claim 14 is rejected for the same reason(s) as claim 3.
As per claim 15:
Claim 15 essentially recites the same limitations of claim 9. Claim 15 additionally recites the following limitations:
a system-on-chip (SoC) (Ruan: Figure 8 element 800, paragraph 86)(Official notice is given that CPUs can fetch accelerator instructions and offload them to accelerators on a single chip for increased performance. Thus, it would have been obvious to one of ordinary skill in the art to implement a CPU that offloads accelerator instructions to the hardware accelerator of Ruan. The combined CPU and accelerator represent a SoC.); and
a negative AND (NAND) flash memory connected to the SoC and configured to store weight data and an instruction of an artificial neural network model (Ruan: Figure 8 elements 807, 811, and 851-853, paragraphs 87-90)(Official notice is given that memories can be implemented as NAND flash memories that are accessed by NAND memory controllers for the advantage of decreased costs and high storage density. Thus, it would have been obvious to one of ordinary skill in the art to implement the external memory as a NAND memory that is accessible via a NAND memory controller. The added NAND memory controller transfers externally stored inputs and weights from the NAND flash memory to the hardware accelerator for storage in the activation and weight buffers for processing.).
As per claim 16:
Ruan and Henry disclosed an operating method of an artificial intelligence (AI) accelerator, the operating method comprising:
storing input data and weight data in a buffer (Ruan: Figure 8 elements 851-853, paragraphs 88-90);
storing a command transmitted from a central processing unit (CPU) core (Ruan: Figure 8 element 800, paragraph 86)(Official notice is given that CPUs can fetch accelerator instructions and offload them to accelerator instruction buffers for increased performance. Thus, it would have been obvious to one of ordinary skill in the art to implement a CPU that offloads accelerator instructions to the hardware accelerator of Ruan and stores them within an instruction buffer.);
generating an address to access the buffer in which the input data and the weight data are stored (Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 elements 851-853, paragraphs 88-90)(Henry disclosed a sequencer to generate memory addresses to input and weight RAMs for reading/writing purposes. The combination implements the sequencer within Ruan to generate memory addresses on stored data to be processed by the dense tensor compute circuit.);
generating control signals by decoding the command (Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 elements 851-853, paragraphs 88-90)(Henry disclosed a sequencer that generates control signals for processing NPU commands. The combination implements the sequencer within Ruan to generate control signals for hardware accelerator operations.);
selecting, from the input data and the weight data stored in the buffer, at least some input data and at least some weight data used for a convolution computation (Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 elements 851-853, paragraphs 88-90 and 94)(The combination implements the sequencer within Ruan to generate memory addresses on stored data to be processed by the dense tensor compute circuit. The generated memory addresses select stored data to be output and sent to the dense tensor compute circuit.); and
performing the convolution computation by rearranging positions of the selected at least some input data and the selected at least some weight data according to computation units (Ruan: Figure 8 element 861, paragraph 94 and 97-98)(The dense tensor compute circuit performs a dot product calculation (i.e. convolution) on the input matrices.).
Ruan disclosed an activation buffer that divides input data into subunits that are fetched from different memory addresses, but doesn’t disclose generating memory addresses to fetch input data and weight data from their corresponding buffers. One of ordinary skill in the art would have been motivated by this lack of teaching in Ruan to find the Henry reference that generates memory addresses to input and weight RAMs. Thus, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date to implement the sequencer of Henry within the hardware accelerator of Ruan to generate memory addresses to fetch data for processing by the dense tensor compute circuit.
As per claim 17:
Ruan and Henry disclosed the operating method of claim 16, wherein the command comprises at least one of a storage position of the input data, a storage position of the weight data, a type of the convolution computation (Ruan: Figure 8 element 861, paragraph 94 and 97-98)(The dense tensor compute circuit performs a dot product calculation (i.e. convolution) on the input matrices.), a length of data involved in the convolution computation, a stride interval for the convolution computation, and a dilation rate.
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over Ruan (U.S. 2024/0412051), in view of Henry et al. (U.S. 2018/0165575), further in view of Diamant (U.S. 2021/0173656).
As per claim 18:
Ruan and Henry disclosed the operating method of claim 16.
Ruan and Henry failed to teach wherein the performing of the convolution computation comprises: when pieces of information comprised in the command instruct performance of a dilated convolution computation, performing the dilated convolution computation; and when the pieces of information comprised in the command instruct performance of a transposed convolution computation, performing the transposed convolution computation.
However, Diamant combined with Ruan and Henry disclosed wherein the performing of the convolution computation comprises:
when pieces of information comprised in the command instruct performance of a dilated convolution computation, performing the dilated convolution computation (Diamant: Figure 3 element 304, paragraph 48)(Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 element 861, paragraph 94 and 97-98)(Ruan disclosed a dense tensor compute circuit performs a dot product calculation (i.e. convolution) on the input matrices. Henry disclosed a sequencer that fetches NNU instruction from program memory. Diamant disclosed a CNN performing multiple different types of convolution operations, including a dilated convolution operation. The combination allows for fetching and executing dilated convolution operations in Ruan.); and
when the pieces of information comprised in the command instruct performance of a transposed convolution computation, performing the transposed convolution computation (Diamant: Figure 3 element 304, paragraph 48)(Henry: Figure 1 elements 122-124 and 128, paragraph 86)(Ruan: Figure 8 element 861, paragraph 94 and 97-98)(Ruan disclosed a dense tensor compute circuit performs a dot product calculation (i.e. convolution) on the input matrices. Henry disclosed a sequencer that fetches NNU instruction from program memory. Diamant disclosed a CNN performing multiple different types of convolution operations, including a transposed convolution operation. The combination allows for fetching and executing transposed convolution operations in Ruan.).
The advantage of dilated convolution operations are that they provide a larger receptive field and preserve resolution compared to pooling or striding. The advantage of transposed convolution operations are that they are useful for upsampling, restoring spatial resolution after downsampling, and increase image resolution while learning upsampling filters. Thus, it would have been obvious to one of ordinary skill in the art at the time of the effective filing date to implement both dilated and transposed convolution operations in the combination for the above advantages.
Conclusion
The following is text cited from 37 CFR 1.111(c): In amending in reply to a rejection of claims in an application or patent under reexamination, the applicant or patent owner must clearly point out the patentable novelty which he or she thinks the claims present in view of the state of the art disclosed by the references cited or the objections made. The applicant or patent owner must also show how the amendments avoid such references or objections.
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
Mills (U.S. 2024/0320470), taught a neural engine input buffer with a shifter.
Li (U.S. 2023/0049323), taught a sparsity aware compute-in-memory.
Surti et al. (U.S. 2022/0129521), taught a matrix accelerator.
Park et al. (U.S. 2020/0167637), taught a neural network processor.
Talpes et al. (U.S. 11,893,393), taught a matrix computational array.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JACOB A. PETRANEK whose telephone number is (571)272-5988. The examiner can normally be reached on M-F 8:00-4:30.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jyoti Mehta can be reached on (571) 270-3995. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACOB PETRANEK/Primary Examiner, Art Unit 2183
Prosecution Insights