DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to the Application filed on 03/28/2023. Claims 1-20 are pending in the case. Claims 1, 10, and 19 are independent claims.
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant are advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. § 103 as being unpatentable over Mills et al. (U.S. Pat. App. Pub. No. 2019/0340486, hereinafter Mills) in view of Goo et al. (U.S. Pat. App. Pub. No. 2023/0401420, hereinafter Goo) and Mathaikutty et al. (U.S. Pat. App. Pub. No. 2023/0059976, hereinafter Mathaikutty).
As to independent claim 1, Mills teaches
A neural processor circuit, comprising (Figure 3, neural processor circuit 218):
a kernel interface circuit configured to receive kernel data from a source external to the neural processor circuit, the kernel data including kernel coefficients (Figure 4, kernel extract 432. Paragraph 58, "Kernel extract circuit 432 is a circuit that receives kernel data 326 from kernel DMA 324 and extracts kernel coefficients 422");
a data interface circuit configured to receive input data and… bias from the source, (Figure 3, buffer DMA 320. Paragraph 54, "Buffer DMA 320 includes a read circuit that receives a portion (e.g., tile) of the input data from a source (e.g., system memory 230) for storing in data buffer 318." Paragraph 109, "accumulators 414 of neural engine 314 may be pre-loaded with one or more bias values"); and
a neural engine circuit configured to (Figure 4, neural engine 314):…
perform multiply-accumulate operations by using the input data, the adjusted kernel coefficients, and the adjusted bias to generate an accumulator value for generating an output of the neural processor circuit (Paragraph 60, "Accumulator 414 in combination with MAD circuits form a multiply-accumulator (MAC) 404." Paragraph 54, "Buffer DMA 320 includes a read circuit that receives a portion (e.g., tile) of the input data from a source (e.g., system memory 230) for storing in data buffer 318." Paragraph 58, "Kernel extract circuit 432 is a circuit that receives kernel data 326 from kernel DMA 324 and extracts kernel coefficients 422." Paragraph 109, "accumulators 414 of neural engine 314 may be pre-loaded with one or more bias values").
Mills does not appear to expressly teach adjusted bias and the adjusted bias representing a bias for a convolution operation as adjusted by at least a first adjustment value to account for asymmetric quantization of the input data.
Goo teaches adjusted bias and the adjusted bias representing a bias for a convolution operation as adjusted by at least a first adjustment value to account for asymmetric quantization of the input data (Paragraph 8, "The combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
Mills does not appear to expressly teach adjust the kernel coefficients to account for asymmetric quantization of the kernel coefficients to generate the adjusted kernel coefficients.
Mathaikutty teaches adjust the kernel coefficients to account for asymmetric quantization of the kernel coefficients to generate the adjusted kernel coefficients (Paragraph 23, "The quantization subtractors may subtract zeropoints from the quantized activations and quantized weights and generate intermediate activations and intermediate weights").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the zeropoint weight subtractor techniques of Mathaikutty to improve efficiency (see Mathaikutty at paragraph 2).
As to dependent claim 2, Goo further teaches the first adjustment value is determined during a compilation process (Paragraph 26, "The offline compiling is also referred to as ahead-of-time compiling, which is performed when the inference hardware 350 has not started executing the computer program").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 3, Goo further teaches the output is generated by applying an activation function to the accumulator value (Paragraph 24).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 4, Mills further teaches:
wherein the neural engine circuit comprises (Figure 3, neural processor circuit 218):
an input buffer circuit configured to buffer the input data (Figure 3, data buffer 318);
a kernel extract circuit configured to receive the kernel data and extract the kernel coefficients (Figure 4, kernel extract 432. Paragraph 58, "Kernel extract circuit 432 is a circuit that receives kernel data 326 from kernel DMA 324 and extracts kernel coefficients 422"); and
a multiply-accumulator circuit comprising (Figure 4, MAC 404):…
a plurality of multiply-add circuits configured to perform multiplication and add operations on the adjusted kernel coefficients and the input data to generate a processed value (Figure 4, MAD0 to MADN 412); and
an accumulator configured to perform an accumulation operation on the processed value applied with the… bias to generate the accumulator value (Figure 4, accumulator 414).
Mills does not appear to expressly teach adjusted bias.
Goo teaches adjusted bias (Paragraph 8, "The combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
Mills does not appear to expressly teach a kernel adjust circuit configured to generate the adjusted kernel coefficients by subtracting a kernel zero offset value from each of the kernel coefficients.
Mathaikutty teaches a kernel adjust circuit configured to generate the adjusted kernel coefficients by subtracting a kernel zero offset value from each of the kernel coefficients (Paragraph 23, "The quantization subtractors may subtract zeropoints from the quantized activations and quantized weights and generate intermediate activations and intermediate weights").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the zeropoint weight subtractor techniques of Mathaikutty to improve efficiency (see Mathaikutty at paragraph 2).
As to dependent claim 5, Mills does not appear to expressly teach the input buffer circuit is configured to process elements of the input data of a first bit size, the kernel extract circuit is configured to process each of the kernel coefficients in the first bit size, and the multiply- accumulator circuit is configured to process the elements of the input data and the kernel coefficients in a second bit size that is one bit longer than the first bit size.
Mathaikutty teaches the input buffer circuit is configured to process elements of the input data of a first bit size, the kernel extract circuit is configured to process each of the kernel coefficients in the first bit size, and the multiply- accumulator circuit is configured to process the elements of the input data and the kernel coefficients in a second bit size that is one bit longer than the first bit size (Paragraph 72).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the zeropoint weight subtractor techniques of Mathaikutty to improve efficiency (see Mathaikutty at paragraph 2).
As to dependent claim 6, Mills further teaches the neural engine circuit further comprises a post-processor configured to generate an adjusted version of the output by at least applying an activation function to the accumulator value (Figure 4, post-processor 428).
Mills does not appear to expressly teach adding an output zero offset value to each element of the output.
Goo teaches adding an output zero offset value to each element of the output (Paragraph 8, "The combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 7, Goo further teaches the output zero offset accounts for a non-linear component of the activation function but not a linear component of the activation function (Paragraph 37, "FIG. 5B shows that an asymmetric PReLU operation can be expressed as").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 8, Goo further teaches the bias for the convolution operation is further adjusted by a second adjustment value to account for the linear component of the activation function (Paragraph 39, "The compiler computes the constants BiasA and BiasB, each of which is a combined bias value").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 9, Mills further teaches the… bias is loaded onto the accumulator via the kernel extract circuit before performing the multiply-accumulate operations (Paragraph 92, "Neural engine 314 may further extract, from kernel extract circuit 432 loaded by kernel DMA 324, bias element 914 as a corresponding kernel coefficient 422").
Mills does not appear to expressly teach adjusted bias.
Goo teaches adjusted bias (Paragraph 8, "The combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to independent claim 10, Mills teaches
A method of operating a neural processor circuit, comprising (Title and abstract):…
receiving kernel data from the source, the kernel data including kernel coefficients (Figure 4, kernel extract 432. Paragraph 58, "Kernel extract circuit 432 is a circuit that receives kernel data 326 from kernel DMA 324 and extracts kernel coefficients 422");
receiving input data subsequent to receiving of the adjusted bias (Figure 3, buffer DMA 320. Paragraph 54, "Buffer DMA 320 includes a read circuit that receives a portion (e.g., tile) of the input data from a source (e.g., system memory 230) for storing in data buffer 318." Paragraph 109, "accumulators 414 of neural engine 314 may be pre-loaded with one or more bias values");…
performing multiply-accumulate operations by using the input data, the adjusted kernel coefficients, and the adjusted bias to generate an accumulator value (Paragraph 60, "Accumulator 414 in combination with MAD circuits form a multiply-accumulator (MAC) 404." Paragraph 54, "Buffer DMA 320 includes a read circuit that receives a portion (e.g., tile) of the input data from a source (e.g., system memory 230) for storing in data buffer 318." Paragraph 58, "Kernel extract circuit 432 is a circuit that receives kernel data 326 from kernel DMA 324 and extracts kernel coefficients 422." Paragraph 109, "accumulators 414 of neural engine 314 may be pre-loaded with one or more bias values"); and
processing the accumulator value to generate an output (Paragraph 5, "The at least one neural engine circuit performs multiplication between the matrix and the vector as a convolution operation to produce at least one output channel of the output data").
Mills does not appear to expressly teach receiving an adjusted bias from a source external to the neural processor circuit, the adjusted bias representing a bias for a convolution operation as adjusted by at least a first adjustment value to account for asymmetric quantization of input data.
Goo teaches receiving an adjusted bias from a source external to the neural processor circuit, the adjusted bias representing a bias for a convolution operation as adjusted by at least a first adjustment value to account for asymmetric quantization of input data (Paragraph 8, "The combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
Mills does not appear to expressly teach adjusting the kernel coefficients to account for asymmetric quantization of the kernel coefficients to generate the adjusted kernel coefficients.
Mathaikutty teaches adjusting the kernel coefficients to account for asymmetric quantization of the kernel coefficients to generate the adjusted kernel coefficients (Paragraph 23, "The quantization subtractors may subtract zeropoints from the quantized activations and quantized weights and generate intermediate activations and intermediate weights").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the zeropoint weight subtractor techniques of Mathaikutty to improve efficiency (see Mathaikutty at paragraph 2).
As to dependent claim 11, Goo further teaches determining the first adjustment value during a compilation process (Paragraph 26, "The offline compiling is also referred to as ahead-of-time compiling, which is performed when the inference hardware 350 has not started executing the computer program").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 12, Goo further teaches the output is generated by applying an activation function to the accumulator value (Paragraph 24).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 13, Mills further teaches:
buffering the input data (Figure 3, data buffer 318);
extracting the kernel coefficients from the kernel data (Figure 4, kernel extract 432. Paragraph 58, "Kernel extract circuit 432 is a circuit that receives kernel data 326 from kernel DMA 324 and extracts kernel coefficients 422");…
performing multiplication and adding operations on the adjusted kernel coefficients and the input data to generate a processed value (Figure 4, MAC 404); and
performing an accumulation operation on the processed value applied with the… bias to generate the accumulator value (Figure 4, accumulator 414).
Mills does not appear to expressly teach adjusted bias.
Goo teaches adjusted bias (Paragraph 8, "The combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
Mills does not appear to expressly teach generating the adjusted kernel coefficients by subtracting a kernel zero offset value from each of the kernel coefficients.
Mathaikutty teaches generating the adjusted kernel coefficients by subtracting a kernel zero offset value from each of the kernel coefficients (Paragraph 23, "The quantization subtractors may subtract zeropoints from the quantized activations and quantized weights and generate intermediate activations and intermediate weights").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the zeropoint weight subtractor techniques of Mathaikutty to improve efficiency (see Mathaikutty at paragraph 2).
As to dependent claim 14, Mills does not appear to expressly teach each element of the buffered input data is of a first bit size, each of the extracted kernel coefficients is of the first bit size, and the multiplication and adding operations are performed in a second bit size that is one bit longer than the first bit size.
Mathaikutty teaches each element of the buffered input data is of a first bit size, each of the extracted kernel coefficients is of the first bit size, and the multiplication and adding operations are performed in a second bit size that is one bit longer than the first bit size (Paragraph 72).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the zeropoint weight subtractor techniques of Mathaikutty to improve efficiency (see Mathaikutty at paragraph 2).
As to dependent claim 15, Mills further teaches generating an adjusted version of the output by at least applying an activation function to the accumulator value (Figure 4, post-processor 428).
Mills does not appear to expressly teach adding an output zero offset value to each element of the output.
Goo teaches adding an output zero offset value to each element of the output (Paragraph 8, "The combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 16, Goo further teaches the output zero offset accounts for a non-linear component of the activation function but not a linear component of the activation function (Paragraph 37, "FIG. 5B shows that an asymmetric PReLU operation can be expressed as").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 17, Goo further teaches the bias for the convolution operation is further adjusted by a second adjustment value to account for the linear component of the activation function (Paragraph 39, "The compiler computes the constants BiasA and BiasB, each of which is a combined bias value").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to dependent claim 18, Mills further teaches loading the… bias onto an accumulator of the neural processor circuit via a kernel extract circuit before performing the multiply-accumulate operations (Paragraph 92, "Neural engine 314 may further extract, from kernel extract circuit 432 loaded by kernel DMA 324, bias element 914 as a corresponding kernel coefficient 422").
Mills does not appear to expressly teach adjusted bias.
Goo teaches adjusted bias (Paragraph 8, "The combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
As to independent claim 19, Mills teaches
A neural engine circuit comprising (Figure 4, neural engine 314):
an input buffer circuit configured to buffer input data (Figure 3, buffer DMA 320);
a kernel extract circuit configured to receive kernel data and extract kernel coefficients from the kernel data (Figure 4, kernel extract 432. Paragraph 58, "Kernel extract circuit 432 is a circuit that receives kernel data 326 from kernel DMA 324 and extracts kernel coefficients 422"); and
a multiply-accumulator circuit comprising (Figure 4, MAC 404):…
a plurality of multiply-add circuits configured to perform multiplication and add operations on the adjusted kernel coefficients and the input data to generate a processed value (Figure 4, MAD0 to MADN 412); and
an accumulator configured to (Figure 4, accumulator 414):…
perform an accumulation operation using the adjusted bias to generate an accumulator value (Paragraph 60, "Accumulator 414 in combination with MAD circuits form a multiply-accumulator (MAC) 404." Paragraph 54, "Buffer DMA 320 includes a read circuit that receives a portion (e.g., tile) of the input data from a source (e.g., system memory 230) for storing in data buffer 318." Paragraph 58, "Kernel extract circuit 432 is a circuit that receives kernel data 326 from kernel DMA 324 and extracts kernel coefficients 422." Paragraph 109, "accumulators 414 of neural engine 314 may be pre-loaded with one or more bias values").
Mills does not appear to expressly teach store an adjusted bias representing a bias for a convolution operation as adjusted by at least a first adjustment value to account for asymmetric quantization of the input data.
Goo teaches store an adjusted bias representing a bias for a convolution operation as adjusted by at least a first adjustment value to account for asymmetric quantization of the input data (Paragraph 8, "The combined bias value is a constant computed by the compiler by merging at least zero points of input and output of the given asymmetric operation").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
Mills does not appear to expressly teach a kernel adjust circuit configured to adjust the kernel coefficients to account for asymmetric quantization of the kernel coefficients to generate the adjusted kernel coefficients.
Mathaikutty teaches a kernel adjust circuit configured to adjust the kernel coefficients to account for asymmetric quantization of the kernel coefficients to generate the adjusted kernel coefficients (Paragraph 23, "The quantization subtractors may subtract zeropoints from the quantized activations and quantized weights and generate intermediate activations and intermediate weights").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the zeropoint weight subtractor techniques of Mathaikutty to improve efficiency (see Mathaikutty at paragraph 2).
As to dependent claim 20, Goo further teaches he first adjustment value is determined during a compilation process (Paragraph 26, "The offline compiling is also referred to as ahead-of-time compiling, which is performed when the inference hardware 350 has not started executing the computer program").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the neural processor of Mills to include the combined bias techniques of Goo for better computational accuracy and hardware design (see Goo at paragraph 7).
Citation of Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Feddo et al. (U.S. Pat. App. Pub. No. 2021/0279557) teaches an electronic device that includes a neural processor having multiple neural engine circuits that operate in multiple modes of different bit width. A neural engine circuit may include a first multiply circuit and a second multiply circuit. The first and second multiply circuits may be combined to work as a part of a combined computation circuit. In a first mode, the first multiply circuit generates first output data of a first bit width by multiplying first input data with a first kernel coefficient. The second multiply circuit generates second output data of the first bit width by multiplying second input data with a second kernel coefficient. In a second mode, the combined computation circuit generates third output data of a second bit width by multiplying third input data with a third kernel coefficient..
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Casey R. Garner whose telephone number is 571-272-2467. The examiner can normally be reached Monday to Friday, 8am to 5pm, Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/Casey R. Garner/Primary Examiner, Art Unit 2123
Prosecution Insights