DETAILED ACTION
This Office Action is sent in response to the Applicant’s Communication received on 07/08/2025 for application number 17/350,619. The Office hereby acknowledges receipt of the following and placed of record in file: Specification, Drawings, Abstract, Oath/Declaration, IDS, and Claims.
Claims 1-25 are pending.
Claims 1, 3-6, 9, 13, 15, 17, and 19 are amended.
Claims 21-25 are newly added.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Non-statutory obviousness-type double patenting
Applicant argues claim 1 of U.S. Patent No. 12,236,338 is directed to a combined matrix multiplication and bias tensor adding function, which is different from an aspect of applicant's claimed invention (e.g., claim 1), which is directed to a combined convolution and bias tensor adding function. For at least these reasons, applicant respectfully requests withdrawal of the double patenting rejection.
The non-statutory obviousness-type double patenting rejection is based on the teachings of U.S. Patent No. 12,236,338 in view of Zheng et al. (US 11809849 B1) hereinafter Zheng, Majumdar et al. (US 20190042945 A1), hereinafter Majumdar, and Afzal (US 11520561 B1), hereinafter Afzal; not U.S. Patent No. 12,236,338 alone. Therefore, Applicant’s arguments with respect to non-statutory obviousness-type double patenting rejection to claim(s) 1, 7, and 8 have been considered but are moot because the ground of rejection is based on a combined teaching.
35 USC 101
Applicant argues that the claimed invention is directed to a judicial exception without significantly more. Applicant submits that applicant's claimed aspect of "wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor" is not an abstract idea without significantly more. These steps are more than mental processes and go beyond the perceived abstract idea. Further they are tied to computer technology and improve the functioning of a computer, since, for instance, there is "a single invocation of the processor" to perform the combined function of "convolution and the adding the values." By performing the combined function in a single invocation of the processor, the number of times the processor is invoked is reduced, saving on processor resources and improving system performance (see, e.g., paragraphs 46-47).
Applicant submits that claim 1 recites elements beyond any judicial exception. For instance, claim 1 recites "wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor."
Applicant submits that one or more of the elements recited in claim 1 reflect an improvement to a technology or technological field and/or to the functioning of the computer. For instance, applicant's recited aspect of "wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor" facilitates improvements to the technological field of computing and/or the functioning of a computer. Processing within the computing environment is improved by performing a combined function with a single invocation of a processor, instead of multiple invocations of the processor to perform operations of the combined function. This reduces the use of processor resources and improves system performance.
Applicant submits that one or more of applicant's claimed aspects of: "wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor" are unconventional or otherwise more than well- understood, routine, conventional activity previously known to the industry, specified at a high level of generality and therefore, claim 1 recites significantly more than a judicial exception.
Examiner, in view of the newly amended claim limitation "wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor," finds Applicant’s argument persuasive. Specifically, “Processing within the computing environment is improved by performing a combined function with a single invocation of a processor, instead of multiple invocations of the processor to perform operations of the combined function. This reduces the use of processor resources and improves system performance,” which “facilitates improvements to the technological field of computing and/or the functioning of a computer.” Therefore, the 35 U.S.C. 101 rejection has been withdrawn.
35 USC 103
Applicant’s arguments with respect to claim(s) 1-25 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Examiner notes that Claims 1, 7, and 8 of the instant application are patentably indistinct from claims 1 and 4 of U.S. Patent No. 12236338, and are therefore rejected on the ground of nonstatutory double patenting, as indicated below.
Instant application
U.S. Patent No. 12236338
Claim 1:
A computer program product for facilitating processing within a computing environment, the computer program product comprising:
one or more computer readable storage media and program instructions collectively stored on the one or more computer readable storage media to perform a method comprising: performing, by a processor, a combined function specified by an instruction, the combined function including a plurality of operations performed as part of one invocation of the combined function, wherein the performing the combined function comprises:
performing a convolution using a first tensor and a second tensor to obtain one or more intermediate results, the second tensor comprising an adjusted weight tensor created using a plurality of multipliers;
and adding values of a bias tensor to the one or more intermediate results to obtain one or more combined function results for the combined function
wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor
Claim 8:
wherein the one or more intermediate results are input to the adding absent a storing and reloading of the one or more intermediate results in a location externally accessible to one or more processors.
Claim 1:
A computer program product for facilitating processing within a computing environment, the computer program product comprising:
one or more computer readable storage media and program instructions collectively stored on the one or more computer readable storage media to perform a method comprising: performing a combined function specified by an instruction, the combined function including a plurality of operations performed as part of one invocation of the combined function, wherein the performing the combined function comprises:
performing a matrix multiplication of a first tensor and a second tensor to obtain one or more intermediate results, the second tensor comprising an adjusted weight tensor created using a multiplier;
and adding values of a bias tensor to the one or more intermediate results to obtain one or more results for the combined function,
the one or more results being at least a part of an output tensor,
and wherein the one or more intermediate results are input to the adding absent a storing and reloading of the one or more intermediate results in a location externally accessible to one or more processors
Claim 7
wherein the method further comprises creating the adjusted weight tensor, the creating including multiplying a weight tensor by the plurality of multipliers to provide the adjusted weight tensor
Claim 4
wherein the method further comprises creating the adjusted weight tensor, the creating including multiplying a weight tensor by the multiplier to provide the adjusted weight tensor
Regarding claims 1 and 8,
Claims 1 and 8 are rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. 12236338 in view of Zheng, Majumdar, and Afzal.
Claims 1 and 8 of the instant application are taught by claim 1 of the reference patent. However, reference patent 338’ does not teach “performing a convolution using a first tensor and a second tensor,” “weight tensor created using a plurality of multipliers,” “a processor,” and “one invocation of combined function specified by instruction and with a single invocation of the processor”
Regarding claims 1 and 8, Zheng teaches,
performing a convolution using tensors [See Fig. 4A; Col 2, lines 39-43, a convolution operator between two input tensors (e.g., an image tensor and a weight tensor) can include a set of repetitive multiply-and-accumulation operations, with each operation involving multiplying two elements from the two input tensors to generate a product]
Zheng is analogous to the claimed invention as they both relate to neural network computation. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of the reference patent to incorporate the teachings of Zheng and provide performing a convolution using tensors in order to, by using such operations, increase sophistication of neural network layer computations, allowing machine learning models to make inferences from more complex sets of data.
Zheng does not teach a weight tensor created using a plurality of multipliers, “a processor,” and “one invocation of combined function specified by instruction and with a single invocation of the processor”.
Majumdar teaches,
a weight tensor created using a plurality of multipliers [See Fig. 1C; Para 0046, The forward-feeding computations multiply, with point-wise multiplication 1224, the weight 1220;
PNG
media_image1.png
111
755
media_image1.png
Greyscale
Note: there’s a discrepancy between Fig. 1C and its corresponding description. The description reads “weight scalar multiplier (Sw) 1232” while the figure reads “weight scalar (Sw) 1222.”]
Majumdar is analogous to the claimed invention as they both relate to neural network computations. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the reference patent and Zheng’s teachings to incorporate the teachings of Majumdar and provide a weight tensor created using a plurality of multipliers in order to improve data fitting.
Zheng-Majumdar teach the above limitations of claim 1 including the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction.
Zheng-Majumdar do not teach “a processor,” and “one invocation of combined function specified by instruction and with a single invocation of the processor”.
Afzal teaches,
a processor [Col 3, lines 17-21, The NNA 100 comprises… an optional processor 114… a plurality of neural processing units (NPUs) 124, 126, and 128],
one invocation (Col 12, lines 58-66, execution) of combined function (Col 12, lines 58-66, three operations are fused into a single operation) specified by instruction (Col 12, lines 58-66, PERCEPT instruction can be used to specify the details of this fused operation) and with a single invocation of the processor (Col 12, lines 58-66, NNA 100) [Col 12, lines 58-66, If executed on a general purpose processor, each of the three phases (multiplication plus summation, bias, and activation function) would involve a separate memory access. In NNA 100, all three phases are performed in-line, with the results of one phase being passed directly to the next. Essentially, three operations are fused into a single operation for execution on the NPUs 126, 126, and 128. The PERCEPT instruction can be used to specify the details of this fused operation; Col 5, lines 3-9, The processor 114 is an optional general purpose processor for performing certain types of processing in parallel with processing performed by the NPUs 124, 126, and 128. For example, processor 114 may include a floating point unit or other arithmetic logic unit for performing general arithmetic operations in parallel with matrix operations performed by the NPUs 124, 126, and 128.].
Afzal is analogous to the claimed invention as they both relate to neural network accelerators. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zheng and Majumdar’s teachings to incorporate the teachings of Afzal and provide one invocation of a combined function with a single invocation of the processor [Afzal, Col 12, lines 58-66] in order to improve memory efficiency by removing the need for separate memory access.
Regarding claim 7, Zheng-Majumdar-Afzal teach the limitations of claim 1.
Claims 7 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 4 of U.S. Patent No. 12236338 in view of Zheng, Majumdar, and Afzal.
Claim 7 of the instant application is taught by claim 4 of the reference patent 338’. However, reference patent 338’ does not teach “multiplying a weight tensor by the plurality of multipliers”.
Majumdar teaches,
multiplying a tensor (Fig. 1C, point-wise multiplication 1224) by a plurality of multipliers (Fig. 1C, weight 1220 and weight scalar (Sw) 1222) [See Fig. 1C; Para 0046, The forward-feeding computations multiply, with point-wise multiplication 1224, the weight 1220 with the weight scalar multiplier (Sw) 1232 and multiply the input parameter 1210 with the product of the weight 1220 with the weight scalar multiplier (Sw) 1232 with matrix multiplication 1226;
PNG
media_image1.png
111
755
media_image1.png
Greyscale
Note: there’s a discrepancy between Fig. 1C and its corresponding description. The description reads “weight scalar multiplier (Sw) 1232” while the figure reads “weight scalar (Sw) 1222.”]
Majumdar is analogous to the claimed invention as they both relate to neural network computations. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the reference patent, Zheng, and Afzal’s teachings to incorporate the teachings of Majumdar and provide multiplying a tensor by a plurality of multipliers in order to improve data scaling.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 7-8, 13-14, 16-18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Fowers et al. (A Configurable Cloud-Scale DNN Processor for Real-Time AI, published 2018, hereinafter Fowers) in view of Zheng, Majumdar, and Afzal.
Regarding claim 1, Fowers teaches,
A computer program product [Sect I, pg. 1, Col 2, para 4, the architecture and microarchitecture of the BW NPU] for facilitating processing within a computing environment, the computer program product comprising: one or more computer readable storage media and program instructions collectively stored on the one or more computer readable storage media [Sect I, pg. 1, Col 2, para 4, the architecture and microarchitecture of the BW NPU, which is at the heart of the BW system. In its current form, the BW NPU is a DNN-optimized “soft processor” synthesized onto FPGAs] to perform a method comprising:
performing a combined function (Sect V, pg. 6, col 2, para 1, VLP) specified by an instruction (Sect V, pg. 6, col 2, para 1, single DNN request) [Sect V, pg. 6, col 2, para 1, The goal of the BW NPU microarchitecture, as explained in the introduction and in Section III, is to maximally exploit the vector-level parallelism (VLP) of a single DNN request], the combined function including a plurality of operations performed as part of one invocation of the combined function [Sect V, pg. 7, col 2, para 1, The MFUs support vector-vector operations such as multiplication and addition as well as unary vector activation functions like ReLU, sigmoid, and tanh]
Fowers does not teach wherein the performing the combined function comprises: performing a convolution using a first tensor and a second tensor to obtain one or more intermediate results, the second tensor comprising an adjusted weight tensor created using a plurality of multipliers; adding values of a bias tensor to the one or more intermediate results to obtain one or more combined function results; a processor, and wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor.
Zheng teaches,
wherein the performing the combined function comprises: performing a convolution using a first tensor (Col 18, lines 10-13, input tensor 401) and a second tensor (Col 18, lines 10-13, a weight tensor 403) to obtain one or more intermediate results (Col 18, lines 10-13, a first intermediate output tensor 410) [See Fig. 4A; Col 18, lines 10-13, In the illustrated example, an input tensor 401 is received by FCL operator 402-1, which also receives a weight tensor 403. FCL operator 402-1 can generate a first intermediate output tensor 410],
and adding values (Col 18, lines 13-17, addition operator 402-2) of a bias tensor (Col 18, lines 13-17, bias tensor 405) to the one or more intermediate results (Col 18, lines 13-17, Intermediate output tensor 410) to obtain one or more combined function results (Col 18, lines 13-17, second intermediate output tensor 414) [See Fig. 4A; Col 18, lines 13-17, Intermediate output tensor 410 is then processed by addition operator 402-2, which can add a bias from bias tensor 405 to each element of first intermediate output tensor 410 to generate a second intermediate output tensor 414.].
Zheng is analogous to the claimed invention as they both relate to neural network computation. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers’s teachings to incorporate the teachings of Zheng and provide performing a convolution using a first tensor and a second tensor to obtain one or more intermediate results and adding values of a bias tensor to the one or more intermediate results to obtain one or more combined function results in order to, by using such operations, increase sophistication of neural network layer computations, allowing machine learning models to make inferences from more complex sets of data.
Fowers-Zheng does not teach an adjusted weight tensor created using a plurality of multipliers; a processor, and wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor.
Majumdar teaches,
an adjusted weight tensor (Fig. 1C, point-wise multiplication 1224) created using a plurality of multipliers (Fig. 1C, weight scalar (Sw) 1222 and weight 1220) [See Fig. 1C; Para 0046, The forward-feeding computations multiply, with point-wise multiplication 1224, the weight 1220;
PNG
media_image1.png
111
755
media_image1.png
Greyscale
Note: there’s a discrepancy between Fig. 1C and its corresponding description. The description reads “weight scalar multiplier (Sw) 1232” while the figure reads “weight scalar (Sw) 1222.”]
Majumdar is analogous to the claimed invention as they both relate to neural network computations. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers and Zheng’s teachings to incorporate the teachings of Majumdar and provide the second tensor comprising an adjusted weight tensor created using a plurality of multipliers in order to improve data fitting.
Fowers-Zheng-Majumdar teach the above limitations of claim 1 including the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction.
Fowers-Zheng-Majumdar do not teach a processor, and one invocation of combined function specified by instruction and with a single invocation of the processor.
Afzal teaches,
a processor [Col 27, lines 52-55, As part of executing its own program code, the host processor 612 may generate program code (e.g., an NNA context) for execution on NNA 614],
and one invocation (Col 12, lines 58-66, execution) of combined function (Col 12, lines 58-66, three operations are fused into a single operation) specified by instruction (Col 12, lines 58-66, PERCEPT instruction can be used to specify the details of this fused operation) and with a single invocation of the processor (Col 12, lines 58-66, NNA 100) [Col 12, lines 58-66, If executed on a general purpose processor, each of the three phases (multiplication plus summation, bias, and activation function) would involve a separate memory access. In NNA 100, all three phases are performed in-line, with the results of one phase being passed directly to the next. Essentially, three operations are fused into a single operation for execution on the NPUs 126, 126, and 128. The PERCEPT instruction can be used to specify the details of this fused operation; Col 27, lines 48-50, The host processor 612 can be a general purpose integrated circuit that is capable of executing program instructions].
Afzal is analogous to the claimed invention as they both relate to neural network accelerators. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers, Zheng, and Majumdar’s teachings to incorporate the teachings of Afzal and provide one invocation of a combined function with a single invocation of the processor [Afzal, Col 12, lines 58-66] in order to improve memory efficiency by removing the need for separate memory access.
Regarding claim 2, Fowers-Zheng-Majumdar-Afzal teach all the limitations of claim 1.
Fowers further teaches,
wherein the performing the combined function further includes performing a selected activation (Sect V, pg. 7, col 2, para 1 & 2, vector-vector operations) on the one or more combined function results (Sect V, pg. 7, col 2, para 1 & 2, output from the MVM) to provide one or more activation results of the selected activation (Sect V, pg. 7, col 2, para 1 & 2, MFU’s… output), wherein the one or more activation results of the selected activation are at least a part of an output tensor (vectors can be pipelined through the MFU) [Sect V, pg. 7, col 2, para 1 & 2, The output from the MVM is routed through a series of vector multifunction units (MFUs). The MFUs support vector-vector operations such as multiplication and addition as well as unary vector activation functions like ReLU, sigmoid, and tanh… The crossbar is configured according to the current instruction chain to route the MFU’s input to its output via any sequence or sub-sequence of the internal function units (including a complete bypass). Once configured, a sequence of vectors can be pipelined through the MFU].
Regarding claim 7, Fowers-Zheng-Majumdar-Afzal teach all the limitations of claim 1.
Majumdar further teaches,
wherein the method further comprises creating the adjusted weight tensor (Fig. 1C, output of point-wise multiplication 1224), the creating including multiplying a weight tensor (Fig. 1C, point-wise multiplication 1224) by the plurality of multipliers (Fig. 1C, weight 1220 and weight scalar (Sw) 1222) to provide the adjusted weight tensor (Fig. 1C, output of point-wise multiplication 1224) [See Fig. 1C; Para 0046, The forward-feeding computations multiply, with point-wise multiplication 1224, the weight 1220 with the weight scalar multiplier (Sw) 1232 and multiply the input parameter 1210 with the product of the weight 1220 with the weight scalar multiplier (Sw) 1232 with matrix multiplication 1226;
PNG
media_image1.png
111
755
media_image1.png
Greyscale
Note: there’s a discrepancy between Fig. 1C and its corresponding description. The description reads “weight scalar multiplier (Sw) 1232” while the figure reads “weight scalar (Sw) 1222.”]
Majumdar is analogous to the claimed invention as they both relate to neural network computations. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers, Zheng, and Afzel’s teachings to incorporate the teachings of Majumdar and provide creating the adjusted weight tensor, the creating including multiplying a weight tensor by the plurality of multipliers to provide the adjusted weight tensor in order to improve data scaling.
Regarding claim 8, Fowers-Zheng-Majumdar-Afzal teach all the limitations of claim 1.
Fowers doesn’t teach wherein the one or more intermediate results are input to the adding absent a storing and reloading of the one or more intermediate results in a location externally accessible to one or more processors.
Zheng teaches,
wherein the one or more intermediate results (Fig. 4A, Intermediate output tensor 410) are input to the adding (Fig. 4A, addition operator 402-2) absent a storing and reloading of the one or more intermediate results in a location externally accessible to one or more processors [Col 18, lines 14-16, Intermediate output tensor 410 is then processed by addition operator 402-2; Note: Zheng does not teach the one or more intermediate results are absent a storing and reloading of the one or more intermediate results
PNG
media_image2.png
521
396
media_image2.png
Greyscale
in a location externally accessible to one or more processors].
Zheng is analogous to the claimed invention as they both relate to neural network computation. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers, Majumdar, and Afzal’s teachings to incorporate the teachings of Zheng and provide wherein the one or more intermediate results are input to the adding absent a storing and reloading of the one or more intermediate results in a location externally accessible to one or more processors in order to increase efficiency of data access and reduce latency.
Regarding claim 13, Fowers further teaches,
A computer system for facilitating processing within a computing environment [Sect I, pg. 1, Col 2, para 4, the architecture and microarchitecture of the BW NPU], the computer system comprising: a memory [Sect I, pg. 1, Col 2, para 4, NPU]; and at least one processor in communication with the memory, wherein the computer system is configured to perform a method [Sect I, pg. 1, Col 2, para 4, This paper details the architecture and microarchitecture of the BW NPU, which is at the heart of the BW system. In its current form, the BW NPU is a DNN-optimized “soft processor” synthesized onto FPGAs.], said method comprising:
performing a combined function (Sect V, pg. 6, col 2, para 1, VLP) specified by an instruction (Sect V, pg. 6, col 2, para 1, single DNN request) [Sect V, pg. 6, col 2, para 1, The goal of the BW NPU microarchitecture, as explained in the introduction and in Section III, is to maximally exploit the vector-level parallelism (VLP) of a single DNN request], the combined function including a plurality of operations performed as part of one invocation of the combined function [Sect V, pg. 7, col 2, para 1, The MFUs support vector-vector operations such as multiplication and addition as well as unary vector activation functions like ReLU, sigmoid, and tanh]
Fowers does not teach wherein the performing the combined function comprises: performing a convolution using a first tensor and a second tensor to obtain one or more intermediate results, the second tensor comprising an adjusted weight tensor created using a plurality of multipliers; adding values of a bias tensor to the one or more intermediate results to obtain one or more combined function results; a processor, and wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor.
Zheng further teaches,
wherein the performing the combined function comprises: performing a convolution using a first tensor (Col 18, lines 10-13, input tensor 401) and a second tensor (Col 18, lines 10-13, a weight tensor 403) to obtain one or more intermediate results (Col 18, lines 10-13, a first intermediate output tensor 410) [See Fig. 4A; Col 18, lines 10-13, In the illustrated example, an input tensor 401 is received by FCL operator 402-1, which also receives a weight tensor 403. FCL operator 402-1 can generate a first intermediate output tensor 410],
and adding values (Col 18, lines 13-17, addition operator 402-2) of a bias tensor (Col 18, lines 13-17, bias tensor 405) to the one or more intermediate results (Col 18, lines 13-17, Intermediate output tensor 410) to obtain one or more combined function results (Col 18, lines 13-17, second intermediate output tensor 414) [See Fig. 4A; Col 18, lines 13-17, Intermediate output tensor 410 is then processed by addition operator 402-2, which can add a bias from bias tensor 405 to each element of first intermediate output tensor 410 to generate a second intermediate output tensor 414.].
Zheng is analogous to the claimed invention as they both relate to neural network computation. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers’s teachings to incorporate the teachings of Zheng and provide performing a convolution using a first tensor and a second tensor to obtain one or more intermediate results and adding values of a bias tensor to the one or more intermediate results to obtain one or more combined function results in order to, by using such operations, increase sophistication of neural network layer computations, allowing machine learning models to make inferences from more complex sets of data.
Fowers-Zheng does not teach an adjusted weight tensor created using a plurality of multipliers; a processor, and wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor.
Majumdar teaches,
an adjusted weight tensor (Fig. 1C, point-wise multiplication 1224) created using a plurality of multipliers (Fig. 1C, weight scalar (Sw) 1222 and weight 1220) [See Fig. 1C; Para 0046, The forward-feeding computations multiply, with point-wise multiplication 1224, the weight 1220;
PNG
media_image1.png
111
755
media_image1.png
Greyscale
Note: there’s a discrepancy between Fig. 1C and its corresponding description. The description reads “weight scalar multiplier (Sw) 1232” while the figure reads “weight scalar (Sw) 1222.”]
Majumdar is analogous to the claimed invention as they both relate to neural network computations. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers and Zheng’s teachings to incorporate the teachings of Majumdar and provide the second tensor comprising an adjusted weight tensor created using a plurality of multipliers in order to improve data fitting.
Fowers-Zheng-Majumdar teach the above limitations of claim 1 including the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction.
Fowers-Zheng-Majumdar do not teach a processor, and one invocation of combined function specified by instruction and with a single invocation of the processor.
Afzal teaches,
a processor [Col 27, lines 52-55, As part of executing its own program code, the host processor 612 may generate program code (e.g., an NNA context) for execution on NNA 614],
and one invocation (Col 12, lines 58-66, execution) of combined function (Col 12, lines 58-66, three operations are fused into a single operation) specified by instruction (Col 12, lines 58-66, PERCEPT instruction can be used to specify the details of this fused operation) and with a single invocation of the processor (Col 12, lines 58-66, NNA 100) [Col 12, lines 58-66, If executed on a general purpose processor, each of the three phases (multiplication plus summation, bias, and activation function) would involve a separate memory access. In NNA 100, all three phases are performed in-line, with the results of one phase being passed directly to the next. Essentially, three operations are fused into a single operation for execution on the NPUs 126, 126, and 128. The PERCEPT instruction can be used to specify the details of this fused operation; Col 27, lines 48-50, The host processor 612 can be a general purpose integrated circuit that is capable of executing program instructions].
Afzal is analogous to the claimed invention as they both relate to neural network accelerators. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers, Zheng, and Majumdar’s teachings to incorporate the teachings of Afzal and provide one invocation of a combined function with a single invocation of the processor [Afzal, Col 12, lines 58-66] in order to improve memory efficiency by removing the need for separate memory access.
Claim 14 is a computer system claim reciting the same limitations as claim 2. Therefore, claim 14 is rejected using the same rationale as claim 2.
Claim 16 is a computer-implemented method claim reciting the same limitations as claim 8. Therefore, claim 16 is rejected using the same rationale as claim 8.
Regarding claim 17, Fowers further teaches,
A computer-implemented method of facilitating processing within a computing environment [Sect I, pg. 1, Col 2, para 4, the architecture and microarchitecture of the BW NPU, which is at the heart of the BW system. In its current form, the BW NPU is a DNN-optimized “soft processor” synthesized onto FPGAs], the computer-implemented method comprising:
performing a combined function (Sect V, pg. 6, col 2, para 1, VLP) specified by an instruction (Sect V, pg. 6, col 2, para 1, single DNN request) [Sect V, pg. 6, col 2, para 1, The goal of the BW NPU microarchitecture, as explained in the introduction and in Section III, is to maximally exploit the vector-level parallelism (VLP) of a single DNN request], the combined function including a plurality of operations performed as part of one invocation of the combined function [Sect V, pg. 7, col 2, para 1, The MFUs support vector-vector operations such as multiplication and addition as well as unary vector activation functions like ReLU, sigmoid, and tanh]
Fowers does not teach wherein the performing the combined function comprises: performing a convolution using a first tensor and a second tensor to obtain one or more intermediate results, the second tensor comprising an adjusted weight tensor created using a plurality of multipliers; adding values of a bias tensor to the one or more intermediate results to obtain one or more combined function results; a processor, and wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor.
Zheng further teaches,
wherein the performing the combined function comprises: performing a convolution using a first tensor (Col 18, lines 10-13, input tensor 401) and a second tensor (Col 18, lines 10-13, a weight tensor 403) to obtain one or more intermediate results (Col 18, lines 10-13, a first intermediate output tensor 410) [See Fig. 4A; Col 18, lines 10-13, In the illustrated example, an input tensor 401 is received by FCL operator 402-1, which also receives a weight tensor 403. FCL operator 402-1 can generate a first intermediate output tensor 410],
and adding values (Col 18, lines 13-17, addition operator 402-2) of a bias tensor (Col 18, lines 13-17, bias tensor 405) to the one or more intermediate results (Col 18, lines 13-17, Intermediate output tensor 410) to obtain one or more combined function results (Col 18, lines 13-17, second intermediate output tensor 414) [See Fig. 4A; Col 18, lines 13-17, Intermediate output tensor 410 is then processed by addition operator 402-2, which can add a bias from bias tensor 405 to each element of first intermediate output tensor 410 to generate a second intermediate output tensor 414.].
Zheng is analogous to the claimed invention as they both relate to neural network computation. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers’s teachings to incorporate the teachings of Zheng and provide performing a convolution using a first tensor and a second tensor to obtain one or more intermediate results and adding values of a bias tensor to the one or more intermediate results to obtain one or more combined function results in order to, by using such operations, increase sophistication of neural network layer computations, allowing machine learning models to make inferences from more complex sets of data.
Fowers-Zheng does not teach an adjusted weight tensor created using a plurality of multipliers; a processor, and wherein the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction and with a single invocation of the processor.
Majumdar teaches,
an adjusted weight tensor (Fig. 1C, point-wise multiplication 1224) created using a plurality of multipliers (Fig. 1C, weight scalar (Sw) 1222 and weight 1220) [See Fig. 1C; Para 0046, The forward-feeding computations multiply, with point-wise multiplication 1224, the weight 1220;
PNG
media_image1.png
111
755
media_image1.png
Greyscale
Note: there’s a discrepancy between Fig. 1C and its corresponding description. The description reads “weight scalar multiplier (Sw) 1232” while the figure reads “weight scalar (Sw) 1222.”]
Majumdar is analogous to the claimed invention as they both relate to neural network computations. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers and Zheng’s teachings to incorporate the teachings of Majumdar and provide the second tensor comprising an adjusted weight tensor created using a plurality of multipliers in order to improve data fitting.
Fowers-Zheng-Majumdar teach the above limitations of claim 1 including the performing the convolution and the adding the values are performed as part of the one invocation of the combined function specified by the instruction.
Fowers-Zheng-Majumdar do not teach a processor, and one invocation of combined function specified by instruction and with a single invocation of the processor.
Afzal teaches,
a processor [Col 27, lines 52-55, As part of executing its own program code, the host processor 612 may generate program code (e.g., an NNA context) for execution on NNA 614],
and one invocation (Col 12, lines 58-66, execution) of combined function (Col 12, lines 58-66, three operations are fused into a single operation) specified by instruction (Col 12, lines 58-66, PERCEPT instruction can be used to specify the details of this fused operation) and with a single invocation of the processor (Col 12, lines 58-66, NNA 100) [Col 12, lines 58-66, If executed on a general purpose processor, each of the three phases (multiplication plus summation, bias, and activation function) would involve a separate memory access. In NNA 100, all three phases are performed in-line, with the results of one phase being passed directly to the next. Essentially, three operations are fused into a single operation for execution on the NPUs 126, 126, and 128. The PERCEPT instruction can be used to specify the details of this fused operation; Col 27, lines 48-50, The host processor 612 can be a general purpose integrated circuit that is capable of executing program instructions].
Afzal is analogous to the claimed invention as they both relate to neural network accelerators. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers, Zheng, and Majumdar’s teachings to incorporate the teachings of Afzal and provide one invocation of a combined function with a single invocation of the processor [Afzal, Col 12, lines 58-66] in order to improve memory efficiency by removing the need for separate memory access.
Claim 18 is a computer system claim reciting the same limitations as claim 2. Therefore, claim 18 is rejected using the same rationale as claim 2.
Claim 20 is a computer-implemented method claim reciting the same limitations as claim 8. Therefore, claim 20 is rejected using the same rationale as claim 8.
Regarding claim 21, Fowers-Zheng-Majumdar-Afzal teach the limitations of claim 17.
Afzal further teaches,
wherein the instruction specifying the combined function (Col 12, lines 62-66, PERCEPT) is executing on another processor (Col 12, lines 62-66, NPUs 126, 126, and 128) coupled to the processor performing the combined function (Fig 1, Host Processor; Col 27, lines 52-55, the host processor 612) [Col 12, lines 62-66, Essentially, three operations are fused into a single operation for execution on the NPUs 126, 126, and 128. The PERCEPT instruction can be used to specify the details of this fused operation; Col 27, lines 52-55, As part of executing its own program code, the host processor 612 may generate program code (e.g., an NNA context) for execution on NNA 614],
the instruction including an operation code specifying the instruction (Col 23, lines 35-37, opcode) and a function code specifying the combined function (Col 23, lines 35-37, PERCEPTx) [Col 23, lines 35-58, Each instruction or extended instruction can be identified by a corresponding operational code (opcode), in the same manner as the instructions described above… In the table above, the extended instructions are labeled with an “x”, e.g., LOADx, PERCEPTx, and EWOPx. Not every instruction may have an extended counterpart. Extended instructions can be used when the information needed to indicate the operation to be performed cannot fit within the width of a single instruction].
Afzal is analogous to the claimed invention as they both relate to neural network accelerators. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers, Zheng, and Majumdar’s teachings to incorporate the teachings of Afzal and provide the instruction, operation code, and function code [Afzal, Col 23, lines 32-34] in order to add new functionality in order to support the operation of neural networks as neural networks continue to evolve.
Regarding claim 22, Fowers-Zheng-Majumdar-Afzal teach the limitations of claim 17, including the adding the values of the bias tensor to the one or more intermediate results and absent a storing of the one or more intermediate results to a location externally visible to processors.
Afzal further teaches,
The method being performed within the single invocation of the processor [Col 12, lines 58-66, If executed on a general purpose processor, each of the three phases (multiplication plus summation, bias, and activation function) would involve a separate memory access. In NNA 100, all three phases are performed in-line, with the results of one phase being passed directly to the next. Essentially, three operations are fused into a single operation for execution on the NPUs 126, 126, and 128. The PERCEPT instruction can be used to specify the details of this fused operation; Col 27, lines 48-50, The host processor 612 can be a general purpose integrated circuit that is capable of executing program instructions].
Afzal is analogous to the claimed invention as they both relate to neural network accelerators. Therefore, it would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Fowers, Zheng, and Majumdar’s teachings to incorporate the teachings of Afzal and provide the method being performed within the single invocation of the processor [Afzal, Col 12, lines 58-66] in order to improve memory efficiency by removing the need for separate memory access.
Regarding claim 23, Fowers-Zheng-Majumdar-Afzal teach the limitations of claim 1.
Afzal further teaches,
wherein the instruction specifying the combined function (The PERCEPT instruction can be used to specify the details of this fused o