DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to applicant’s RCE filed on 12/10/2025.
Claims 1-11, 13-17, 26-27, 29-32 are pending and examined.
Response to Arguments
Applicant’s arguments filed on 12/10/2025 have been fully considered.
Per 103 rejection, applicant’s argument is moot in light of new grounds of rejection with a new reference (Woo) applied.
Per 101 rejection, applicant argued that “Like the claims at issue in Desjardins, Applicant's claim recites features that constitute an improvement to how a processor cause the neural network to operate by reusing parameter data in portions of a neural network assigned to a group. Therefore, Applicant's claim as a whole integrates the alleged judicial exception into a practical application such that the claim is not directed to the alleged judicial exception”, “Furthermore, as stated by Applicant's background, Applicant's features as recited in Applicant's claims are not well understood, routine, nor conventional in the field”. The examiner respectfully disagrees, reusing parameter data in a neural network computation is a known practice in the field of the art, as evidenced in the updated 103 rejection below. Therefore, applicant’s claimed features do not constitute an improvement in the field of the art, the claimed features are well understood, routine or conventional in the field.
The examiner is available for a phone interview with applicant.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11, 13-17, 26-27, 29-32 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
Statutory Category: Claim 26 recites a method comprising: obtaining an inference from a neural network by performing one or more layers of the neural network using parameter data obtained by a single memory access for a group of two or more portions of the one or more layers of the neural network assigned to the group according to a group size, wherein the group size is determined based, at least in part, on a size of the parameter data, wherein the group is one of a plurality of different groups that include different portions of the one or more layers of the neural network, and wherein the one or more layers are to: obtain respective input data for each of the two or more portions in the group; and reuse the parameter data at each of the two or more portions in the group to compute respective output data for each of the two or more portions based on the respective input data.
Step 2A – Prong 1: Claim 26 recites obtaining an inference from a neural network by performing one or more layers of the neural network using parameter data (a mental step, performing a mental analysis based on received data and generating a result); wherein the group size is determined based, at least in part, on a size of the parameter data, wherein the group is one of a plurality of different groups that include different portions of the one or more layers of the neural network (a mental step of determination); reuse the parameter data at each of the two or more portions in the group to compute respective output data for each of the two or more portions based on the respective input data (a mental step of computing output data based on received data). These limitations as drafted, is a process that, under their broadest reasonable interpretation, covers an abstract idea of performance of the limitation in the mind or manually. That is, nothing in the claim elements precludes the steps from practically being performed mentally or using pen and paper. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the mental process grouping of abstract idea. Accordingly, the claim recites an abstract idea under step 2A prong 1.
This judicial exception is not integrated into a practical application. In particular, the claim 26 recites additional elements such as parameter data “obtained by a single memory access for a group of two or more portions of the one or more layers of the neural network”, “obtain respective input data for each of the two or more portions in the group”. Examiner would like to point out that with the broad reasonable interpretation, this element amounts to mere data gathering for the mental process, which does not impose any meaningful limits on practicing the mental process (insignificant additional element). Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to insignificant additional elements under Step 2B prong 2.
This judicial exception is not integrated into a practical application. In particular, the claim 26 recites additional elements such as utilizing “neural network” to perform mental steps. Examiner would like to point out that with the broad reasonable interpretation, this element amounts to no more than generic software component with instructions to apply the exception (as evidenced in Goyal, claims 1-9, 18-21; using layers in a neural network to perform computations). Accordingly, these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. The claim is directed to insignificant additional elements under Step 2A prong 2.
Dependent claims 27, 29-32 do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of dependent claims 27, 29-32 recite more steps of a mental process (assigning, evaluating, computing) which can be performed mentally or using pen and paper. Therefore, these claims are not patent eligible. The dependent claims also recite limitations as such as loading data, this amounts to an extra solution activity of data gathering. The dependent claims also recite further descriptions of a neural network. Accordingly, this additional element does not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Therefore, these claims are not patent eligible.
Independent claim 1 (a processor to perform a method similar to claim 26) with dependent claims 2-8 are rejected under the similar rational as claims 26-27, 29-32. The additional elements in the claim amounts to no more than generic hardware component with instructions to apply the exception, which cannot integrate a judicial exception into a practical application or provide an inventive concept.
Independent claim 9 (a system with processors to perform a method similar to claim 26) with dependent claims 10-11, 13-17 are rejected under the similar rational as claims 26-27, 29-32. The additional elements in the claim amounts to no more than generic hardware component with instructions to apply the exception, which cannot integrate a judicial exception into a practical application or provide an inventive concept.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-5, 9, 13-14, and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal et al. (US PGPUB 2017/0316312) hereinafter Goyal, in view of Woo (US PGPUB 2018/0373976).
Per claim 1, Goyal discloses “a processor, comprising: one or more circuits to cause one or more layers of a neural network to generate a final output of the neural network based, at least in part, on a single memory access that retrieves parameter data for a group of two or more portions of the one or more layers of the neural network” (Fig. 2; claims 1-9, 18-21; a processor to generate deep learning processing results (final output) for pattern recognition and classification based on a neural network; each of the convolutional network engines is configured to keep and repeatedly apply a same kernel (parameter data) on different parts of the input data at each layer of the neural network wherein the kernel is loaded into the memory only once (single memory access) during the convolution operations; a neural network comprises different layers, each layer comprises of two or more neurons, thus, each layer is a group of two or more portions of the one or more layers of the neural network); “wherein the one or more layers are to: obtain respective input data for each of the two or more portions in the group; and reuse the parameter data at each of the two or more portions in the group to compute respective output data for each of the two or more portions based on the respective input data” (Fig. 2; claims 1-9, 18-21; each of the convolutional network engines is configured to keep and repeatedly apply a same kernel (parameter data) on different parts of the input data at each layer of the neural network wherein the kernel is loaded into the memory only once (single memory access) during the convolution operations; i.e. each layer receives its input data, applying (reusing) the same kernel to the input data to produce output data).
Goyal does not explicitly teach portions of the one or more layers of the neural network “assigned to the group according to a group size, wherein the group size is determined based, at least in part, on a size of the parameter data, wherein the group is one of a plurality of different groups that include different portions of the one or more layers of the neural network”. However, Woo suggests the above (paragraphs [0052][0099][0100]; partition layers of a neural network into groupings of layers; the size parameter of the inputs and the aggregate memory used for the parameters to determine a total on-chip memory usage for one or more groups of layers; circuit can compare the total memory usage for each group of layers to the on-chip storage capacity, circuit can then determine grouping of layers that form a sequence of superlayers based on the results of the comparison; i.e. determine a size of parameters and its memory usage, grouping portions of layers of the neural network to different groups based on the size of parameters). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Goyal and Woo to group portions of neural network layers to different groups based on the size of parameter data, to ensure there is enough allocated memory resources for the parameter data (see Woo, paragraphs [0098]-[0100]).
Per claim 4, Goyal further discloses wherein the single memory access retrieves the parameter data from a main-memory and stores the parameter data in processor memory (claims 1-9; an on-system memory (OSM) and one or more controllers configured to access a plurality of external memory resources via direct memory access; wherein the kernel is loaded into the memory only once (single memory access) during the convolution operations; i.e. the kernel is loaded into on system memory from an external memory).
Per claim 5, Goyal further discloses “wherein the one or more layers of the neural network include a convolution layer and the parameter data retrieved by the single memory access includes data associated with the convolution layer” (claims 1-9, 18-21; an on-system memory (OSM) and one or more controllers configured to access a plurality of external memory resources via direct memory access; wherein the kernel is loaded into the memory only once (single memory access) during the convolution operations, performing operations at convolution layers).
Claims 9, 13-14 recite similar limitations as claims 1, 4-5. Therefore, claims 9, 13-14 are rejected under similar rationales as claims 1, 4-5.
Claim 26 is rejected under similar rationales as claim 1.
Claims 2-3, 10-11, 17 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal, in view of Woo, in view of Ferdman et al. (US PGPUB 2019/0220734) hereinafter Ferdman.
Per claim 2, Goyal does not explicitly teach “the one or more layers of the neural network comprise a first layer and a second layer of the neural network, wherein the first and second layers are fused by retention of output of the first layer in processor memory an evaluation of the second layer based, at least in part, on the output of the first layer retained in processor memory”. However, Ferdman suggests the above (Fig. 1B; paragraphs [0050]-[0056]; a chip retrieves data from external memory, processes the data through multiple layers of a neural network without accessing the external memory during the processing; “the fused layer convolutional neural network exploits inter-layer data locality among feature-map data (e.g., the three-dimensional tile structures) of the convolutional layers such that already processed intermediate data of the first convolutional layer can be reused by the second convolutional layer , and so on with other fused convolutional layers, without reading and writing intermediate data between convolutional layers off chip to and from the external memory”). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Goyal, Woo and Ferdman to fuse two layers to produce an output based on the output of the first layer, this would reduce external memory access and increase performance.
Per claim 3, Goyal does not explicitly teach “wherein the plurality of different groups correspond to a fused layer that includes the one or more layers”. However, Ferdman suggests the above (Fig. 1B; paragraphs [0050]-[0056]; a chip retrieves data from external memory, processes the data through multiple layers of a neural network without accessing the external memory during the processing; each layer is a sub-block; two or more sub-blocks are utilized to process the input data in a single memory access; “the fused layer convolutional neural network exploits inter-layer data locality among feature-map data (e.g., the three-dimensional tile structures) of the convolutional layers such that already processed intermediate data of the first convolutional layer can be reused by the second convolutional layer , and so on with other fused convolutional layers, without reading and writing intermediate data between convolutional layers off chip to and from the external memory”; i.e. each fused layer (comprised of two layers) can be viewed as a group). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Goyal, Woo and Ferdman to fuse two layers as one group to produce an output based on the output of the first layer, this would reduce external memory access and increase performance.
Claims 10 and 11 are rejected under similar rationales as claim 2.
Claims 27 is rejected under similar rationales as claim 2.
Per claim 17, Goyal does not explicitly teach “wherein a first portion of the group generates a first output, and wherein a second portion of the group generates a second output based, at least in part, on the first output and the parameter data”. However, Ferdman suggests wherein a first portion of the group generates a first output, and wherein a second portion of the group generates a second output based, at least in part, on the first output (Fig. 1B; paragraphs [0050]-[0056]; a chip retrieves data from external memory, processes the data through multiple layers of a neural network without accessing the external memory during the processing; each layer is a sub-block; two or more sub-blocks are utilized to process the input data in a single memory access; “the fused layer convolutional neural network exploits inter-layer data locality among feature-map data (e.g., the three-dimensional tile structures) of the convolutional layers such that already processed intermediate data of the first convolutional layer can be reused by the second convolutional layer , and so on with other fused convolutional layers, without reading and writing intermediate data between convolutional layers off chip to and from the external memory”; i.e. each fused layer (comprised of two layers) can be viewed as a group, the first layer is the first portion, the second layer is the second portion, the output of the first portion is used by the second portion to produce a second output). Goyal further discloses (Fig. 2; claims 1-9, 18-21; each of the convolutional network engines is configured to keep and repeatedly apply a same kernel (parameter data) on different parts of the input data at each layer of the neural network wherein the kernel is loaded into the memory only once (single memory access) during the convolution operations; i.e. each layer receives its input data, applying (reusing) the same kernel (parameter data) to the input data to produce output data). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Goyal, Woo and Ferdman to fuse two layers as one group to produce an output based on the output of the first layer, this would reduce external memory access and increase performance.
Claims 6, 15, 29 and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal, in view of Woo, in view of BORSE et al. (US PGPUB 2021/0150347) hereinafter BORSE.
Per claim 6, Goyal does not explicitly teach “wherein the one or more layers comprise a depthwise separable convolution layer fused with a pointwise convolution layer”. However, BORSE suggests the above (paragraphs [0045]-[0047]; depthwise convolution layer fused with pointwise convolution layer). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Goyal, Woo and BORSE that the layers comprise a depthwise separable convolution layer fused with a pointwise convolution layer, in order to reduce the computational burden.
Claims 15 and 29 are rejected under similar rationales as claim 6.
Per claim 30, Goyal suggests evaluating two or more of portions of the one or more convolution layers using the parameter data stored in processor memory by the single memory access, without fetching again the parameter data from non-processor memory (claims 1-9, 18-21; each of the convolutional network engines is configured to keep and repeatedly apply a same kernel (parameter data) on different parts of the input data at each layer of the neural network wherein the kernel is loaded into the memory only once (single memory access) during the convolution operations, each layer contains two or more neurons). Goyal does not explicitly teach “wherein the one or more layers comprise a depthwise separable convolution layer fused with a pointwise convolution layer”. However, BORSE suggests the above (paragraphs [0045]-[0047]; depthwise convolution layer fused with pointwise convolution layer). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Goyal, Woo and BORSE that the layers comprise a depthwise separable convolution layer fused with a pointwise convolution layer, in order to reduce the computational burden.
Claims 7, 16 and 31 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal, in view of Woo, in view of Franca-Neto (US PGPUB 2019/0244086).
Per claim 7, Goyal does not explicitly teach “wherein performance of a further group of the plurality of different groups is performed by a different processor”. However, Franca-Neto suggests (paragraphs [0043][0111]; assigning different subsets of processing units to process different layers of a neural network, based on load balancing; neural networks are inherently parallel algorithms, meaning that different computations within the network can be executed a piece at a time on different processing devices, with the computations later combined to get the end result). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Goyal, Woo and Franca-Neto to assign different processors to process different layers (groups) in a neural network, for load balancing purpose (more efficient utilization of computing resources).
Claims 16 and 31 are rejected under similar rationales as claim 7.
Claims 8, 17 and 32 are rejected under 35 U.S.C. 103 as being unpatentable over Goyal, in view of Woo, in view of Gangani et al. (US PGPUB 2021/0240524) hereinafter Gangani.
Per claim 8, Goyal does not explicitly teach wherein the performance of the one or more layers further interleaves computation of a first portion in the group with fetching data for a second portion of the group. However, Gangani suggests (paragraph [0029]; in performing machine learning operations by a GPU, allowing interleaving memory access by writing batch output data generated by the execution of a first batch of computational jobs while loading input data associated with a second batch of computational jobs; i.e. interleaving task execution and data loading). It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine Goyal, Woo and Gangani to interleave execution of first computational jobs with loading filter data for second computational jobs; this would increase performance of a GPU executing machine learning tasks.
Claims 17 and 32 are rejected under similar rationales as claim 8.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HANG PAN whose telephone number is (571)270-7667. The examiner can normally be reached 9 AM to 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chat Do can be reached at 571-272-3721. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HANG PAN/Primary Examiner, Art Unit 2193