Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/18/2026 has been entered.
Remarks
This Office Action is responsive to Applicants' Amendment filed on May 18, 2026, in which claims 1-8 and 12-20 are currently amended. Claims 1-23 are currently pending.
Response to Arguments
Applicant’s arguments with respect to rejection of claims 1-23 under 35 U.S.C. 102/103 based on amendment have been considered.
With respect to Applicant’s arguments on p. 10 of the Remarks submitted 5/18/2026 that Guan does not “disclose using a tiling scheme, based on data associating that tiling scheme with combinations of neural network information and processing-circuitry properties, to divide each input-data instance into portions before neural-network processing, process the portions through multiple neural-network layers to generate corresponding partial outputs, and recombine those partial outputs to generate an output for the input-data instance”, Examiner respectfully disagrees. Guan discloses ([p. 154] "The Model Mapper uses the model description to extract information about model structure and configurations of each layer. Although storing model parameters and intermediate results in on-chip BRAM can significantly improve performance, we do not have enough on-chip BRAMs on a single FPGA to store all of them for modern DNNs. As a result, we have to allocate data buffers in off-chip DDR memory for storing intermediate activations and model parameters. Then, Model Mapper generates an Execution Graph shown in Figure 2c, which shows ideally how the model inference is performed on hardware.") where Guan explicitly ties neural network information and processing circuitry properties together in the tile mapping data. Guan also teaches ([p. 155 §IVA] "MM takes in two tiles of input matrices, and performs the tiled multiplication vector by vector. All the input data are fed into multipliers simultaneously, then the intermediate results are summed up") where Guan divides input layer data into patches, tiles, vectors, buffers, and/or batched input vectors. Guan then processes those portions through layer-by-layer DNN inference using a shared tiled MM computation engine and Data Arranger. Guan then explicitly stores intermediate activations and combines intermediate/tiled results through reduction, matrix multiplication output assembly, channel-major output generation, and subsequent layer aggregation to generate the final model output.
This same interpretation applies to Applicant’s arguments on p. 11 of the Remarks submitted 5/18/2026 that Guan “does not disclose the claimed data that associates tiling schemes with combination of neural-network information and processing circuitry properties for processing input-data instances in the manner recited in independent claims 1 and 13”.
With respect to Applicant’s arguments on p. 11 of the Remarks submitted 5/18/2026 that “Nor does Guan disclose determining the given tiling scheme by matching the acquired neural network information and processing-circuitry properties to one of the combinations associated in the data with respective tiling schemes”, this argument is persuasive, however, is moot in view of a new ground of rejection set forth below.
With respect to Applicant’s arguments on p. 11 of the Remarks submitted 5/18/2026 that “Guan does not disclose determining patch-processing tiling parameters for dividing an input-data instance into patches that are processed as portions of the instance” or “determining overlap between neighboring patches as part of such patch processing”, Examiner respectfully disagrees. Applicant appears to import an unreasonably narrow interpretation of “patch” into the claim with the argument “Guan’s use of “patch” […] is not the claimed patch processing tiling scheme”. Neither the instant claims, instant specification, or Applicant’s arguments point out the difference in the claimed “patch”. Guan explicitly teaches ([p. 156 §IV] "Firstly, we need to turn the input features from a 3-D array into a 2-D array that we can calculate as a matrix. To get a single feature in an output channel, we need to convolve a 3-D cube of input features (also known as a patch) with the corresponding convolution kernels. So we take each one of these input patches and flatten them into a single row of input matrix").
With respect to Applicant’s arguments on pp. 11-12 of the Remarks submitted 5/18/2026 that Guan does not disclose “for the layer processing tiling scheme: the portions of each instance of input data comprise channels or other subdivisions from among a plurality of channels in the instances of input data; and layer processing is used to process groups of two or more channels or other subdivisions.”, Examiner respectfully disagrees. Guan objectively subdivides input data for the layer processing tiling scheme ([p. 155 §4] "According to the rules of matrix multiplication, each output channel is serialized into a column of output matrix" [p. 156 §IV] "Convolutional layers: For communication optimizations in convolutional layers, we use Figure 4 as a simplified example to illustrate the problems and our solutions. In this example, we set the number of input channels as 8, and each channel has 3×3 elements, so we get 72 input elements in total").
With respect to Applicant’s arguments on p. 12 of the Remarks submitted 5/18/2026 that Guan does not teach “wherein processing the portions in the neural network using the given tiling scheme includes fusing groups of two or more channels or other subdivisions for processing in adjacent layers of the neural network such that intermediate results generated from processing the fused groups are stored in a local memory of the processing circuitry”, this argument is persuasive, however, is moot in view of a new ground of rejection set forth below.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4, 6-11, 13-16, and 19- 23 are rejected under U.S.C. §102(a)(1) as being anticipated by Guan ("FP-DNN: An Automated Framework for Mapping Deep Neural Networks onto FPGAs with RTL-HLS Hybrid Templates", 2017).
PNG
media_image1.png
542
588
media_image1.png
Greyscale
FIG. 1 of Guan
Regarding claim 1, Guan teaches An electronic device, comprising: processing circuitry configured to: acquire information about a neural network ([p. 152] "we propose FP-DNN (Field Programmable DNN), an end-to-end framework that takes TensorFlow-described DNNs as input, and automatically generates the hardware implementations on FPGA boards with RTL-HLS hybrid templates" [p. 154 §III] "The Model Mapper uses the model description to extract information about model structure and configurations of each layer. Although storing model parameters and intermediate results in on-chip BRAM can significantly improve performance, we do not have enough on-chip BRAMs on a single FPGA to store all of them for modern DNNs. As a result, we have to allocate data buffers in off-chip DDR memory for storing intermediate activations and model parameters. Then, Model Mapper generates an Execution Graph shown in Figure 2c, which shows ideally how the model inference is performed on hardware" See also FIG. 1. Guan explicitly acquires DNN information by taking a TensorFlow described DNN model description as input.)
and properties of the processing circuitry;([p. 152] "we propose FP-DNN (Field Programmable DNN), an end-to-end framework that takes TensorFlow-described DNNs as input, and automatically generates the hardware implementations on FPGA boards with RTL-HLS hybrid templates" [p. 154] "strategies to allocate hardware resources reasonably" [p. 158] "our FP-DNN performs convolution with a generalized MM kernel, which is designed and optimized under certain hardware constraints")
process each instance of input data with the neural network using a given tiling scheme([p. 155 §IV] "we take advantage of the tiling strategy to perform matrix multiplication. To insure the multiplication is correctly performed in a tiling manner, we pad zeros to input matrices if any dimension of them is not divisible by its tiling size [...] MM takes in two tiles of input matrices, and performs the tiled multiplication vector by vector" [p. 152] "We implement accelerators with RTL-HLS hybrid templates, and convert model inference into general-purpose computations like matrix multiplication")
based on data that associates the given tiling scheme with combinations of neural-network information and processing-circuitry properties, ([p. 154] "The Model Mapper uses the model description to extract information about model structure and configurations of each layer. Although storing model parameters and intermediate results in on-chip BRAM can significantly improve performance, we do not have enough on-chip BRAMs on a single FPGA to store all of them for modern DNNs. As a result, we have to allocate data buffers in off-chip DDR memory for storing intermediate activations and model parameters. Then, Model Mapper generates an Execution Graph shown in Figure 2c, which shows ideally how the model inference is performed on hardware." Guan explicitly ties neural network information and processing circuitry properties together in the tile mapping data)
wherein, to process each instance of input data with the neural network using the given tiling scheme, the processing circuitry is configured to: divide the instance of input data into a plurality of portions before the portions are processed in the neural network([p. 156] "For communication optimizations in convolutional layers, we use Figure 4 as a simplified example to illustrate the problems and our solutions [...] In Figure 4, we show three different layout schemes for comparison: Im2col, Row-major and Channel-major. For each scheme, we show its DRAM layout and DRAM accessing pattern for the first tile" [p. 157 §IVC] "Channel-major: Different from Row-major, Channel-major stores Input Features in a channel-major manner. Thus, Input Matrix needs to be reorganized correspondingly: each row
(input patch) is also flattened in a channel-major manner. The contents of reorganized first tile is also shown in Figure 4c. So there are in total 72 elements stored in DRAM for Input Features without any data duplication. And DRAM is also accessed continuously for fetching input elements. Furthermore, in this scheme, the outputs are also generated in a channel-major manner, which indicates no extra operations for data reorganizing or duplication are needed." See FIG. 4)
process each of the portions through multiple layers of the neural network to generate a corresponding partial output for each of the portions; and([p. 157 §IVC] "So there are in total 72 elements stored in DRAM for Input Features without any data duplication. And DRAM is also accessed continuously for fetching input elements.
Furthermore, in this scheme, the outputs are also generated in a channel-major manner")
combine the corresponding partial outputs to generate an output for the instance of input data([p. 155 §IVA] "MM takes in two tiles of input matrices, and performs the tiled multiplication vector by vector. All the input data are fed into multipliers simultaneously, then the intermediate results are summed up" Guan divides input layer data into patches, tiles, vectors, buffers, and/or batched input vectors. Guan then processes those portions through layer-by-layer DNN inference using a shared tiled MM computation engine and Data Arranger. Guan then explicitly stores intermediate activations and combines intermediate/tiled results through reduction, matrix multiplication output assembly, channel-major output generation, and subsequent layer aggregation to generate the final model output).
Regarding claim 2, Guan teaches The electronic device of claim 1, wherein the data associates a plurality of tiling schemes with combinations of neural-network information and properties of processing circuitry(Guan [p. 154] "The Model Mapper uses the model description to extract information about model structure and configurations of each layer. Although storing model parameters and intermediate results in on-chip BRAM can significantly improve performance, we do not have enough on-chip BRAMs on a single FPGA to store all of them for modern DNNs. As a result, we have to allocate data buffers in off-chip DDR memory for storing intermediate activations and model parameters. Then, Model Mapper generates an Execution Graph shown in Figure 2c, which shows ideally how the model inference is performed on hardware." Guan explicitly ties neural network information and processing circuitry properties together in the tile mapping data).
Regarding claim 3, Guan teaches The electronic device of claim 2, wherein the data comprises a table, database, or other record that relates combinations of neural-network information and properties of processing circuitry to respective tiling schemes(Guan [p. 154] "The Model Mapper uses the model description to extract information about model structure and configurations of each layer. Although storing model parameters and intermediate results in on-chip BRAM can significantly improve performance, we do not have enough on-chip BRAMs on a single FPGA to store all of them for modern DNNs. As a result, we have to allocate data buffers in off-chip DDR memory for storing intermediate activations and model parameters. Then, Model Mapper generates an Execution Graph shown in Figure 2c, which shows ideally how the model inference is performed on hardware." Guan's execution graph is interpreted as a record that relates combinations of neural network information and properties of processing circuitry to respective tiling schemes).
Regarding claim 4, Guan teaches The electronic device of claim 2, wherein the set of tiling schemes includes two or more of: a line buffer processing tiling scheme; a patch processing tiling scheme; and(Guan [p. 156 §IV] "Firstly, we need to turn the input features from a 3-D array into a 2-D array that we can calculate as a matrix. To get a single feature in an output channel, we need to convolve a 3-D cube of input features (also known as a patch) with the corresponding convolution kernels. So we take each one of these input patches and flatten them into a single row of input matrix")
a layer processing tiling scheme.(Guan [p. 154 §IIB] "W Generator is in fact a library of RTL-HLS hybrid templates for various types of layers." [p. 155 §IV] "we divide the operations involved in each layer into computation-intensive part and layer-specific part").
Regarding claim 6, Guan teaches The electronic device of claim 4, wherein, for the patch processing tiling scheme: the portions of the instances of input data comprise patches from among a plurality of patches in the instances of input data; and patch processing is used to process the patches(Guan [p. 156 §IV] "Firstly, we need to turn the input features from a 3-D array into a 2-D array that we can calculate as a matrix. To get a single feature in an output channel, we need to convolve a 3-D cube of input features (also known as a patch) with the corresponding convolution kernels. So we take each one of these input patches and flatten them into a single row of input matrix").
Regarding claim 7, Guan teaches The electronic device of claim 6, wherein use of the patch processing tiling scheme includes determining one or more of: a size and/or shape of the patches; and an overlap of each patch with one or more neighboring patches.(Guan [p. 156 §IV] "Firstly, we need to turn the input features from a 3-D array into a 2-D array that we can calculate as a matrix. To get a single feature in an output channel, we need to convolve a 3-D cube of input features (also known as a patch) with the corresponding convolution kernels. So we take each one of these input patches and flatten them into a single row of input matrix").
Regarding claim 8, Guan teaches The electronic device of claim 4, wherein, for the layer processing tiling scheme: the portions of each instance of input data comprise channels or other subdivisions from among a plurality of channels in the instances of input data; and layer processing is used to process groups of two or more channels or other subdivisions.(Guan [p. 155 §4] "According to the rules of matrix multiplication, each output channel is serialized into a column of output matrix" [p. 156 §IV] "Convolutional layers: For communication optimizations in convolutional layers, we use Figure 4 as a simplified example to illustrate the problems and our solutions. In this example, we set the number of input channels as 8, and each channel has 3×3 elements, so we get 72 input elements in total").
Regarding claim 9, Guan teaches The electronic device of claim 1, wherein the information about the neural network includes information about one or more of: an internal arrangement of the neural network; properties of filters used in the neural network; feature sizes for the neural network; and channel sizes for the neural network.(Guan [p. 155 §4] "According to the rules of matrix multiplication, each output channel is serialized into a column of output matrix" [p. 156 §IV] "Convolutional layers: For communication optimizations in convolutional layers, we use Figure 4 as a simplified example to illustrate the problems and our solutions. In this example, we set the number of input channels as 8, and each channel has 3×3 elements, so we get 72 input elements in total").
Regarding claim 10, Guan teaches The electronic device of claim 1, wherein the information about the neural network includes information about one or more of: properties of instances of input data to be processed in the neural network; and properties of outputs of the neural network.(Guan [p. 155 §IVB] "Convolutional Layers: Convolutional layers are overwhelmingly popular in applications like image recognition, object detection, object classification, etc. Suppose we have Nin input channels and Nout output channels").
Regarding claim 11, Guan teaches The electronic device of claim 1, wherein the information about the properties of the processing circuitry includes information about one or more of: an amount of local memory available for storing data by the processing circuitry; (Guan [p. 154 §IIIB] "we do not have enough on-chip BRAMs on a single FPGA to store all of them for modern DNNs. As a result, we have to allocate data buffers in off-chip DDR memory for storing intermediate activations and model parameters")
and a processing capacity of the processing circuitry.(Guan [p. 157 §V] "the FPGA platform, we use Catapult [18] system with Altera Stratix-V GSMD5 FPGAs integrated. We use the PikesPeak version of Catapult in our experiments, which has a 4GB DDR3 DRAM as the external memory. The FPGA logic clock frequency is at 150MHz, and the run-time power of the FPGA board is about 25W. This FPGA board is plugged into a PCI-e Gen2 x8 slot of a host computer").
Regarding claims 13-16, claims 13-16 are directed towards the method performed by the device of claims 1-4, respectively. Therefore, the rejection applied to claims 1-4 also applies to claims 13-16.
Regarding claim 19, Guan teaches The method of claim 16, wherein using the patch processing tiling scheme includes determining one or more of: a size and/or shape of the patches; and an overlap of each patch with neighboring patches.(Guan [p. 156 §IV] "Firstly, we need to turn the input features from a 3-D array into a 2-D array that we can calculate as a matrix. To get a single feature in an output channel, we need to convolve a 3-D cube of input features (also known as a patch) with the corresponding convolution kernels. So we take each one of these input patches and flatten them into a single row of input matrix").
Regarding claims 20-23, claims 20-23 are directed towards the method performed by the device of claims 8-11, respectively. Therefore, the rejections applied to claims 8-11 also apply to claims 20-23.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 5, 12, 17, and 18 are rejected under U.S.C. §103 as being unpatentable over the combination of Guan and Zhang (“Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks”, 2015).
Regarding claim 5, Guan teaches The electronic device of claim 2.
However, Guan doesn't explicitly teach, wherein the processing circuitry is configured to determine the given tiling scheme by matching the information about the neural network and the properties of the processing circuitry to one of the combinations associated in the data with the respective tiling schemes.
Zhang, in the same field of endeavor, teaches the processing circuitry is configured to determine the given tiling scheme by matching the information about the neural network and the properties of the processing circuitry to one of the combinations associated in the data with the respective tiling schemes ([p. 163] "loop tiling is mandatory to fit a small portion of data on chip" [p. 164] "the data processing throughput of PEs should match the off-chip bandwidth provided by the FPGA platform." [p. 165] "Given a specific tile size combination Tm,Tn,Tr,Tc , the computational performance (or computational roof in the roofline model) can be calculated by Equation(3). From the equation, we can observe that the computational roof is a function of Tm and Tn" Zhang's respective tiling schemes are the candidate tile size tuples (Tm, Tn, Tr, Tc) which correspond to tiling across output channels, input channels, output feature-map rows, and output feature map columns. Zhang evaluates each tuple against CNN-layer dimensions such as M, N, R, C, K, and against FPGA properties such as number of PEs, BRAM capacity, computational roof, bandwidth roof, and platform-supported bandwidth).
Guan as well as Zhang are directed towards hardware aware neural network tiling. Therefore, Guan as well as Zhang are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Guan with the teachings of Zhang by trying different tile/group sizes for channels and feature-map pieces and selecting the best tiling method based on the hardware and neural network model. Zhang teaches how to choose the right-sized chunks so the work fits and runs efficiently and provides as additional motivation for combination ([p. 165] “Loop pipelining is a key optimization technique in high-level synthesis to improve system throughput by overlapping the execution of operations from different loop iterations. […] Polyhedral-based optimization framework [16] can be used to perform automatic loop transformation to permute the parallel loop levels to the innermost levels to avoid loop carried dependence […] shown in Code 3”).
Regarding claim 12, Guan teaches The electronic device of claim 1.
However, Guan doesn't explicitly teach, wherein processing the portions in the neural network using the given tiling scheme includes fusing groups of two or more channels or other subdivisions for processing in adjacent layers of the neural network such that intermediate results generated from processing the fused groups are stored in a local memory of the processing circuitry.
Zhang, in the same field of endeavor, teaches The electronic device of claim 1, wherein processing the portions in the neural network using the given tiling scheme includes fusing groups of two or more channels or other subdivisions for processing in adjacent layers of the neural network such that intermediate results generated from processing the fused groups are stored in a local memory of the processing circuitry ([p. 164] "too<min(to+tM,M) […] tii<min(ti+Tn,N) […] output_fm [too][trr][tcc] += weights[too][tii][i][j]*input_fm[tii][S*trr*i][S*tcc+j];" Zhang explicitly tiles CNN loop dimensions, including the input/output feature-nap dimensions. It says "row, col, to and ti are tiled," where ‘to’ corresponds to output feature maps and ‘ti’ corresponds to input feature maps/channels which are then shown explicitly as grouped in pseudocode output).
Guan as well as Zhang are directed towards hardware aware neural network tiling. Therefore, Guan as well as Zhang are analogous art in the same field of endeavor. It would have been obvious before the effective filing date of the claimed invention to combine the teachings of Guan with the teachings of Zhang by trying different tile/group sizes for channels and feature-map pieces and selecting the best tiling method based on the hardware and neural network model. Zhang teaches how to choose the right-sized chunks so the work fits and runs efficiently and provides as additional motivation for combination ([p. 165] “Loop pipelining is a key optimization technique in high-level synthesis to improve system throughput by overlapping the execution of operations from different loop iterations. […] Polyhedral-based optimization framework [16] can be used to perform automatic loop transformation to permute the parallel loop levels to the innermost levels to avoid loop carried dependence […] shown in Code 3”).
Regarding claim 17, claim 17 is directed towards a method performed by the system of claim 5. Therefore, the rejection applied to claim 5 also applies to claim 17.
Regarding claim 18, the combination of Guan, and Zhang teaches The method of claim 16, wherein, for the patch processing tiling scheme: the portions of the instances of input data comprise patches from among a plurality of patches in the instances of input data; and patch processing is used for processing patches from the instances of input data.(Guan [p. 156 §IV] "Firstly, we need to turn the input features from a 3-D array into a 2-D array that we can calculate as a matrix. To get a single feature in an output channel, we need to convolve a 3-D cube of input features (also known as a patch) with the corresponding convolution kernels. So we take each one of these input patches and flatten them into a single row of input matrix").
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kao (“Gamma: Automating the hw mapping of dnn models on accelerators via genetic algorithm”, 2020) is directed towards a hardware aware neural network tiling method involving fusing operators.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SIDNEY VINCENT BOSTWICK whose telephone number is (571)272-4720. The examiner can normally be reached M-F 7:30am-5:00pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached on (571)270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SIDNEY VINCENT BOSTWICK/Examiner, Art Unit 2124
Prosecution Insights