DETAILED ACTION
Claims 1-20 are presented for examination.
This office action is in response to submission of application on 05/23/2024.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 10/14/2025 and 07/15/2024 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 7, 8, 10, 11, 18 and 19 are objected to because of the following informalities: acronyms such as “FGPA” and “GraSP” must be defined clearly (phrase followed by its acronym in parentheses) at least once per each claim set, preferably during their first occurrence for respective claim sets. Appropriate correction is required.
Specification
The disclosure is objected to because of the following informalities:
In paragraph 00030, “…in more details below…” should be “…in more detail below…”.
In paragraph 00032, “.The hardware processing elements including logic gates that are…” should be “The hardware processing elements include logic gates that are…”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 18 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “satisfactory” in claim 18 is a relative term which renders the claim indefinite. The term “satisfactory” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. Regarding “performance of the retrained pruned individual is satisfactory” and “performance of the retrained pruned individual subgraph is not satisfactory” in claim 18, such performances are rendered indefinite by the use of the term “satisfactory”.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Ogbogu et al. (“Data Pruning-enabled High Performance and Reliable Graph Neural Network Training on ReRAM-based Processing-in-Memory Accelerators”), as cited in the IDS dated 10/14/2025, hereafter Ogbogu, in view of Zhang et al. ("Accurate, Low-latency, Efficient SAR Automatic Target Recognition on FPGA "), hereafter Zhang.
Regarding claim 1, Ogbogu discloses:
A system, comprising: a device configured to receive a graph neural network model (Fig. 1 and Fig. 2 teaches a device to receive a graph neural network (GNN) model),
a graph neural network hardware accelerator configured to receive the graph neural network model from the device and to divide the graph neural network model into multiple subgraphs (Fig. 1, Fig. 2, and page 72:18, first paragraph, lines 8-10 “During the execution of the GNN training on the ReRAM-based PIM architecture, the pre-trained BGC is used to perform the subgraph pruning.” teaches a GNN PIM accelerator to receive the GNN model and divide the GNN into subgraphs using a pre-trained BGC),
the graph neural network hardware accelerator comprising a memory shared by multiple parallel hardware processing elements … (Fig. 2(b) and page 72:7, para 4, lines 1-2 “ReRAMs enable parallel and efficient in-memory Matrix-Vector-Multiplication (MVM) operations.” teaches the ReRam-PIM accelerator to comprise a memory shared by multiple parallel hardware processing elements),
… processing elements employing pruning algorithms to individual subgraphs which are recombined on the shared memory to generate a trained and pruned graph neural network model to send the trained and pruned graph neural network model to the device for generating output to a received user prompt (Algorithm 1, Fig. 2, page 72:9, final paragraph, lines 6-8 “Ideally, we want to find the smallest subset of accuracy preserving subgraphs … and the corresponding pruning labels … because it aligns with our overall goal” teaches offline processing elements to employ subgraph pruning algorithms to individual subgraphs which are recombined, in line 15 of algorithm 1, to generate a trained and pruned graph neural network model to return to user for generating an output of accuracy preserving subgraphs in response to the user’s prompt).
While Ogbogu discloses the graph neural network hardware accelerator comprising a memory shared by multiple parallel hardware processing elements … and … processing elements employing pruning algorithms to individual subgraphs which are recombined on the shared memory to generate a trained and pruned graph neural network model to send the trained and pruned graph neural network model, they do not disclose the hardware processing elements to employ this pruning.
Zhang discloses:
the graph neural network hardware accelerator comprising a memory shared by multiple parallel hardware processing elements employing pruning (Fig. 3, Fig. 4, page 4, right column, paragraphs 2-3 “The accelerator executes each layer using Scatter-Gather paradigm (SGP). The accelerator exploits the computation parallelism within each layer... In the VUKs, the feature vector of each vertex is multiplied by a weight matrix to obtain the updated feature vector. Due to our weight pruning, the weight matrices have high data sparsity (1%-33% data density).” Teaches hardware processing elements to employ pruning during scatter-gather paradigm executed on the hardware accelerator).
Ogbogu and Zhang are analogous art because they are from the same field of endeavor, accelerators and machine learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu to include the graph neural network hardware accelerator comprising a memory shared by multiple parallel hardware processing elements employing pruning, based on the teachings of Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to reduce the computation complexity, as suggested by Zhang (page 1, right column, paragraph 1).
Regarding claim 2, Ogbogu, in view of Zhang, discloses the system of claim 1 (and thus the rejection of claim 1 is incorporated). Zhang further discloses:
wherein the graph neural network hardware accelerator further comprises a field programable gate array that includes the multiple parallel hardware processing elements employing pruning algorithms (page 5, right column, paragraph 2, lines 1-4 “We implement our accelerator on an embedded FPGA platform Xilinx ZCU104. We implement 8 pipelines (8 Scatter Units and 8 Gather Units). Each Scatter/Gather Unit has 16 processing elements (PEs).” and page 5, left column, paragraph 3, lines 7-8 “…p parallel pipelines.” Teaches field programable gate array that includes the multiple parallel hardware processing elements employing pruning algorithms).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu to include wherein the graph neural network hardware accelerator further comprises a field programable gate array that includes the multiple parallel hardware processing elements employing pruning algorithms, based on the teachings of Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to reduce the computation complexity, as suggested by Zhang (page 1, right column, paragraph 1).
Regarding claim 6, Ogbogu, in view of Zhang, discloses the system of claim 1 (and thus the rejection of claim 1 is incorporated). Ogbogu further discloses:
wherein the graph neural network hardware accelerator is configured in a pipeline configuration with the multiple processing elements arranged in parallel (Fig. 1, Fig. 2, and page 72:7, para 4, lines 1-2 “ReRAMs enable parallel and efficient in-memory Matrix-Vector-Multiplication (MVM) operations.” and page 72:8, paragraph 1, lines 7-9 “Figure 1 (b) shows the end-to-end pipeline for training a L-layer deep GNN for one input subgraph. This deep pipelining strategy allows all layers of the GNN to be computed in parallel.” Teaches the graph neural network hardware accelerator is configured in a pipeline configuration with the multiple processing elements arranged in parallel),
pruning individual subgraphs and passing the pruned subgraphs to the shared memory until all of the subgraphs have been pruned (Algorithm 3).
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Ogbogu et al. (“Data Pruning-enabled High Performance and Reliable Graph Neural Network Training on ReRAM-based Processing-in-Memory Accelerators”), as cited in the IDS dated 10/14/2025, hereafter Ogbogu, in view of Zhang et al. ("Accurate, Low-latency, Efficient SAR Automatic Target Recognition on FPGA "), hereafter Zhang, in further view of Kovvuri et al. (US 2019/0286973 A1), hereafter Kovvuri.
Regarding claim 3, Ogbogu, in view of Zhang, discloses the system of claim 1 (and thus the rejection of claim 1 is incorporated). Zhang further discloses:
wherein the graph neural network hardware accelerator further comprises …circuit … that includes the multiple parallel hardware processing elements employing pruning algorithms (Fig. 3).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu to include wherein the graph neural network hardware accelerator further comprises …circuit … that includes the multiple parallel hardware processing elements employing pruning algorithms, based on the teachings of Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to reduce the computation complexity, as suggested by Zhang (page 1, right column, paragraph 1).
Ogbogu, in view of Zhang, does not explicitly teach this circuit to be an application specific integrated circuit (ASIC).
Kovvuri discloses:
wherein the graph neural network hardware accelerator further comprises an application specific integrated circuit (ASIC)… (¶[0067]).
Ogbogu, Zhang, and Kovvuri are analogous art because they are from the same field of endeavor, accelerators and machine learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, to include wherein the graph neural network hardware accelerator further comprises an application specific integrated circuit (ASIC), based on the teachings of Kovvuri. One of ordinary skill in the art would have been motivated to make this modification in order to accelerate performance for tasks, as suggested by Kovvuri (¶[0019]).
Regarding claim 4, Ogbogu, in view of Zhang, in further view of Kovvuri, discloses the system of claim 3 (and thus the rejection of claim 3 is incorporated). Ogbogu further discloses:
a central processing unit (CPU) and a graphical processing unit (GPU) (Table 4),
wherein the CPU is configured to send the graph neural network model to the graph neural network hardware accelerator, and wherein the trained and pruned graph neural network model is employed on the GPU (page 72:15, Table 4 and paragraph below Table 4 “We incorporate a Python wrapper function based on PyTorch API into NeuroSIM to obtain the GNN model test accuracies for the different GNN models”).
Claims 7 and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Ogbogu et al. (“Data Pruning-enabled High Performance and Reliable Graph Neural Network Training on ReRAM-based Processing-in-Memory Accelerators”), as cited in the IDS dated 10/14/2025, hereafter Ogbogu, in view of Zhang et al. ("Accurate, Low-latency, Efficient SAR Automatic Target Recognition on FPGA "), hereafter Zhang, in further view of Walston et al. (US 20240169135 A1), hereafter Walston.
Regarding claim 7, Ogbogu, in view of Zhang, discloses the system of claim 6 (and thus the rejection of claim 6 is incorporated). Zhang further discloses:
wherein the pipeline configuration comprises…FPGA comprising multiple processing elements arranged in parallel to one another (Fig. 3).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu to include -wherein the pipeline configuration comprises…FPGA comprising multiple processing elements arranged in parallel to one another, based on the teachings of Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to reduce the computation complexity, as suggested by Zhang (page 1, right column, paragraph 1).
While Zhang discloses wherein the pipeline configuration comprises …FPGA comprising multiple processing elements arranged in parallel to one another, they don’t disclose multiple FPGAs arranged in parallel.
Walston discloses:
multiple FPGAs arranged in parallel (¶[0087] teaches FPGAs in parallel).
Ogbogu, Zhang, and Walston are analogous art because they are from the same field of endeavor, GNNs and machine learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, to include multiple FPGAs arranged in parallel, based on the teachings of Walston. One of ordinary skill in the art would have been motivated to make this modification in order to complete cycles in a shorter amount of time, as suggested by Walston (¶[0087]).
Regarding claim 19, Ogbogu discloses:
A graph neural network hardware accelerator, comprising: … multiple parallel arranged processing elements; a shared memory … (Fig. 1, Fig. 2, and page 72:7, para 4, lines 1-2 “ReRAMs enable parallel and efficient in-memory Matrix-Vector-Multiplication (MVM) operations.”),
individual processing elements configured to prune a subgraph of a graph neural network model (Fig. 2 teaches individual processing elements configured to prune a subgraph of a graph neural network model)
the shared memory configured to recombine the pruned subgraphs to generate a pruned graph neural network model (Algorithm 1, Fig. 2, page 72:9, final paragraph, lines 6-8 “Ideally, we want to find the smallest subset of accuracy preserving subgraphs … and the corresponding pruning labels … because it aligns with our overall goal” and page 72:15, Table 4 and paragraph below Table 4 “We incorporate a Python wrapper function based on PyTorch API into NeuroSIM to obtain the GNN model test accuracies for the different GNN models” teaches offline processing elements to employ subgraph pruning algorithms to individual subgraphs which are recombined, in line 15 of algorithm 1, to generate a trained and pruned graph neural network model).
While Ogbogu discloses a graph neural network hardware accelerator, comprising: … multiple parallel arranged processing elements; a shared memory, they do not teach multiple FPGAs that each comprise multiple parallel arranged processing elements; a shared memory coupled to the multiple FPGAs.
Zhang discloses:
… FPGAs that each comprise multiple parallel arranged processing elements; a shared memory coupled to the … FPGAs (page 5, right column, paragraph 2, lines 1-4 “We implement our accelerator on an embedded FPGA platform Xilinx ZCU104. We implement 8 pipelines (8 Scatter Units and 8 Gather Units). Each Scatter/Gather Unit has 16 processing elements (PEs).” and page 5, left column, paragraph 3, lines 7-8 “…p parallel pipelines.” Teaches field programable gate array that includes the multiple parallel hardware processing elements and a shared memory coupled to the FPGA).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu to include … FPGAs that each comprise multiple parallel arranged processing elements; a shared memory coupled to the … FPGAs, based on the teachings of Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to reduce the computation complexity, as suggested by Zhang (page 1, right column, paragraph 1).
While Zhang discloses … FPGAs that each comprise multiple parallel arranged processing elements; a shared memory coupled to the … FPGAs, they do not teach multiple FPGAs.
Walston discloses:
multiple FPGAs (¶[0087] teaches multiple FPGAs).
Ogbogu, Zhang, and Walston are analogous art because they are from the same field of endeavor, GNNs and machine learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, to include multiple FPGAs, based on the teachings of Walston. One of ordinary skill in the art would have been motivated to make this modification in order to complete cycles in a shorter amount of time, as suggested by Walston (¶[0087]).
Regarding claim 20, Ogbogu, in view of Zhang, in further view of Walston, discloses the method of claim 19 (and thus the rejection of claim 19 is incorporated). Ogbogu further discloses:
generate the pruned subgraphs (Figure 2 teaches generating pruned subgraphs).
While Ogbogu discloses generating the pruned subgraphs, they do not disclose each processing element employs an instance of a pruning algorithm.
Zhang discloses:
each processing element employs an instance of a pruning algorithm (page 5, right column, paragraph 2, lines 1-4 “We implement our accelerator on an embedded FPGA platform Xilinx ZCU104. We implement 8 pipelines (8 Scatter Units and 8 Gather Units). Each Scatter/Gather Unit has 16 processing elements (PEs).” and page 5, left column, paragraph 3, lines 7-8 “…p parallel pipelines.” Teaches field programable gate array that includes the multiple parallel hardware processing elements employing pruning algorithms).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu to include each processing element employs an instance of a pruning algorithm, based on the teachings of Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to reduce the computation complexity, as suggested by Zhang (page 1, right column, paragraph 1).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Ogbogu et al. (“Data Pruning-enabled High Performance and Reliable Graph Neural Network Training on ReRAM-based Processing-in-Memory Accelerators”), as cited in the IDS dated 10/14/2025, hereafter Ogbogu, in view of Zhang et al. ("Accurate, Low-latency, Efficient SAR Automatic Target Recognition on FPGA "), hereafter Zhang, in further view of Kovvuri et al. (US 2019/0286973 A1), hereafter Kovvuri, in further view of Cunegatti et al. (“Peeking Inside Sparse Neural Networks Using Multi-Partite Graph Representations”), hereafter Grasp.
Regarding claim 5, Ogbogu, in view of Zhang, in further view of Kovvuri, discloses the system of claim 4 (and thus the rejection of claim 4 is incorporated). Ogbogu, in view of Zhang, in further view of Kovvuri, does not disclose:
wherein the pruning algorithm comprises a Gradient Signal Preservation (GraSP) pruning algorithm.
Grasp discloses:
wherein the pruning algorithm comprises a Gradient Signal Preservation (GraSP) pruning algorithm (Figure 1 and Table 2 teaches pruning via a Gradient Signal Preservation pruning algorithm).
Ogbogu, Zhang, Kovvuri, and Grasp are analogous art because they are from the same field of endeavor, graphs and machine learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, in further view of Kovvuri, to include wherein the pruning algorithm comprises a Gradient Signal Preservation (GraSP) pruning algorithm, based on the teachings of Grasp. One of ordinary skill in the art would have been motivated to make this modification in order to discover the best subnetwork while maintaining cheap computation cost, as suggested by Grasp (page 3, paragraph “Pruning at initialization”).
Claims 8-18 are rejected under 35 U.S.C. 103 as being unpatentable over Ogbogu et al. (“Data Pruning-enabled High Performance and Reliable Graph Neural Network Training on ReRAM-based Processing-in-Memory Accelerators”), as cited in the IDS dated 10/14/2025, hereafter Ogbogu, in view of Zhang et al. ("Accurate, Low-latency, Efficient SAR Automatic Target Recognition on FPGA "), hereafter Zhang, in further view of Walston et al. (US 20240169135 A1), hereafter Walston, in further view of Cunegatti et al. (“Peeking Inside Sparse Neural Networks Using Multi-Partite Graph Representations”), hereafter Grasp.
Regarding claim 8, Ogbogu discloses:
A device-implemented method, comprising: receiving an untrained graph neural network (GNN) model at a hardware accelerator comprising … multiple parallel arranged hardware processing elements and a shared memory (Fig. 1, Fig. 2, and page 72:7, para 4, lines 1-2 “ReRAMs enable parallel and efficient in-memory Matrix-Vector-Multiplication (MVM) operations.” teaches a device to receive an untrained graph neural network (GNN) model at a hardware accelerator comprising multiple parallel arranged hardware processing elements and a shared memory),
dividing the untrained GNN model into multiple subgraphs (Fig. 1 and Fig. 2),
distributing the multiple subgraphs among the multiple hardware processing elements …for parallel processing (Fig. 1 and page 72:8, paragraph 1, lines 7-9 “Figure 1 (b) shows the end-to-end pipeline for training a L-layer deep GNN for one input subgraph. This deep pipelining strategy allows all layers of the GNN to be computed in parallel.” Page 72:8, paragraph 3, lines 8-11 “From Equation ( 6 ) and Figures 1 (a)–(c), we can see that reducing the number of subgraph partitions ( K), for example, by creating fewer partitions, but larger subgraphs will lead to a reduction in pipeline depth ( D) and number of ReRAM cell write operations ( cw).” Teaches distributing the multiple subgraphs among the multiple hardware processing elements for parallel processing),
… prune individual subgraphs on individual hardware processing elements utilizing … algorithms (Algorithm 3, Fig. 2, teaches pruning individual subgraphs on individual hardware processing elements utilizing algorithms),
recombining the pruned subgraphs into a trained and pruned GNN model; and, sending the trained and pruned GNN model to a device comprising a central processing unit or graphics processing unit for processing of user queries (Algorithm 1, Fig. 2, page 72:9, final paragraph, lines 6-8 “Ideally, we want to find the smallest subset of accuracy preserving subgraphs … and the corresponding pruning labels … because it aligns with our overall goal” and page 72:15, Table 4 and paragraph below Table 4 “We incorporate a Python wrapper function based on PyTorch API into NeuroSIM to obtain the GNN model test accuracies for the different GNN models” teaches offline processing elements to employ subgraph pruning algorithms to individual subgraphs which are recombined, in line 15 of algorithm 1, to generate a trained and pruned graph neural network model to return to user for generating an output of accuracy preserving subgraphs in response to the user’s queries).
`While Ogbogu discloses receiving an untrained graph neural network (GNN) model at a hardware accelerator comprising…multiple parallel arranged hardware processing elements and a shared memory, they don’t disclose a hardware accelerator comprising multiple field programmable gate arrays (FPGAs) that each comprise multiple parallel arranged hardware processing elements.
Zhang discloses:
a hardware accelerator comprising … field programmable gate arrays (FPGAs) that each comprise multiple parallel arranged hardware processing elements (page 5, right column, paragraph 2, lines 1-4 “We implement our accelerator on an embedded FPGA platform Xilinx ZCU104. We implement 8 pipelines (8 Scatter Units and 8 Gather Units). Each Scatter/Gather Unit has 16 processing elements (PEs).” and page 5, left column, paragraph 3, lines 7-8 “…p parallel pipelines.” Teaches a hardware accelerator comprising field programable gate array that includes the multiple parallel hardware processing elements).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu to include a hardware accelerator comprising … field programmable gate arrays (FPGAs) that each comprise multiple parallel arranged hardware processing elements, based on the teachings of Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to reduce the computation complexity, as suggested by Zhang (page 1, right column, paragraph 1).
While Ogbogu discloses pruning individual subgraphs on individual hardware processing elements utilizing algorithms, they do not disclose employing parallel processing across the multiple hardware processing elements to prune.
Zhang discloses:
employing parallel processing across the multiple hardware processing elements to prune (Fig. 3, Fig. 4, page 4, right column, paragraphs 2-3 “The accelerator executes each layer using Scatter-Gather paradigm (SGP). The accelerator exploits the computation parallelism within each layer... In the VUKs, the feature vector of each vertex is multiplied by a weight matrix to obtain the updated feature vector. Due to our weight pruning, the weight matrices have high data sparsity (1%-33% data density).” Teaches hardware processing elements to employ pruning during scatter-gather paradigm executed on the hardware accelerator).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu to include employing parallel processing across the multiple hardware processing elements to prune, based on the teachings of Zhang. One of ordinary skill in the art would have been motivated to make this modification in order to reduce the computation complexity, as suggested by Zhang (page 1, right column, paragraph 1).
While Zhang discloses a hardware accelerator comprising … field programmable gate arrays (FPGAs) that each comprise multiple parallel arranged hardware processing elements, they do not disclose multiple field programmable gate arrays.
Walston discloses:
multiple field programmable gate arrays (¶[0087] teaches multiple FPGAs in parallel).
Ogbogu, Zhang, and Walston are analogous art because they are from the same field of endeavor, GNNs and machine learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, to include multiple field programmable gate arrays, based on the teachings of Walston. One of ordinary skill in the art would have been motivated to make this modification in order to complete cycles in a shorter amount of time, as suggested by Walston (¶[0087]).
While Ogbogu discloses distributing the multiple subgraphs among the multiple hardware processing elements … for parallel processing, they do not disclose multiple FPGAs for parallel processing.
Walston discloses:
multiple FPGAs for parallel processing (¶[0087] teaches FPGAs in parallel for parallel processing).
Ogbogu, Zhang, and Walston are analogous art because they are from the same field of endeavor, GNNs and machine learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, to include multiple FPGAs for parallel processing, based on the teachings of Walston. One of ordinary skill in the art would have been motivated to make this modification in order to complete cycles in a shorter amount of time, as suggested by Walston (¶[0087]).
While Ogbogu, in view of Zhang, teaches employing parallel processing across the multiple hardware processing elements to prune individual subgraphs on individual hardware processing elements utilizing … algorithms, they do not disclose utilizing GraSP algorithms to do so.
GraSP discloses:
to prune individual subgraphs … utilizing GraSP algorithms (Figure 1, page 5, second paragraph, line 1 “The regional metrics are calculated over linked subgraphs in the MGE… (hence any pair of consecutive BGEs, i.e., each tripartite slice)”, Table 2, and paragraph below table 2, lines 4-6 “We considered five sparsity values to cover a broad spectrum, namely…We trained each combination of <pruning algorithm, dataset, architecture, sparsity> for 3 runs, obtaining a pool of sparse architectures” teaches pruning individual subgraphs in Figure 1 utilizing the GraSP algorithm).
Ogbogu, Zhang, Walston, and Grasp are analogous art because they are from the same field of endeavor, graphs and machine learning models.
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, in further view of Kovvuri, to include to prune individual subgraphs … utilizing GraSP algorithms, based on the teachings of Grasp. One of ordinary skill in the art would have been motivated to make this modification in order to discover the best subnetwork while maintaining cheap computation cost, as suggested by Grasp (page 3, paragraph “Pruning at initialization”).
Regarding claim 9, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 8 (and thus the rejection of claim 8 is incorporated). Ogbogu further discloses:
wherein employing parallel processing comprises performing initial training of an individual subgraph on an individual hardware processing element … (page 72:9, first paragraph, lines 1-3 “the number of subgraphs has a direct impact on the performance, reliability, and predictive accuracy of GNN model for a given GNN training workload (model and dataset) executed on the ReRAM based manycore platform.” Teaches performing initial training of an individual subgraph on an individual hardware processing element).
While Ogbogu discloses employing parallel processing comprises performing initial training of an individual subgraph on an individual hardware processing element, but does not teach doing so with the GraSP algorithm.
Grasp discloses:
performing initial training of an individual subgraph on … hardware processing element with the GraSP algorithm (Figure 1 and Table 2, and page 14, final paragraph, lines 1-3 “Each combination <dataset, architecture, sparsity, pruning algorithm> has been evaluated over 3 runs. We trained the models on a cluster with 8 NVIDIA A100 GPUs.”).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, in further view of Walston, to include performing initial training of an individual subgraph on … hardware processing element with the GraSP algorithm, based on the teachings of Grasp. One of ordinary skill in the art would have been motivated to make this modification in order to discover the best subnetwork while maintaining cheap computation cost, as suggested by Grasp (page 3, paragraph “Pruning at initialization”).
Regarding claim 10, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 9 (and thus the rejection of claim 9 is incorporated). Ogbogu further discloses:
further comprising calculating gradients of the individual subgraph (page 72:7, third paragraph, final 2 lines “which is then used to compute gradients for the weight update process.” Teaches calculating gradients of the individual subgraph).
Ogbogu teaches further comprising calculating gradients of the individual subgraph but does not disclose doing so for the GraSP algorithm.
Grasp discloses:
further comprising calculating gradients… for the GraSP algorithm (Figure 1, Table 2, and page 2, paragraph “pruning at initialization”, line 4 “GraSP [8] applies a gradient signal preservation mechanism based on Hessian-gradient product” teaches calculating gradients for the GraSP algorithm).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, in further view of Walston, to include further comprising calculating gradients… for the GraSP algorithm, based on the teachings of Grasp. One of ordinary skill in the art would have been motivated to make this modification in order to discover the best subnetwork while maintaining cheap computation cost, as suggested by Grasp (page 3, paragraph “Pruning at initialization”).
Regarding claim 11, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 10 (and thus the rejection of claim 10 is incorporated). Ogbogu further discloses:
further comprising calculating scores for neurons and connections of the individual subgraph … (page 72:7, third paragraph, final 2 lines “which is then used to compute gradients for the weight update process.” And equations (1)-(5) teaches calculating scores for neurons and connections of the individual subgraph).
Ogbogu teaches further comprising calculating scores for neurons and connections of the individual subgraph but does not disclose doing so with the GraSP algorithm.
Grasp discloses:
calculating scores for neurons and connections … with the GraSP algorithm (Figure 1, Table 2, Table 9, and Figure 10 teaches calculating scores for neurons and connections with the GraSP algorithm).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, in further view of Walston, to include calculating scores for neurons and connections … with the GraSP algorithm, based on the teachings of Grasp. One of ordinary skill in the art would have been motivated to make this modification in order to discover the best subnetwork while maintaining cheap computation cost, as suggested by Grasp (page 3, paragraph “Pruning at initialization”).
Regarding claim 12, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 11 (and thus the rejection of claim 11 is incorporated). Ogbogu further discloses:
further comprising setting an initial threshold for the individual subgraph (page 72:10, last paragraph, last line “maximum top k % subgraphs from the sorted list”, page 72:11, first paragraph, line 3 “k was chosen to be 10,” teaches setting an initial threshold of 10 for subgraphs) .
Regarding claim 13, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 12 (and thus the rejection of claim 12 is incorporated). Ogbogu further discloses:
further comprising comparing the calculated scores for the neurons and connections of the individual subgraph to the initial threshold (Algorithm 1 and page 72:10, last paragraph, last line “maximum top k % subgraphs from the sorted list based on the importance scores that can be pruned” teaches comparing the calculated scores for the neurons and connections of the individual subgraph to the initial threshold).
Regarding claim 14, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 13 (and thus the rejection of claim 13 is incorporated). Ogbogu further discloses:
further comprising pruning the neurons and/or connections of the individual subgraph having calculated scores below the initial threshold (Algorithm 1 teaches pruning the neurons and/or connections of the individual subgraph having calculated scores below the initial threshold).
Regarding claim 15, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 14 (and thus the rejection of claim 14 is incorporated). Ogbogu further discloses:
further comprising sending the pruned individual subgraph to memory that is shared by all of the processing elements (Algorithm 1 and Figure 2 teaches sending the pruned individual subgraph to a shared memory).
Regarding claim 16, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 15 (and thus the rejection of claim 15 is incorporated). Ogbogu further discloses:
retraining the pruned individual subgraph on the shared memory (Algorithm 1 and Figure 2 teaches retraining the pruned individual subgraph on the shared memory).
Regarding claim 17, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 16 (and thus the rejection of claim 16 is incorporated). Ogbogu further discloses:
further comprising evaluating performance of the retrained pruned individual subgraph (Algorithm 1 teaches evaluating performance, i.e. accuracy, of the retrained pruned individual subgraph).
Regarding claim 18, Ogbogu, in view of Zhang, in further view of Walston, in further review of Grasp, discloses the method of claim 17 (and thus the rejection of claim 17 is incorporated). Ogbogu further discloses:
wherein in an instance where the performance of the retrained pruned individual subgraph is satisfactory, further comprising integrating the retrained pruned individual subgraph with other retrained pruned individual subgraphs to form the trained and pruned GNN model (Algorithm 1 and Algorithm 2 teaches, in instances of satisfactory performance, integrating the pruned subgraph to form the GNN model),
in an alternative instance where the performance of the retrained pruned individual subgraph is not satisfactory iteratively returning to score the neurons and connections of the retrained pruned individual subgraphs … on the individual processing elements (Algorithm 1 and Algorithm 2 teaches, in instances of unsatisfactory performance, iteratively returning to score the neurons and connections of the retrained pruned individual subgraphs).
While Ogbogu discloses in an alternative instance where the performance of the retrained pruned individual subgraph is not satisfactory iteratively returning to score the neurons and connections of the retrained pruned individual subgraphs … on the individual processing elements, they do not teach doing so with the GraSP algorithms.
Grasp teaches:
iteratively .. score the neurons and connections … with the GraSP algorithms (Figure 1, Table 2, Table 9, and Figure 10 teaches iteratively calculating scores for neurons and connections with the GraSP algorithm).
It would have been obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings of Ogbogu, in view of Zhang, in further view of Walston, to include iteratively .. score the neurons and connections … with the GraSP algorithms, based on the teachings of Grasp. One of ordinary skill in the art would have been motivated to make this modification in order to discover the best subnetwork while maintaining cheap computation cost, as suggested by Grasp (page 3, paragraph “Pruning at initialization”).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
US 20240160932 A1: Zhu et al. teaches pruning, parallel processing, and FPGA.
US 10373053 B2: Barham et Al. teaches subgraphs, parallel processing, and FPGA.
Zhang et al. (“Dynasparse: Accelerating GNN Inference through Dynamic Sparsity Exploitation”) teaches FPGA and GNN models.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUMAIRA ZAHIN MAUNI whose telephone number is (703)756-5654. The examiner can normally be reached Monday - Friday, 9 am - 5 pm (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MATT ELL can be reached at (571) 270-3264. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.Z.M./Examiner, Art Unit 2141
/MATTHEW ELL/Supervisory Patent Examiner, Art Unit 2141
Prosecution Insights