DETAILED ACTION
This office action is in response to amendments filed on 02/26/2026.
Claims 1-28 have been amended. Claims 29-38 have been canceled. Claims 1-28 are pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Rejections Under 35 USC § 112:
In light of applicant’s amendments to the claims (pg. 2-6), the previous rejections under 35 USC § 112(b) have been withdrawn. However, a new rejection under 35 USC § 112(b) has been introduced for the amended claims.
Rejections Under 35 USC § 101:
Applicant's arguments regarding the rejections under 35 USC § 101 (pg. 10) have been fully considered but they are not persuasive. Applicant argues that the claimed invention is directed to a technological improvement in the field of machine learning, as the claimed neural network node permutation enables efficient neural network processing using hardware accelerators, thereby saving memory, time, and computational resources.
Examiner points to MPEP 2106.05(a), paragraph 6, which reads, “It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements… In addition, the improvement can be provided by the additional element(s) in combination with the recited judicial exception.” In this case, the improvement to neural network processing using hardware accelerators is provided by the application of permutation strategies to satisfy sparsity constraints, which, as can be seen in the rejection below, can be considered a mental process. Since the improvement is provided by the mental process (i.e. the judicial exception), it is not sufficient to integrate the judicial exception into a practical application.
The rejections under 35 USC § 101 have been updated to include the amended limitations and to clarify the reasoning given for the limitations that were not amended.
Prior Art Rejections
Applicant's arguments regarding the prior art rejections (pg. 7-9) have been fully considered but they are not persuasive. Applicant argues that the cited reference Xue fails to disclose the features of the amended independent claim. Applicant specifically argues that while Xue teaches reordering the rows and columns of an adjacency matrix, the claim is directed to reordering nodes of a neural network. Examiner respectfully notes that in Xue, rows and columns of the adjacency matrix represent nodes of a graph neural network, and thus reordering the rows and columns, as disclosed in Xue 0005-0006, amounts to reordering the nodes of a neural network.
The prior art rejections have been updated to include the amended limitations and to clarify the reasoning given for the limitations that were not amended.
Claim Objections
Claims 5, 9, and 22 are objected to because of the following informalities:
In claim 5, “…permutations comprise a comprise a greedy block swap…”
In claim 9, “…includes a a structured sparsity constraint…”
In claim 22, “…wherein wherein the one or more hardware…”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 18, 27 and 28 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 18, 27 and 28 each recite the limitation “the neural network graph data”. There is insufficient antecedent basis for this limitation in the claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-28 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1:
Step 1: The claim is directed to a processor, which falls within the statutory category of a
machine/manufacture.
Step 2A Prong 1: The claim is directed to an abstract idea. Specifically, the claim recites:
apply one or more permutation strategies to change an order of nodes in one or more neural networks to satisfy one or more sparsity constraints of one or more hardware accelerators comprised of at least a subset of the one or more circuits; and (Abstract idea – mental process. Applying permutation strategies to change the order of neural network nodes based on sparsity constraints can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and reordering its rows and columns by hand to ensure a minimum number of zeroes per quadrant. The courts have recognized that claims can recite a mental process even if they are claimed as being performed on a computer. See MPEP 2106.04(a)(2)(III).)
Step 2A Prong 2: The additional elements recited in the claim do not integrate the abstract idea into a practical application, individually or in combination. Specifically, the claim recites the additional elements:
A processor comprising: one or more circuits (This limitation is interpreted as a generic computing environment, and thus amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
cause the changed order of nodes in the one or more neural networks to be processed using the one or more hardware accelerators comprised of at least the subset of the one or more circuits. (Processing nodes of a neural network using a hardware accelerator amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. Specifically, the claim recites the additional elements:
A processor comprising: one or more circuits (This limitation is interpreted as a generic computing environment, and thus amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
cause the changed order of nodes in the one or more neural networks to be processed using the one or more hardware accelerators comprised of at least the subset of the one or more circuits. (Processing nodes of a neural network using a hardware accelerator amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f).)
Claims 2-28:
Claim 2 recites The processor of claim 1, wherein the one or more neural networks are one or more graph neural networks. This limitation merely qualifies the data upon which the mental process is performed as data representing a graph neural network, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea. Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 3 recites The processor of claim 1, wherein at least one of the one or more sparsity constraints is an M:N structured sparsity constraint comprising partitioning of data into N-element sets every partition that contains at most M non-zero values. Satisfying an M:N structured sparsity constraint in neural network data is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and reordering its rows and columns by hand such that at most two out of each set of four contiguous elements in each row are non-zero. Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 4 recites The processor of claim 1, wherein the one or more permutations comprise a random swap of different nodes of the one or more neural network networks. Performing a random swap of neural network nodes is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and randomly swapping its rows by hand. Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 5 recites The processor of claim 1, wherein the one or more permutations comprise a comprise a greedy block swap of different ones of the nodes of the one or more neural networks. Performing a greedy block swap of neural network nodes is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and swapping groups of its rows by hand to improve sparsity. Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 6 recites The processor of claim 1, wherein the one or more permutations comprise a greedy channel swap of different ones of the nodes of the one or more neural networks. Performing a greedy channel swap of neural network nodes is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and swapping its rows by hand to improve sparsity. Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 7 recites The processor of claim 1, wherein the one or more neural networks comprise a plurality of layers. Generic neural networks comprising a plurality of layers are standard in the field of machine learning, and thus this limitation amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f). Therefore, the claim does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 8 recites The processor of claim 1, wherein the one or more circuits are to cause a graph neural network represented by the one or more neural networks to be subdivided into a plurality of subgraphs. This amounts to adding insignificant extra-solution activity to the judicial exception – see MPEP2106.05(g). Further, dividing a graph neural network into subgraphs is well-understood, routine, and conventional in the field of machine learning, per Besta: “Some graphs may have more than 250 billion vertices and beyond 10 trillion edges [18], [147], and each vertex and/or edge may have a large associated feature vector [110]. Thus, one inevitably must distribute such graphs over different workers as they do not fit into one server memory. We refer to this form of GNN parallelism as the graph partition parallelism… Graph partition parallelism is commonly used to alleviate large memory requirements of full-batch training” (Besta et al., “Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis”, pg. 7-8, section 3.1). See MPEP 2106.05(d)(II). Therefore, the claim does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 9 recites The processor of claim 1, wherein at least one of the one or more sparsity constraints includes a structured sparsity constraint. According to specification paragraph 0063 of the instant application, “’2:4’ sparsity indicates that, at most, two out of each set of four contiguous elements in a row of a matrix are non-zero, for every row of a matrix… In at least one embodiment, ‘2:4’ sparsity is referred to as ‘2:4 structured sparsity.’” Satisfying a structured sparsity constraint in neural network data is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and reordering its rows and columns by hand to enforce 2:4 sparsity. Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 10 recites The processor of claim 1, wherein at least one of the one or more sparsity constraints includes a fine-grained structured sparsity constraint. According to specification paragraph 0063 of the instant application, “’2:4’ sparsity indicates that, at most, two out of each set of four contiguous elements in a row of a matrix are non-zero, for every row of a matrix… In at least one embodiment, ‘2:4’ sparsity is referred to as ‘2:4 fine-grained structured sparsity.’” Satisfying a structured sparsity constraint in neural network data is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and reordering its rows and columns by hand to enforce 2:4 sparsity. Therefore, the claim merges with the abstract idea recited in claim 1, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 11 is a method claim containing substantially the same elements as system claim 1, and is rejected on the same grounds under 35 U.S.C 101 as claim 1.
The additional components of A computer-implemented method are interpreted as a general-purpose computer and mere instructions to apply the judicial exception on the computer. Therefore, the claims do not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 12 recites The computer-implemented method of claim 11, wherein the one or more neural networks are one or more graph neural networks. This limitation merely qualifies the data upon which the mental process is performed as data representing a graph neural network, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea. Therefore, the claim merges with the abstract idea recited in claim 11, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 13 recites The computer-implemented method of claim 11, wherein the one or more hardware accelerators comprised of the least a subset of the one or more circuits are one or more graphics processing units (GPUs). Processing neural network data using a GPU amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f). Therefore, the claim does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 14 recites The computer-implemented method of claim 11, wherein the one or more permutations comprise a guided greedy search. Performing a guided greedy search on neural network data is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper, noting the effects on sparsity of potential row swaps, and performing the best row swap by hand. Therefore, the claim merges with the abstract idea recited in claim 11, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 15 recites The computer-implemented method of claim 11, wherein the one or more permutations comprise an exhaustive search. Performing an exhaustive search on neural network data is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper, noting the effects on sparsity of every possible row swap, and performing the best row swap by hand. Therefore, the claim merges with the abstract idea recited in claim 11, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 16 recites The computer-implemented method of claim 11, wherein the one or more permutations comprise a greedy block swap of different nodes of the one or more neural network networks. Performing a greedy block swap of neural network nodes is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and swapping groups of its rows by hand to improve sparsity. Therefore, the claim merges with the abstract idea recited in claim 11, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 17 recites The computer-implemented method of claim 11, performing the applying the one or more permutations and the causing the changed order of the nodes responsive to an invocation of an application programming interface (API) that at least specifies the one or more neural networks and the one or more sparsity constraints. Invoking an API to receive the neural network data and sparsity constraints amounts to adding insignificant extra-solution activity (necessary data gathering) to the judicial exception – see MPEP2106.05(g). Further, the limitation is directed to receiving or transmitting data over a network, which the courts have found to be well-understood, routine, and conventional in the computer arts. See MPEP 2106.05(d)(II). Therefore, the claim does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 18 recites The computer-implemented method of claim 11, wherein applying the one or more permutation strategies comprises relabeling one or more elements of the neural network graph data. Relabeling elements of neural network graph data is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing the labeled neural network graph data on a sheet of paper and altering its labels by hand. Therefore, the claim merges with the abstract idea recited in claim 11, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 19 recites The computer-implemented method of claim 11, wherein the one or more sparsity constraints includes a constraint on a maximum number of non-zero values of a set of values of an adjacency matrix of the one or more neural networks. Constraining sparsity of neural network adjacency matrix values based on a maximum number of non-zero values is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix on a sheet of paper and altering its values by hand to limit the number of non-zero values. Therefore, the claim merges with the abstract idea recited in claim 11, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 20 is a system claim containing substantially the same elements as system claim 1, and is rejected on the same grounds under 35 U.S.C 101 as claim 1.
The additional components of A computer system comprising: one or more processors and memory storing executable instructions are interpreted as a general-purpose computer and mere instructions to apply the judicial exception on the computer. Therefore, the claims do not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 21 recites The computer system of claim 20, wherein the one or more neural networks are one or more graph neural networks. This limitation merely qualifies the data upon which the mental process is performed as data representing a graph neural network, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea. Therefore, the claim merges with the abstract idea recited in claim 20, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 22 recites The computer system of claim 20, wherein the one or more hardware accelerators comprised of at least the subset of the one or more circuits are one or more graphics processor units (GPUs). Processing neural network data using a GPU amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea – see MPEP 2106.05(f). Therefore, the claim does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 23 recites The computer system of claim 20, wherein the one or more processors and memory storing executable instructions that, if performed by the one or more processors, are to cause a graph neural network represented by the one or more neural networks to be subdivided into a plurality of subgraphs. This amounts to adding insignificant extra-solution activity to the judicial exception – see MPEP2106.05(g). Further, dividing a graph neural network into subgraphs is well-understood, routine, and conventional in the field of machine learning, per Besta: “Some graphs may have more than 250 billion vertices and beyond 10 trillion edges [18], [147], and each vertex and/or edge may have a large associated feature vector [110]. Thus, one inevitably must distribute such graphs over different workers as they do not fit into one server memory. We refer to this form of GNN parallelism as the graph partition parallelism… Graph partition parallelism is commonly used to alleviate large memory requirements of full-batch training” (Besta et al., “Parallel and Distributed Graph Neural Networks: An In-Depth Concurrency Analysis”, pg. 7-8, section 3.1). See MPEP 2106.05(d)(II). Therefore, the claim does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 24 recites The computer system of claim 20, wherein the one or more processors and memory storing executable instructions that, if performed by the one or more processors, are to invoke an application programming interface (API) that at least specifies the one or more neural networks and the one or more sparsity constraints that causes performance of the application of the one or more permutation strategies and the cause of the changed order of the nodes in the one or more neural networks to be processed. Invoking an API to receive the neural network data and sparsity constraints amounts to adding insignificant extra-solution activity (necessary data gathering) to the judicial exception – see MPEP2106.05(g). Further, the limitation is directed to receiving or transmitting data over a network, which the courts have found to be well-understood, routine, and conventional in the computer arts. See MPEP 2106.05(d)(II). Therefore, the claim does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 25 recites The computer system of claim 20, wherein the one or more permutations comprise a random swap. Performing a random swap of neural network nodes is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and randomly swapping its rows by hand. Therefore, the claim merges with the abstract idea recited in claim 20, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 26 recites The computer system of claim 20, wherein the one or more permutations comprise a greedy channel swap. Performing a greedy channel swap of neural network nodes is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and swapping its rows by hand to improve sparsity. Therefore, the claim merges with the abstract idea recited in claim 20, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 27 recites The computer system of claim 20, wherein the one or more permutations comprise a greedy block swap of the neural network graph data. Performing a greedy block swap of neural network nodes is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper and swapping groups of its rows by hand to improve sparsity. Therefore, the claim merges with the abstract idea recited in claim 20, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim 28 recites The computer system of claim 20, wherein the neural network graph data is to be organized by performing a heuristic guided greedy search of the neural network graph data. Performing a heuristic guided greedy search on neural network data is an abstract idea (mental process) because it can practically be performed in the human mind or with the aid of pen and paper, for example, by viewing an adjacency/weight matrix representing a graph neural network on a sheet of paper, noting the effects on sparsity of potential row swaps, and performing the best row swap by hand. Therefore, the claim merges with the abstract idea recited in claim 20, and does not recite additional elements that are sufficient to amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-2, 7-8, 11-14, 17-18, 20-24, and 28 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by
Xue et al. (hereinafter Xue), U.S. Patent Application Publication US 20220343145 A1.
Regarding Claim 1,
Xue teaches A processor comprising: one or more circuits to: ([0104]: “Each process, methods, and algorithms described in the preceding sections may be embodied in, and fully or partially automated by, code modules executed by one or more computer systems or computer processors comprising computer hardware. The processes and algorithms may be implemented partially or wholly in application-specific circuit.”)
apply one or more permutation strategies to change an order of nodes in one or more neural networks to satisfy one or more sparsity constraints of one or more hardware accelerators comprised of at least a subset of the one or more circuits; and (0005-0007: “According to one aspect, a hardware accelerator for accelerating Graph Neural Network (GNN) computations is described. The hardware accelerator may include a matrix partitioning circuit configured to partition an adjacency matrix of an input graph for GNN computations into a plurality of sub-matrices; a sub-matrix reordering circuit configured to reorder rows and columns of the plurality of sub-matrices… the sub-matrix reordering circuit is further configured to reorder the rows and the columns of the plurality of sub-matrices based on a number of non-zero values in each of the rows and the columns… the one or more processors for performing the GNN computations are configured in different computation modes optimized for processing data sets with different levels of sparsity.” Rows and columns of a GNN adjacency matrix (i.e. nodes of a neural network) are reordered based on a number of non-zero values (i.e. a sparsity constraint) for a GNN computation processor (i.e. hardware accelerator).)
cause the changed order of nodes in the one or more neural networks to be processed using the one or more hardware accelerators comprised of at least the subset of the one or more circuits. (0005: “The hardware accelerator may include… a tile partitioning circuit configured to divide the plurality of sub-matrices with reordered rows and columns into a plurality of tiles based on processing granularities of one or more processors; and a tile distributing circuit configured to distribute the plurality of tiles to the one or more processors for performing the GNN computations.” The adjacency matrices with reordered rows and columns (i.e. the changed order of nodes in the neural networks) are distributed to the GNN computation processors (i.e. hardware accelerators) for performing computations (i.e. processing).)
Regarding Claim 2, Xue teaches The processor of claim 1, as shown above.
Xue also teaches wherein the one or more neural networks are one or more graph neural networks. (0005: “The hardware accelerator may include a matrix partitioning circuit configured to partition an adjacency matrix of an input graph for GNN computations into a plurality of sub-matrices; a sub-matrix reordering circuit configured to reorder rows and columns of the plurality of sub-matrices;” The adjacency matrix represents a GNN (graph neural network).)
Regarding Claim 7, Xue teaches The processor of claim 1, as shown above.
Xue also teaches wherein the one or more neural networks comprise a plurality of layers. (0068: “During an inferencing process using the trained GNN, the input graph 301 may go through the same path through the plurality of GNN layers 302 and eventually be transformed into the output graph 305 with transformed node features.” The GNN (i.e. neural network) comprises a plurality of GNN layers.)
Regarding Claim 8, Xue teaches The processor of claim 1, as shown above.
Xue also teaches wherein the one or more circuits are to cause a graph neural network represented by the one or more neural networks to be subdivided into a plurality of subgraphs. (0072-0074: “At step 402, a row/column reorder and partition unit may be configured to (1) partition the received adjacency matrix representing the input graph of a plurality of nodes into a plurality of sub-matrices based on feature similarities among the plurality of nodes, and (2) reorder rows and columns of each of the plurality of sub-matrices based on a number of non-zero values in each of the rows and columns… After the row/column reordering, a plurality of fine-tuned sub-matrices may be obtained at step 403, which may be referred to as subgraph adjacency matrices.” The adjacency matrix representing the graph neural network is divided into subgraph adjacency matrices (i.e. subgraphs).)
Claim 11 is a method claim containing substantially the same elements as system claim 1. Xue teaches the elements of claim 1, as shown above.
Xue also teaches A computer-implemented method (0104: “Each process, methods, and algorithms described in the preceding sections may be embodied in, and fully or partially automated by, code modules executed by one or more computer systems or computer processors comprising computer hardware. The processes and algorithms may be implemented partially or wholly in application-specific circuit.”)
Regarding Claim 12, Xue teaches The computer-implemented method of claim 11, as shown above.
Xue also teaches wherein the one or more neural networks are one or more graph neural networks. (0005: “The hardware accelerator may include a matrix partitioning circuit configured to partition an adjacency matrix of an input graph for GNN computations into a plurality of sub-matrices; a sub-matrix reordering circuit configured to reorder rows and columns of the plurality of sub-matrices;” The adjacency matrix represents a GNN (graph neural network).)
Regarding Claim 13, Xue teaches The computer-implemented method of claim 11, as shown above.
Xue also teaches wherein the one or more hardware accelerators comprised of the least a subset of the one or more circuits are one or more graphics processing units (GPUs). (0021: “In some embodiments, the one or more processors comprise a graphic processing unit (GPU)…” The GNN computation processor (i.e. hardware accelerator) can be a GPU.)
Regarding Claim 14, Xue teaches The computer-implemented method of claim 11, as shown above.
Xue also teaches wherein the one or more permutations comprise a guided greedy search. (0082: “Steps 503 to 505 illustrate a process to obtain a sub-matrix. At step 503, a node may be randomly selected from the graph. At step 504, a plurality of feature similarity scores between the node and each different node in the graph may be determined. In some embodiments, each of the feature similarity scores between two nodes is determined based on a distance between two feature vectors of the two nodes. Based on the plurality of feature similarity scores, m−1 highest feature similarity scores corresponding to m−1 nodes may be selected. Then a sub-matrix may be constructed based on the m modes, i.e., the randomly selected node and the m−1 similar nodes.” The graph neural network nodes are split into submatrices by selecting a random node and searching for similar nodes. The similarity scores for each node are a heuristic, and the search for the m−1 closest nodes is guided by this heuristic. The search is greedy because for each randomly selected node, its similar nodes are selected without backtracking or considering how the selection will affect the unselected nodes.
Regarding Claim 17, Xue teaches The computer-implemented method of claim 11, as shown above.
Xue also teaches performing the applying the one or more permutations and the causing the changed order of the nodes responsive to an invocation of an application programming interface (API) that at least specifies the one or more neural networks and the one or more sparsity constraints. (0111: “Moreover, the one or more processors may also operate to support performance of the relevant operations in a ‘cloud computing’ environment or as a ‘software as a service’ (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an Application Program Interface (API)).”)
Regarding Claim 18, Xue teaches The computer-implemented method of claim 11, as shown above.
Xue also teaches wherein applying the one or more permutation strategies comprises relabeling one or more elements of the neural network graph data. ([0066]: “The feature vectors of the nodes in the input graph 301 may be collectively represented as a feature matrix.” [0078]: “In some embodiments, similar preprocessing steps may also apply to a feature matrix of the input graph. For example, a row reordering (e.g., swapping row A and B) in the adjacency matrix may correspond to a column reordering (e.g., swapping column A and B) in the feature matrix, and a column reordering in the adjacency matrix may correspond to a row reordering in the feature matrix.” The feature matrix represents labels for the nodes (i.e. elements) of the input graph (i.e. neural network graph data), and reordering the rows and columns of the feature matrix amounts to relabeling the nodes.)
Claim 20 is a system claim containing substantially the same elements as system claim 1. Xue teaches the elements of claim 1, as shown above.
Xue also teaches A computer system comprising: one or more processors and memory storing executable instructions (0104: “Each process, methods, and algorithms described in the preceding sections may be embodied in, and fully or partially automated by, code modules executed by one or more computer systems or computer processors comprising computer hardware. The processes and algorithms may be implemented partially or wholly in application-specific circuit.” 0109: “The algorithm may be comprised in program codes or instructions stored in a memory…”)
Regarding Claim 21, Xue teaches The computer system of claim 20, as shown above.
Xue also teaches wherein the one or more neural networks are one or more graph neural networks. (0005: “The hardware accelerator may include a matrix partitioning circuit configured to partition an adjacency matrix of an input graph for GNN computations into a plurality of sub-matrices; a sub-matrix reordering circuit configured to reorder rows and columns of the plurality of sub-matrices;” The adjacency matrix represents a GNN (graph neural network).)
Regarding Claim 22, Xue teaches The computer system of claim 20, as shown above.
Xue also teaches wherein the one or more hardware accelerators comprised of at least the subset of the one or more circuits are one or more graphics processor units (GPUs). (0021: “In some embodiments, the one or more processors comprise a graphic processing unit (GPU)…” The GNN computation processor (i.e. hardware accelerator) can be a GPU.)
Regarding Claim 23, Xue teaches The computer system of claim 20, as shown above.
Xue also teaches wherein the one or more processors and memory storing executable instructions that, if performed by the one or more processors, are to cause a graph neural network represented by the one or more neural networks to be subdivided into a plurality of subgraphs. (0072-0074: “At step 402, a row/column reorder and partition unit may be configured to (1) partition the received adjacency matrix representing the input graph of a plurality of nodes into a plurality of sub-matrices based on feature similarities among the plurality of nodes, and (2) reorder rows and columns of each of the plurality of sub-matrices based on a number of non-zero values in each of the rows and columns… After the row/column reordering, a plurality of fine-tuned sub-matrices may be obtained at step 403, which may be referred to as subgraph adjacency matrices.” The adjacency matrix representing the graph neural network is divided into subgraph adjacency matrices (i.e. subgraphs).)
Regarding Claim 24, Xue teaches The computer system of claim 20, as shown above.
Xue also teaches wherein the one or more processors and memory storing executable instructions that, if performed by the one or more processors, are to invoke an application programming interface (API) that at least specifies the one or more neural networks and the one or more sparsity constraints that causes performance of the application of the one or more permutation strategies and the cause of the changed order of the nodes in the one or more neural networks to be processed. (0111: “Moreover, the one or more processors may also operate to support performance of the relevant operations in a ‘cloud computing’ environment or as a ‘software as a service’ (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an Application Program Interface (API)).”)
Regarding Claim 28, Xue teaches The computer system of claim 20, as shown above.
Xue also teaches wherein the neural network graph data is to be organized by performing a heuristic guided greedy search of the neural network graph data. (0082: “Steps 503 to 505 illustrate a process to obtain a sub-matrix. At step 503, a node may be randomly selected from the graph. At step 504, a plurality of feature similarity scores between the node and each different node in the graph may be determined. In some embodiments, each of the feature similarity scores between two nodes is determined based on a distance between two feature vectors of the two nodes. Based on the plurality of feature similarity scores, m−1 highest feature similarity scores corresponding to m−1 nodes may be selected. Then a sub-matrix may be constructed based on the m modes, i.e., the randomly selected node and the m−1 similar nodes.” The graph neural network nodes (i.e. neural network graph data) are split into submatrices (i.e. organized) by selecting a random node and searching for similar nodes. The similarity scores for each node are a heuristic, and the search for the m−1 closest nodes is guided by this heuristic. The search is greedy because for each randomly selected node, its similar nodes are selected without backtracking or considering how the selection will affect the unselected nodes.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 3, 9, 10, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Xue in view of
Nvidia, “Nvidia a100 tensor core gpu architecture”.
Regarding Claim 3, Xue teaches The processor of claim 1, as shown above.
Xue does not appear to explicitly disclose wherein at least one of the one or more sparsity constraints is an M:N structured sparsity constraint comprising partitioning of data into N-element sets every partition that contains at most M non-zero values.
However, Nvidia teaches wherein at least one of the one or more sparsity constraints is an M:N structured sparsity constraint comprising partitioning of data into N-element sets every partition that contains at most M non-zero values. (Pg. 31, para. 3-6: “Fine grained structured sparsity imposes a constraint on the allowed sparsity pattern, making it more efficient for hardware to do the necessary alignment of input operands… Structure is enforced through a new 2:4 sparse matrix definition that allows two non-zero values in every four-entry vector.” Sparsity is constrained by partitioning data into four-entry vectors (i.e. N-element sets) which each contain up to two non-zero values (i.e. M non-zero values).)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Xue and Nvidia. Xue teaches accelerating graph neural network computations by partitioning and reordering graph data to satisfy sparsity constraints. Nvidia teaches a fine-grained M:N structured sparsity constraint for deep neural networks. One of ordinary skill would have motivation to combine Xue and Nvidia because the fine-grained structured sparsity constraint “doubles compute throughput for deep neural networks,” and “does not impact the accuracy” (Nvidia, pg. 31, para. 1-3).
Regarding Claim 9, Xue teaches The processor of claim 1, as shown above.
Xue does not appear to explicitly disclose wherein at least one of the one or more sparsity constraints includes a a structured sparsity constraint.
However, Nvidia teaches wherein at least one of the one or more sparsity constraints includes a a structured sparsity constraint. (Pg. 31, para. 3: “Fine grained structured sparsity imposes a constraint on the allowed sparsity pattern, making it more efficient for hardware to do the necessary alignment of input operands.”)
Regarding Claim 10, Xue teaches The processor of claim 1, as shown above.
Xue does not appear to explicitly disclose wherein at least one of the one or more sparsity constraints includes a fine-grained structured sparsity constraint.
However, Nvidia teaches wherein at least one of the one or more sparsity constraints includes a fine-grained structured sparsity constraint. (Pg. 31, para. 3: “Fine grained structured sparsity imposes a constraint on the allowed sparsity pattern, making it more efficient for hardware to do the necessary alignment of input operands.”)
Regarding Claim 19, Xue teaches The computer-implemented method of claim 11, as shown above.
Xue does not appear to explicitly disclose wherein the one or more sparsity constraints includes a constraint on a maximum number of non-zero values of a set of values of an adjacency matrix of the one or more neural networks.
However, Nvidia teaches wherein the one or more sparsity constraints includes a constraint on a maximum number of non-zero values of a set of values of an adjacency matrix of the one or more neural networks. (Pg. 31, para. 3-6: “Fine grained structured sparsity imposes a constraint on the allowed sparsity pattern, making it more efficient for hardware to do the necessary alignment of input operands… Structure is enforced through a new 2:4 sparse matrix definition that allows two non-zero values in every four-entry vector.” Each four-entry vector is a set of values, and the set is constrained to a maximum of two non-zero values.)
Claims 4 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over Xue in view of
Ogielski et al. (hereinafter Ogielski), “Sparse Matrix Computations on Parallel Processor Arrays”.
Regarding Claim 4, Xue teaches The processor of claim 1, as shown above.
Xue does not appear to explicitly disclose wherein the one or more permutations comprise a random swap of different nodes of the one or more neural network networks.
However, Ogielski teaches wherein the one or more permutations comprise a random swap of different nodes of the one or more neural network networks. (Pg. 2-3, section 1: “Suppose that before the loading a random permutation of matrix rows and a random permutation of matrix columns are performed, and then the permuted matrix is partitioned into blocks of size
[
M
/
m
]
×
[
N
/
n
]
(the rightmost and lowest blocks may be smaller). This results in an
m
×
n
matrix of blocks which are assigned in the natural way to the
m
×
n
processor array.” Rows and columns of the matrix (i.e. neural network nodes) are randomly permuted (i.e. randomly swapped) to balance the load.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Xue and Ogielski. Xue teaches accelerating graph neural network computations by partitioning and reordering graph data to satisfy sparsity constraints. Ogielski teaches load balancing to speed up sparse matrix computations on parallel systems. One of ordinary skill would have motivation to combine Xue and Ogielski because “Efficient computation in the data-parallel mode is achieved with distributed data structures that balance the processors' computation load and promote maximum parallelism of data communications” (Ogielski, pg. 1, section 1). Further, Ogielski’s scheme “guarantees the upper limit on the dimension of the submatrices allocated to each processor. This simplifies memory management and algorithm design for SIMD computers.” (Ogielski, pg. 10, section 3).
Regarding Claim 25, Xue teaches The computer system of claim 20, as shown above.
Xue does not appear to explicitly disclose wherein the one or more permutations comprise a random swap.
However, Ogielski teaches wherein the one or more permutations comprise a random swap. (Pg. 2-3, section 1: “Suppose that before the loading a random permutation of matrix rows and a random permutation of matrix columns are performed, and then the permuted matrix is partitioned into blocks of size
[
M
/
m
]
×
[
N
/
n
]
(the rightmost and lowest blocks may be smaller). This results in an
m
×
n
matrix of blocks which are assigned in the natural way to the
m
×
n
processor array.” Rows and columns of the matrix (i.e. neural network nodes) are randomly permuted (i.e. randomly swapped) to balance the load.)
Claims 5-6, 16, and 26-27 are rejected under 35 U.S.C. 103 as being unpatentable over Xue in view of
Nastea et al. (hereinafter Nastea), “Load-Balancing in Sparse Matrix-Vector Multiplication”.
Regarding Claim 5, Xue teaches The processor of claim 1, as shown above.
Xue does not appear to explicitly disclose wherein the one or more permutations comprise a comprise a greedy block swap of different ones of the nodes of the one or more neural networks.
However, Nastea teaches wherein the one or more permutations comprise a comprise a greedy block swap of different ones of the nodes of the one or more neural networks. (Pg. 219, section 2: “An iterative load-balancing algorithm is introduced by Aliaga and Hernandez [1] in a paper that considers also a sparse matrix - vector multiplication problem. Their algorithm generates swaps of matrix rows among processors to gradually smooth minima and maxima of load. This algorithm, henceforth referenced as Aliaga, comprises three steps: Sorting: rows are sorted according to the number of non-zero elements they contain; Initial allocation: rows are mapped onto processors; Adjustment: an iterative swapping process between processors with the smallest and the largest buckets is carried out. The data swapping process continues as long as the operation brings both of the buckets towards the average bucket size.” Rows (i.e. blocks) of the matrix (i.e. neural network nodes) are swapped to balance the load. The swapping process is greedy because each iteration only considers how its own swap will affect the distribution, and does not backtrack to find an optimal solution.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Xue and Nastea. Xue teaches accelerating graph neural network computations by partitioning and reordering graph data to satisfy sparsity constraints. Nastea teaches load balancing to speed up sparse matrix computations on parallel systems. One of ordinary skill would have motivation to combine Xue and Nastea because “Load balancing is a key issue in sparse matrix computations given the diversity of sparse matrix non-zero distributions encountered in real-life problems” (Nastea, pg. 219, section 2).
Regarding Claim 6, Xue teaches The processor of claim 1, as shown above.
Xue does not appear to explicitly disclose wherein the one or more permutations comprise a greedy channel swap of different ones of the nodes of the one or more neural networks.
However, Nastea teaches wherein the one or more permutations comprise a greedy channel swap of different ones of the nodes of the one or more neural networks. (Pg. 219, section 2: “An iterative load-balancing algorithm is introduced by Aliaga and Hernandez [1] in a paper that considers also a sparse matrix - vector multiplication problem. Their algorithm generates swaps of matrix rows among processors to gradually smooth minima and maxima of load. This algorithm, henceforth referenced as Aliaga, comprises three steps: Sorting: rows are sorted according to the number of non-zero elements they contain; Initial allocation: rows are mapped onto processors; Adjustment: an iterative swapping process between processors with the smallest and the largest buckets is carried out. The data swapping process continues as long as the operation brings both of the buckets towards the average bucket size.” Rows (i.e. channels) of the matrix (i.e. neural network nodes) are swapped to balance the load. The swapping process is greedy because each iteration only considers how its own swap will affect the distribution, and does not backtrack to find an optimal solution.)
Regarding Claim 16, Xue teaches The computer-implemented method of claim 11, as shown above.
Xue does not appear to explicitly disclose wherein the one or more permutations comprise a greedy block swap of different nodes of the one or more neural network networks.
However, Nastea teaches wherein the one or more permutations comprise a greedy block swap of different nodes of the one or more neural network networks. (Pg. 219, section 2: “An iterative load-balancing algorithm is introduced by Aliaga and Hernandez [1] in a paper that considers also a sparse matrix - vector multiplication problem. Their algorithm generates swaps of matrix rows among processors to gradually smooth minima and maxima of load. This algorithm, henceforth referenced as Aliaga, comprises three steps: Sorting: rows are sorted according to the number of non-zero elements they contain; Initial allocation: rows are mapped onto processors; Adjustment: an iterative swapping process between processors with the smallest and the largest buckets is carried out. The data swapping process continues as long as the operation brings both of the buckets towards the average bucket size.” Rows (i.e. blocks) of the matrix (i.e. neural network nodes) are swapped to balance the load. The swapping process is greedy because each iteration only considers how its own swap will affect the distribution, and does not backtrack to find an optimal solution.)
Regarding Claim 26, Xue teaches The computer system of claim 20, as shown above.
Xue does not appear to explicitly disclose wherein the one or more permutations comprise a greedy channel swap.
However, Nastea teaches wherein the one or more permutations comprise a greedy channel swap. (Pg. 219, section 2: “An iterative load-balancing algorithm is introduced by Aliaga and Hernandez [1] in a paper that considers also a sparse matrix - vector multiplication problem. Their algorithm generates swaps of matrix rows among processors to gradually smooth minima and maxima of load. This algorithm, henceforth referenced as Aliaga, comprises three steps: Sorting: rows are sorted according to the number of non-zero elements they contain; Initial allocation: rows are mapped onto processors; Adjustment: an iterative swapping process between processors with the smallest and the largest buckets is carried out. The data swapping process continues as long as the operation brings both of the buckets towards the average bucket size.” Rows (i.e. channels) of the matrix (i.e. neural network nodes) are swapped to balance the load. The swapping process is greedy because each iteration only considers how its own swap will affect the distribution, and does not backtrack to find an optimal solution.)
Regarding Claim 27, Xue teaches The computer system of claim 20, as shown above.
Xue does not appear to explicitly disclose wherein the one or more permutations comprise a greedy block swap of the neural network graph data.
However, Nastea teaches wherein the one or more permutations comprise a greedy block swap of the neural network graph data. (Pg. 219, section 2: “An iterative load-balancing algorithm is introduced by Aliaga and Hernandez [1] in a paper that considers also a sparse matrix - vector multiplication problem. Their algorithm generates swaps of matrix rows among processors to gradually smooth minima and maxima of load. This algorithm, henceforth referenced as Aliaga, comprises three steps: Sorting: rows are sorted according to the number of non-zero elements they contain; Initial allocation: rows are mapped onto processors; Adjustment: an iterative swapping process between processors with the smallest and the largest buckets is carried out. The data swapping process continues as long as the operation brings both of the buckets towards the average bucket size.” Rows (i.e. blocks) of the matrix (i.e. neural network nodes) are swapped to balance the load. The swapping process is greedy because each iteration only considers how its own swap will affect the distribution, and does not backtrack to find an optimal solution.)
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Xue in view of
Mishra et al. (hereinafter Mishra), “Accelerating Sparse Deep Neural Networks” (published 04/16/2021).
Regarding Claim 15, Xue teaches The computer-implemented method of claim 11, as shown above.
Xue does not appear to explicitly disclose wherein the one or more permutations comprise an exhaustive search.
However, Mishra teaches wherein the one or more permutations comprise an exhaustive search. (Pg. 7, figure 6: “Permuting columns of a weight matrix prior to pruning the matrix can reduce the effect of the 2:4 sparsity constraint on weight magnitude.” Pg. 14, section 6: “The PyTorch ASP [36] library provides a simple greedy approach, as well as an exhaustive search, that seeks to minimize the weight magnitude lost by pruning;” An exhaustive search is provided to permute the adjacency/weight matrix such that the 2:4 sparsity constraint can be satisfied.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the present application to combine Xue and Mishra. Xue teaches accelerating graph neural network computations by partitioning and reordering graph data to satisfy sparsity constraints. Mishra teaches accelerating neural network computations by permuting and pruning the weight matrix to satisfy sparsity constraints. One of ordinary skill would have motivation to combine Xue and Mishra because “2:4 sparsity…enables hardware to fully-utilize large memory reads” (Mishra, pg. 4, section 3.1), however, achieving 2:4 sparsity via pruning can result in a significant loss in weight magnitude. The disclosed exhaustive search “minimize[s] the weight magnitude lost by pruning” (Mishra, pg. 14, section 6), enabling the neural network to “maintain accuracy while achieving inference speedup” (Mishra, pg. 3, section 2).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BENJAMIN M ROHD whose telephone number is (571)272-6445. The examiner can normally be reached Mon-Thurs 8:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Viker Lamardo can be reached at (571) 270-5871. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.M.R./Examiner, Art Unit 2147 /VIKER A LAMARDO/Supervisory Patent Examiner, Art Unit 2147
Prosecution Insights