DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the amendments filed 03 July 2025. Claims 2-6 and 11-20 are cancelled. Claims 1 and 7-10 are amended. Claims 21-30 are newly added. Claims 1, 7-10, and 21-30 are pending and have been examined.
Response to Arguments
Applicant’s arguments, see page 8, filed 03 July 2025, with respect to the rejection of Claims 11-20 under 35 U.S.C. 112(b) have been fully considered and are persuasive. The rejection of Claims 11-20 under 35 U.S.C. 112(b) has been withdrawn.
APPLICANT'S ARGUMENT: Applicant appears to argue (page 8, paragraph 4) that "Applicant does not necessarily agree with the Examiner, to expedite allowance of this Application, Applicant has canceled claims 11 and 18, and added new claims 21-30."
EXAMINER'S RESPONSE: The rejection of Claims 11-20 under 35 U.S.C. 112(b) has been withdrawn in view of arguments/amendments.
Applicant's arguments, see pages 8-11, filed 03 July 2025, with respect to the rejection of Claim 1 under 35 U.S.C. 101 have been fully considered but they are not persuasive.
APPLICANT'S ARGUMENT: Applicant appears to argue (page 9, paragraphs 3-4) that the features of amended Claim 1 "implement a structured separation of GNN processing stages across multiple distinct hardware components ....The GNN accelerator ... autonomously accesses external memory using direct memory access (DMA)-a hardware-specific operation that is not generic or software-executed. ... This architectural division and local buffering enables the GNN accelerator to execute a multi-stage sampling pipeline without CPU intervention, and cannot be performed in the human mind. ¶ These additional elements involve specific, non-conventional processor interactions and memory control mechanisms that are grounded in specialized hardware implementation. Accordingly, the claim as a whole is not directed to a judicial exception, and Step 2A, Prong One is not met."
EXAMINER'S RESPONSE: Examiner respectfully disagrees. Amended Claim 1 includes several limitations that appear to recite mental concepts that may be performed in the human mind or with the aid of pen and paper. At the recited level of generality, the limitations reciting the steps of conditionally sampling nodes from a graph appear to recite mental processes. Per MPEP 2106.04(a)(2)(C), a claim that requires a computer may still recite a mental process when the concept that is performed in the human mind is merely recited as being performed on a generic computer, in a computer environment, or using a computer as a tool to perform the concept. The additional elements argued by Applicant (e.g., a GNN accelerator, external memory, CPU) appear to recite generic computing components and therefore do not provide sufficient limitation so as to be characterized as providing practical application or significantly more.
APPLICANT'S ARGUMENT: Applicant appears to argue (page 10, paragraphs 2-3) that "traditional GNN processing systems suffer from performance bottlenecks caused by the division of sampling responsibilities between the host processor and the GNN accelerator ... introduc[ing] latency, inter-component stalls, and resource contention, thereby degrading the efficiency of the GNN training pipeline. ¶ The amended claim 1 addresses this problem through restructuring the processing responsibilities. ... The GNN accelerator then autonomously performs all aspects of the GNN sampling process.... The GNN accelerator uses a local buffer ... critical to supporting efficient, pipelined sampling logic."
EXAMINER'S RESPONSE: Examiner respectfully disagrees. As indicated above, amended Claim 1 recites several steps of conditional graph sampling, which appear to be mental process steps. A GNN accelerator component appears to recite generic computing hardware, and therefore does not represent an improvement in a computer or computing technology. Storage using a buffer appears to recite a step of mere data gathering, and therefore does not represent an improvement in a computer or computing technology. In the absence of additional elements providing a practical application or significantly more, the claim is directed to an abstract idea.
APPLICANT'S ARGUMENT: Applicant appears to argue (page 10, paragraphs 4) that "the GNN accelerator retrieves provisional nodes from local buffer and graph structure&attribute data from external memory using a direct memory access (DMA) interface, which bypasses the CPU and allows high-throughput, low-latency access. ... Each processor in this system-host, accelerator, and NN processor-has a clearly defined and non-overlapping role, allowing the overall pipeline to be modular, parallelized, and scalable. This design eliminates CPU wait states, streamlines circuit partitioning ASIC and FPGA implementations, and enables better parallelization of accelerator and host operations."
EXAMINER'S RESPONSE: Examiner respectfully disagrees. As recited, both the step of memory access using DMA and the step of storing nodes in a local buffer amount to mere data gathering under Step 2A Prong Two analysis. Further, it is noted that the features upon which applicant relies (i.e., hardware components such as "CPU," "ASIC," and "FPGA," structural features such as a processing pipeline, and operational properties such as high throughput and low latency) are not recited in the rejected claim. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). In consequence, the purported improvements are not found to be represented by additional elements that that provide practical application through improvement in a particular computer or in computer technology.
Applicant’s arguments with respect to the rejection of original Claims 1-3, 11-14, and 18 under 35 U.S.C. 102(a)(2) and original Claims 4-10, 15-17, 19 and 20 under 35 U.S.C. 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
APPLICANT'S ARGUMENT: Applicant appears to argue (page 11, paragraphs 3) that "Applicant has made clarifying amendments to claims 1 and 7-10 to further clarify the distinction between the claims and the cited art. ... Claim 1, by contrast, defines a multi-processor architecture where the host initiates sampling logic via a configuration signal, enabling hardware-level decoupling of sampling responsibilities. This alone introduces a structural distinction over She's architecture."
EXAMINER'S RESPONSE: Examiner notes that Applicant's arguments are moot. A new ground of rejection of amended Claims 1, 7, 8, 10, 21-23, 25-28, and 30 under 35 U.S.C. 103 is made in view of Shi in view of Wang in view of Deisher in view of Zhang. The configuration signal feature of amended Claim 1 is made obvious by teachings of Wang.
APPLICANT'S ARGUMENT: Applicant appears to argue (page 11, paragraphs 5) that "claim 1 recites a two-stage sampling operation. ... She does not disclose or suggest this sequence of operations. While She describes sampling neighbors of a root node and using different sampling policies, it does not teach sampling nodes outside of a preset distance from the root node. ... The use of a local buffer to hold provisional negative nodes and enable multiple rounds of controlled, hierarchical sampling is unique to the claimed invention and is not found in She. ¶ This multi-stage, in-accelerator sampling flow directly supports a pipelined and autonomous sampling mechanism that minimizes host interaction and avoids latency associated with inter-processor synchronization-benefits that She does not recognize or enable."
EXAMINER'S RESPONSE: Examiner notes that Applicant's arguments are moot. A new ground of rejection of amended Claim 1 under 35 U.S.C. 103 is made in view of Shi in view of Wang in view of Deisher in view of Zhang.
The features of positive and negative sampling based on a preset distance of amended Claim 1 are made obvious by teachings of Shi and Wang. The local buffer feature of amended Claim 1 is made obvious by teachings of Zhang. The DMA feature of amended Claim 1 is made obvious by teachings of Deisher.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 7-10, and 21-30 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1
Claim 1 is ineligible.
Step 1
Claim 1 recites a computer-implemented method for accelerating Graph Neural Network (GNN), and thus the claimed process falls within a statutory category of invention.
Step 2A Prong 1
The claim recites a sampling configuration comprising an identifier of a root node of a graph and a sampling mode indicating whether to perform positive sampling, negative sampling, or both, which is a mental process. The claim recites sampling ... a first set of candidate nodes from the graph based on the root node within a preset distance from the root node as first positive nodes, which is a mental process. The claim recites sampling ... one or more provisional negative nodes from graph that are outside of the preset distance from the root node, which is a mental process. The claim recites for each provisional negative node: sampling ... a second set of candidate nodes within the preset distance from the provisional negative node as second positive nodes, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2
The additional element dispatching, by a host processor to a GNN accelerator amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element obtaining, by the GNN accelerator from external memory via a direct memory access (DMA) interface, the root node of the graph for GNN processing amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering"). The additional element by the GNN accelerator invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element storing, by the GNN accelerator, the one or more provisional negative nodes in a local buffer within the GNN accelerator amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element retrieving each of the one or more provisional negative nodes from the local buffer amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element obtaining, by the GNN accelerator via DMA, attribute data corresponding to the first positive nodes and the second positive nodes amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element processing, by one or more neural network processors, the attribute data to train a feature vector of the root node invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
Step 2B
The additional element dispatching, by a host processor to a GNN accelerator is well-understood, routine, conventional activity (see MPEP 2106.05(d), "receiving or transmitting data over a network"). The additional element obtaining, by the GNN accelerator from external memory via a direct memory access (DMA) interface, the root node of the graph for GNN processing is well-understood, routine, conventional activity (see MPEP 2106.05(d), as supported by Soldavini, p. 3, III. Overview of memory components, C. Communication and Coordination Components: "Additional components can be used to better coordinate data transfers. ... Direct memory access (DMA) allows memory access outside of the classic memory-hierarchy. Accelerators can directly interface with main memory, relieving the general-purpose processors from the burden of serving all accelerators at the same time"). The additional element by the GNN accelerator invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element storing, by the GNN accelerator, the one or more provisional negative nodes in a local buffer within the GNN accelerator is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory "). The additional element retrieving each of the one or more provisional negative nodes from the local buffer is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory "). The additional element obtaining, by the GNN accelerator via DMA, attribute data corresponding to the first positive nodes and the second positive nodes is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory "). The additional element processing, by one or more neural network processors, the attribute data to train a feature vector of the root node invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 7
Claim 7 is ineligible.
Step 1, Step 2A Prong 1
Regarding Claim 7, the rejection of Claim 1 is incorporated.
Step 2A Prong 2
The additional element obtaining the attribute data of the one or more second positive nodes and the one or more provisional negative nodes for the GNN processing on the root node of the graph amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting").
Step 2B
The additional element obtaining the attribute data of the one or more second positive nodes and the one or more provisional negative nodes for the GNN processing on the root node of the graph is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 8
Claim 8 is ineligible.
Step 1
Regarding Claim 8, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites minimizing a feature distance from the feature representation of the root node to feature representations of the first positive nodes, which is a mental process. The claim recites wherein the feature representations of the first positive nodes are determined based on the attribute data of the one or more first positive nodes, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element training a neural network to generate a feature representation of the root node invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 9
Claim 9 is ineligible.
Step 1
Regarding Claim 9, the rejection of Claim 1 is incorporated.
Step 2A Prong 1
The claim recites maximizing a feature distance from the feature representation to feature representations of the second positive nodes, which is a mental process. The claim recites wherein the feature representations of the second positive nodes are determined based on the attribute data of the one or more second positive nodes, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2, Step 2B
The additional element training a neural network to generate a feature representation of the root node invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 10
Claim 10 is ineligible.
Step 1, Step 2A Prong 1
Regarding Claim 10, the rejection of Claim 1 is incorporated.
Step 2A Prong 2
The additional element receiving a batch size from the host processor separated from the GNN accelerator amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element obtaining from the graph the batch size of root nodes comprising the root node amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting").
Step 2B
The additional element receiving a batch size from the host processor separated from the GNN accelerator is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory"). The additional element obtaining from the graph the batch size of root nodes comprising the root node is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Regarding Claim 21
Step 1
Claim 21 recites a system comprising a set of processors including a host processor, a GNN accelerator, one or more neural network processors, and one or more memories storing instructions that when executed by the set of processors, cause the set of processors to perform operations, and thus the claimed machine falls within a statutory category of invention.
Step 2A Prong 1
The claim recites a sampling configuration comprising an identifier of a root node of a graph and a sampling mode indicating whether to perform positive sampling, negative sampling, or both, which is a mental process. The claim recites sampling ... a first set of candidate nodes from the graph based on the root node within a preset distance from the root node as first positive nodes, which is a mental process. The claim recites sampling ... one or more provisional negative nodes from graph that are outside of the preset distance from the root node, which is a mental process. The claim recites for each provisional negative node: sampling ... a second set of candidate nodes within the preset distance from the provisional negative node as second positive nodes, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2
The additional element dispatching, by a host processor to a GNN accelerator amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element obtaining, by the GNN accelerator from external memory via a direct memory access (DMA) interface, the root node of the graph for GNN processing amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering"). The additional element by the GNN accelerator invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element storing, by the GNN accelerator, the one or more provisional negative nodes in a local buffer within the GNN accelerator amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element retrieving each of the one or more provisional negative nodes from the local buffer amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element obtaining, by the GNN accelerator via DMA, attribute data corresponding to the first positive nodes and the second positive nodes amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element processing, by one or more neural network processors, the attribute data to train a feature vector of the root node invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
Step 2B
The additional element dispatching, by a host processor to a GNN accelerator is well-understood, routine, conventional activity (see MPEP 2106.05(d), "receiving or transmitting data over a network"). The additional element obtaining, by the GNN accelerator from external memory via a direct memory access (DMA) interface, the root node of the graph for GNN processing is well-understood, routine, conventional activity (see MPEP 2106.05(d), as supported by Soldavini, p. 3, III. Overview of memory components, C. Communication and Coordination Components: "Additional components can be used to better coordinate data transfers. ... Direct memory access (DMA) allows memory access outside of the classic memory-hierarchy. Accelerators can directly interface with main memory, relieving the general-purpose processors from the burden of serving all accelerators at the same time"). The additional element by the GNN accelerator invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element storing, by the GNN accelerator, the one or more provisional negative nodes in a local buffer within the GNN accelerator is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory "). The additional element retrieving each of the one or more provisional negative nodes from the local buffer is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory "). The additional element obtaining, by the GNN accelerator via DMA, attribute data corresponding to the first positive nodes and the second positive nodes is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory "). The additional element processing, by one or more neural network processors, the attribute data to train a feature vector of the root node invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Claims 22-25, dependent on Claim 21, incorporate the rejection of Claim 21. Claims 22-25 incorporate substantively all the limitations of Claims 7-10, respectively, in system form and are rejected under the same rationales.
Regarding Claim 26
Step 1
Claim 21 recites a non-transitory computer-readable medium of a plurality of processors ..., the non-transitory computer-readable medium storing instructions that, when executed by the plurality of processors, cause the plurality of processors to perform operations, and thus the claimed machine falls within a statutory category of invention.
Step 2A Prong 1
The claim recites a sampling configuration comprising an identifier of a root node of a graph and a sampling mode indicating whether to perform positive sampling, negative sampling, or both, which is a mental process. The claim recites sampling ... a first set of candidate nodes from the graph based on the root node within a preset distance from the root node as first positive nodes, which is a mental process. The claim recites sampling ... one or more provisional negative nodes from graph that are outside of the preset distance from the root node, which is a mental process. The claim recites for each provisional negative node: sampling ... a second set of candidate nodes within the preset distance from the provisional negative node as second positive nodes, which is a mental process.
Thus, the claim recites an abstract idea.
Step 2A Prong 2
The additional element comprising a host processor, a GNN accelerator, and one or more neural network processors invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element dispatching, by a host processor to a GNN accelerator amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element obtaining, by the GNN accelerator from external memory via a direct memory access (DMA) interface, the root node of the graph for GNN processing amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering"). The additional element by the GNN accelerator invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element storing, by the GNN accelerator, the one or more provisional negative nodes in a local buffer within the GNN accelerator amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element retrieving each of the one or more provisional negative nodes from the local buffer amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element obtaining, by the GNN accelerator via DMA, attribute data corresponding to the first positive nodes and the second positive nodes amounts to insignificant extra-solution activity (see MPEP 2106.05(g), "mere data gathering and outputting"). The additional element processing, by one or more neural network processors, the attribute data to train a feature vector of the root node invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
Step 2B
The additional element comprising a host processor, a GNN accelerator, and one or more neural network processors invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element dispatching, by a host processor to a GNN accelerator is well-understood, routine, conventional activity (see MPEP 2106.05(d), "receiving or transmitting data over a network"). The additional element obtaining, by the GNN accelerator from external memory via a direct memory access (DMA) interface, the root node of the graph for GNN processing is well-understood, routine, conventional activity (see MPEP 2106.05(d), as supported by Soldavini, p. 3, III. Overview of memory components, C. Communication and Coordination Components: "Additional components can be used to better coordinate data transfers. ... Direct memory access (DMA) allows memory access outside of the classic memory-hierarchy. Accelerators can directly interface with main memory, relieving the general-purpose processors from the burden of serving all accelerators at the same time"). The additional element by the GNN accelerator invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it"). The additional element storing, by the GNN accelerator, the one or more provisional negative nodes in a local buffer within the GNN accelerator is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory "). The additional element retrieving each of the one or more provisional negative nodes from the local buffer is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory "). The additional element obtaining, by the GNN accelerator via DMA, attribute data corresponding to the first positive nodes and the second positive nodes is well-understood, routine, conventional activity (see MPEP 2106.05(d), "storing and retrieving information in memory "). The additional element processing, by one or more neural network processors, the attribute data to train a feature vector of the root node invokes a computer or other machinery merely as a tool to perform an existing process (see MPEP 2106.05(f), "apply it").
The claim lacks additional elements that integrate it into a practical application or provide significantly more, so it is directed to an abstract idea and is ineligible.
Claims 27-30, dependent on Claim 26, incorporate the rejection of Claim 26. Claims 27-30 incorporate substantively all the limitations of Claims 7-10, respectively, in non-transitory computer-readable medium form and are rejected under the same rationales.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 7, 8, 10, 21-23, 25-28, and 30 are rejected under 35 U.S.C. 103 as being unpatentable over Shi, et al. (U.S. 2023/0049817 A1, hereinafter "Shi") in view of Wang, et al., "Reinforced Negative Sampling over Knowledge Graph for Recommendation" (hereinafter "Wang") in view of Deisher, et al. (US 2018/0121796 A1, hereinafter "Deisher") in view of Zhang, et al., "NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding" (hereinafter "Zhang").
Regarding Claim 1, Shi teaches:
A computer-implemented method for accelerating Graph Neural Network (GNN) (Shi, Abstract: "Techniques for implementing a performance-adaptive sampling strategy towards fast and accurate graph neural networks are provided" and [0020]: "embodiments require less computing resources, meaning that embeddings can be refreshed more frequently.... This is possible because optimizing the sampling policy for task performance allows an embedding to choose which neighbors to move towards, leading to the minimum loss more efficiently"), comprising:
dispatching, by a host processor to a GNN accelerator, a sampling configuration comprising an identifier of a root node of a graph (Shi, [0043]: "Neighbor sampler 220 samples one or more neighbors of a node given an identity of the node and graph database 222. ... The identity of the given node may be included in a training instance that identifies the node and includes a label," where Shi's neighbor sampler corresponds to the instant GNN accelerator, where Shi's label corresponds to the instant identifier, and where components other than Shi's neighbor sampler correspond to the instant host processor, as in [0078]: "FIG. 5 is a flow diagram that depicts an example process 500 for training a neighbor sample policy for sampling neighbors when training a graph neural network, in an embodiment. Process 500 may be performed by different components of server system 130 and machine-learning component 200") ... ;
obtaining, by the GNN accelerator... the root node of the graph for GNN processing (Shi, [0066]: "FIG. 3A is a block diagram that depicts an example result 300 of using a sampling policy to sample nodes in a graph, beginning with vertex
v
1
," where Shi's
v
1
corresponds to the instant root node) from external memory (Shi, [0043]: "Neighbor sampler 220 samples one or more neighbors of a node given an identity of the node and graph database 222," where Shi's graph database corresponds to the instant external memory);
sampling, by the GNN accelerator, a first set of candidate nodes from the graph based on the root node (Shi, [0066]: "Result 300 indicates that, given vertex
v
1
302, vertices
v
3
306 and
v
4
308 were selected (from among vertices
v
2
-
v
5
304-310) using the sampling policy. Instance 322 reflects an invoking of the sampling policy relative to vertex
v
1
302, which considers a neighborhood of vertices that is unique to vertex
v
1
302," where Shi's neighborhood of vertices
v
2
-
v
5
corresponds to the instant candidate nodes) within a preset distance from the root node as first positive nodes (Shi, [0066]: "Instance 322 reflects an invoking of the sampling policy relative to vertex
v
1
302, which considers a neighborhood of vertices that is unique to vertex
v
1
302" and [0063]: "The second term (2) (i.e.,
q
r
a
n
d
l
j
i
=
1
/
N
i
) assigns the same sampling probability to each node in the neighborhood, defined as all nodes connected to node
v
i
" where Shi's first-order neighborhood corresponds to the instant preset distance of 1); ...
obtaining, by the GNN accelerator ..., attribute data corresponding to the first positive nodes (Shi, [0043]: "Neighbor sampler 220 samples one or more neighbors of a node given an identity of the node and graph database 222. ... The identity of the given node may be included in a training instance that identifies the node and includes a label, such as a 0 or a 1, if the task is a binary classification," where Shi's label corresponds to the instant attribute data) and ... second positive nodes (Shi, Fig. 3B, blocks 330 and 332, depicting computing loss for node
v
1
based on second-order neighbors, as in [0067]: "FIG. 3B is a block diagram that depicts an example result 310 of computing a performance loss 336 using the sampled nodes and a GNN that comprises two hidden layers," where the loss is used in training node embeddings, as in [0085]: "At block 570, based on the performance loss, the sampling policy is modified. ... Block 570 also involves modifying the GNN (which includes the embeddings) based on a gradient of the performance loss"); and
processing, by one or more neural network processors, the attribute data (Shi, [0067]: "FIG. 3B is a block diagram that depicts an example result 310 of computing a performance loss 336 using the sampled nodes and a GNN ... . The output of second hidden layer 334 and the embedding for vertex
v
1
302 are input into a sigmoid function along with a label (e.g., a classification label indicating one of many classes ...) to produce performance loss 336") to train (Shi, [0085]: "At block 570, based on the performance loss, the sampling policy is modified. ... Block 570 also involves modifying the GNN (which includes the embeddings) based on a gradient of the performance loss" and [0086]: "At block 580, it is determined whether to select more nodes. Block 580 may involve determining whether there are any more training instances that have not yet been considered for training the GNN") a feature vector of the root node (Shi, [0053]: "
h
i
0
denotes the
D
0
dimensional feature vector (or embedding) of node
v
i
," where Shi's embedding corresponds to the instant feature vector)
Shi teaches a method for accelerating GNN comprising steps for a GNN accelerator of obtaining a root node a graph, sampling nodes from the graph based on a preset distance from the root node and obtaining attribute data of the nodes, and including a further step of training a feature vector of the root node.
Shi does not explicitly teach a sampling mode indicating whether to perform positive sampling, negative sampling, or both, sampling ... one or more provisional negative nodes from graph that are outside of the preset distance from the root node, and for each provisional negative node: sampling, by the GNN accelerator, a second set of candidate nodes within the preset distance from the provisional negative node as second positive nodes.
However, Wang teaches:
a sampling mode indicating whether to perform positive sampling, negative sampling (Wang, p. 3, Figure 2, right panel, "Illustration of the proposed knowledge-aware negative sampling framework" and p. 102, 3.2 Knowledge-aware Sampler: "As shown in the right side of Figure 2, the basic idea is to, conditioned on the target user, start from the positive item, learn to navigate over the KG structure, then yield the possible negatives along the exploring paths," where Wang's starting and exploring correspond to the instant positive and negative modes), or both (Wang, p. 102, 3.2.1 Sampling as Reinforcement Learning: "Here we set equal importance for the two reward components, leaving their linear combination controlled by the hyper-parameter future. We verify the rationality of the two reward functions in Section 4.5.2," where Wang's hyperparameter weighs the importance of positive and negative sampling).
sampling ... one or more provisional negative nodes from graph that are outside of the preset distance from the root node (Wang, p. 103, 3.3.2 Neighbor Attention Module: "At state
s
t
=
u
,
e
t
, having established representations for node et and its neighbors
N
e
t
, we need to effectively search relevant actions towards potential items. In particular, we decompose an exploration operation
a
t
=
e
t
→
e
t
'
→
e
t
+
1
into two steps: 1)