DETAILED ACTION
Remarks
Claims 1-20 have been examined and rejected. This Office action is responsive to the amendment filed on 11/17/2025, which has been entered in the above identified application.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, claim 1 recites
a plurality of input graphs each having a plurality of nodes
the plurality of input graphs
the input graphs with the exception of the at least one graph of the plurality of input graphs
The relationship between these elements is unclear. It is unclear whether the graphs are intended to be the same or different graphs. For the purposes of examination, these limitations are interpreted as:
a plurality of input graphs each having a plurality of nodes
a second plurality of input graphs
a third plurality input graphs with an exception of the at least one graph of the plurality of input graphs
Claim 1 further recites
a plurality of nodes
at least some of the nodes of the plurality of nodes having an attribute
one or more sets of nodes of the plurality of nodes
the nodes of each set of the one or more sets of nodes of the plurality of nodes having identical attributes
each set of the one or more sets of nodes of the plurality of nodes
each of the nodes of that set
each node of that set
The relationship between these elements is unclear. The claim lacks antecedent basis for “the nodes of the plurality of nodes having an attribute”. It is unclear whether “having an attribute” is intended to modify the nodes or the plurality of nodes. The claim lacks antecedent basis for “the nodes of each set of the one or more sets of nodes of the plurality of nodes having identical attributes”. It is unclear whether “having identical attributes” is intended to modify the nodes, the sets, or the plurality of nodes. The claim lacks antecedent basis for “each of the nodes of that set”. It is unclear which recited set “that set” is intended to refer.
It is unclear whether the sets are intended to be the same or different sets. For the purposes of examination, these limitations are interpreted as:
a plurality of nodes
at least some nodes of the plurality of nodes, wherein there is an attribute
one or more sets of nodes of the plurality of nodes
a second plurality of nodes of each set of the one or more sets of nodes of the plurality of nodes, wherein there are identical attributes
each set of the one or more sets of nodes of the plurality of nodes
each node of a third plurality of nodes of a first set
each node of the first set
Regarding claims 9 and 16, claims 9 and 16 contain substantially similar limitations to those found in claim 1. Consequently, claims 9 and 16 are rejected for the same reasons. Claims 9 and 16 further recite “at least some of the nodes having an attribute”. It is further unclear which nodes “the nodes” is intended to refer. For the purposes of examination, this limitation is interpreted as: at least some of nodes, wherein there is an attribute
Regarding claims 2-8, 10-15, and 17-20, claims 2-8, 10-15, and 17-20 are also rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for depending on an indefinite parent claim. Claims 2, 6-8, 10, 14, 15, and 17 further additionally recite “that set” limitations and are likewise rejected and interpreted. Claims 2, 4, 12, 17, and 19 additionally recite set of nodes limitations and are likewise rejected and interpreted.
Regarding claims 6-8, 14, and 15, claims 6-8, 14, and 15 recite “the labels assigned to each of the nodes”. The claims lack antecedent basis for these limitations. It is unclear how these limitations are intended to refer back the label limitations of the parent claim. For the purposes of examination, this limitation is interpreted as: a plurality of labels assigned to each of the nodes.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claims 1, 9, and 16
Step 1: Claims 1, 9, and 16 recite a system, a method, and a medium; therefore, they are directed to the statutory categories of a machine, a manufacture, and a method.
Step 2A Prong 1: The claims recite, inter alia:
for at least one graph of the plurality of input graphs: determine one or more sets of nodes of the plurality of nodes, the nodes of each set of the one or more sets of nodes of the plurality of nodes having identical attributes; Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of determining nodes and attributes, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper
for each set of the one or more sets of nodes of the plurality of nodes, assign a label to each of the nodes of that set so that each node of that set has a different label so as to disambiguate each node; Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of assigning different labels, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper
process the one or more sets to form an aggregate value; Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of forming an aggregate value, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper, or is a mathematical concept that is achievable through mathematical computation.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. The additional elements of “A data processing system for implementing a machine learning process in dependence on a graph neural network, the system comprising: at least one processor configured to”, “A method for implementing a machine learning process in dependence on a graph neural network in a data processing system configured to”, “the method comprising” and “A non-transitory computer readable storage medium comprising: a computer program which, when executed by a computer, causes the computer to execute a method for implementing a machine learning process in dependence on a graph neural network in a data processing system” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h). The claimed computer components are recited at a high level of generality and are merely invoked as tool to perform the abstract idea. The additional elements of “receive a plurality of input graphs each having a plurality of nodes, at least some of the nodes of the plurality of nodes having an attribute”, and “taking as input: (i) the input graphs with the exception of the at least one graph of the plurality of input graphs and (ii) the aggregate value” amount to insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)). The additional element of “implement the machine learning process” is merely a post-solution step, a nominal addition to the claim that does not meaningfully limit the claim, and is therefore insignificant extra-solution activity (see MPEP 2106.05(g)). Even when viewed in combination, these additional element do not integrate the abstract idea into a practical application and the claims are thus directed to the abstract idea.
Step 2B: The claim does not contain significantly more than the judicial exception. “A data processing system for implementing a machine learning process in dependence on a graph neural network, the system comprising: at least one processor configured to”, “A method for implementing a machine learning process in dependence on a graph neural network in a data processing system configured to”, “the method comprising” and “A non-transitory computer readable storage medium comprising: a computer program which, when executed by a computer, causes the computer to execute a method for implementing a machine learning process in dependence on a graph neural network in a data processing system” amount to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)). Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. The additional elements of “receive a plurality of input graphs each having a plurality of nodes, at least some of the nodes of the plurality of nodes having an attribute”, and “taking as input: (i) the input graphs with the exception of the at least one graph of the plurality of input graphs and (ii) the aggregate value” amount to insignificant extra-solution activity in the form of mere data gathering and output (see MPEP § 2106.05(g)), and is a well-understood, routine, conventional activity (see MPEP § 2106.05(d); “Receiving or transmitting data over a network”). The additional elements of “implement the machine learning process” is well-understood, routine, conventional activity (see MPEP § 2106.05(d); see background of instant specification) and is merely a post-solution step, a nominal addition to the claim that does not meaningfully limit the claim, and is therefore insignificant extra-solution activity (see MPEP 2106.05(g)). Nothing in the claims provides significantly more than that abstract idea. As such, the claims are ineligible.
Claims 2-8, 10-15, and 17-20
Step 1: Claims 2-8, 10-15, and 17-20, and 19-23 recite systems, methods, and media; therefore, they are directed to the statutory categories of a system, a method, and a machine.
Step 2A Prong 1: claims 2-8, 10-15, and 17-20 merely narrow the previously recited abstract idea limitations. For the reasons described above with respect to claims 1, 9, and 16, this judicial exception is not meaningfully integrated into a practical application, or significantly more than the abstract idea. The claims disclose similar limitations described for the independent claims above and do not provide anything more than the mental processes that are practically capable of being performed in the human mind with the assistance of pen and paper and mathematical concepts that are achievable through mathematical computation.
Claims 2, 10, and 18 further recite the additional element of “process each set of nodes of the one or more sets of nodes of the plurality of nodes to form the aggregate value by processing neighbour nodes of each node of that set using a permutation invariant function”. Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of using a permutation invariant function to form an aggregate value, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper, or is a mathematical concept that is achievable through mathematical computation. The element of “the at least one processor” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)).
Claims 3, 11, and 19 further recite the additional element of “the permutation invariant function is one of a sum, a mean, or a maximum”. Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process using a permutation invariant function, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper, or is a mathematical concept that is achievable through mathematical computation.
Claims 4, 12, and 19 further recite the additional element of “process the one or more sets of nodes of the plurality of nodes by assigning weights to the nodes of the one or more sets of nodes of the plurality of nodes”. Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of assigning weights to nodes, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper, or is a mathematical concept that is achievable through mathematical computation. These element of “the at least one processor” and “wherein the weights are the parameters of a neural network” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)).
Claims 5, 13, and 20 further recite the additional element of “iteratively update the weights”. Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of updating weights, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper, or is a mathematical concept that is achievable through mathematical computation. These element of “the at least one processor” amounts to no more than generally linking the use of a judicial exception to a particular technological environment or field of use (see MPEP § 2106.05(h)).
Claim 6 further recites the additional element of “wherein the labels assigned to each of the nodes of that set are vectors”. Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of assigning labels using vectors, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper, or is a mathematical concept that is achievable through mathematical computation.
Claims 7 and 14 further recite the additional element of “wherein the labels assigned to each of the nodes of that set are colours”. Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of assigning labels using colors, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper.
Claims 8 and 15 further recite the additional element of “wherein the labels assigned to each of the nodes of that set are randomly assigned to the determined nodes”. Under its broadest reasonable interpretation in light of the specification, this limitation encompasses the mental process of randomly assigning labels, which is an evaluation or observation that is practically capable of being performed in the human mind with the assistance of pen and paper, or is a mathematical concept that is achievable through mathematical computation.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3, 6, 9-11, and 16-18 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Xu et al. (“HOW POWERFUL ARE GRAPH NEURAL NETWORKS?”, published 02/22/2019), hereinafter Xu.
Regarding claim 1, Xu teaches:
A data processing system for implementing a machine learning process in dependence on a graph neural network, the system comprising: at least one processor configured to receive a plurality of input graphs each having a plurality of nodes, at least some of the nodes of the plurality of nodes having an attribute, the at least one processor being configured to (Xu abs. Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs; We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler Lehman graph isomorphism test. We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance; 1, We develop a simple neural architecture, Graph Isomorphism Network (GIN), and show that its discriminative/representational power is equal to the power of the WL test; 2, Graph Neural Networks. GNNs use the graph structure and node features Xv to learn a representation vector of a node, hv, or the entire graph) :
for at least one graph of the plurality of input graphs: determine one or more sets of nodes of the plurality of nodes, the nodes of each set of the one or more sets of nodes of the plurality of nodes having identical attributes; for each set of the one or more sets of nodes of the plurality of nodes, assign a label to each of the nodes of that set so that each node of that set has a different label so as to disambiguate each node of that set (Xu 1, To mathematically formalize the above insight, our framework first represents the set of feature vectors of a given node’s neighbors as a multiset, i.e., a set with possibly repeating elements; 3, A GNN recursively updates each node’s feature vector to capture the network structure and features of other nodes around it, i.e., its rooted subtree structure; feature vectors of a set of neighboring nodes form a multiset (Figure 1): the same element can appear multiple times since different nodes can have identical feature vector; Definition 1 (Multiset). A multiset is a generalized concept of a set that allows multiple instances for its elements; we can assign each feature vector a unique label in {a, b, c . . .}. Then, feature vectors of a set of neighboring nodes form a multiset (Figure 1); 7, For social networks we create node features as follows: for the REDDIT
datasets, we set all node feature vectors to be the same (thus, features here are uninformative); for the other social graphs, we use one-hot encodings of node degrees);
process the one or more sets to form an aggregate value; and implement the machine learning process taking as input: (i) the input graphs with the exception of the at least one graph of the plurality of input graphs and (ii) the aggregate value (Xu 1, the neighbor aggregation in GNNs can be thought of as an aggregation function over the multiset; Figure 1: An overview of our theoretical framework. Middle panel: rooted subtree structures (at the blue node) that the WL test uses to distinguish different graphs. Right panel: if a GNN’s aggregation function captures the full multiset of node neighbors, the GNN can capture the rooted subtrees in a recursive manner and be as powerful as the WL test; 4.2, Figure 2: Ranking by expressive power for sum, mean and max aggregators over a multiset. Left panel shows the input multiset, i.e., the network neighborhood to be aggregated. The next three panels illustrate the aspects of the multiset a given aggregator is able to capture: sum captures the full multiset, mean captures the proportion/distribution of elements of a given type, and the max aggregator ignores multiplicities (reduces the multiset to a simple set); 2, There are two tasks of interest: (1) Node classification, where each node v ∈ V has an associated label yv and the goal is to learn a representation vector hv of v such that v’s label can be predicted as yv = f(hv); (2) Graph classification, where, given a set of graphs {G1, ..., GN } ⊆ G and their labels {y1, ..., yN } ⊆ Y, we aim to learn a representation vector hG that helps predict the label of an entire graph, yG = g(hG); 5, Node embeddings learned by GIN can be directly used for tasks like node classification and link prediction. For graph classification tasks we propose the following “readout” function that, given embeddings of individual nodes, produces the embedding of the entire graph; 5.2, In Figure 3a, every node has the same feature a and f(a) is the same across all nodes (for any function f). When performing neighborhood aggregation, the mean or maximum over f(a) remains f(a) and, by induction, we always obtain the same node representation everywhere. Thus, in this case mean and max-pooling aggregators fail to capture any structural information. In contrast, the sum aggregator distinguishes the structures because 2 · f(a) and 3 · f(a) give different values; 7, We evaluate GINs (Eqs. 4.1 and 4.2) and the less powerful GNN variants. Under the GIN framework, we consider two variants: (1) a GIN that learns in Eq. 4.1 by gradient descent).
Regarding claims 9 and 16, claims 9 and 16 contain substantially similar limitations to those found in claim 1. Consequently, claims 9 and 16 are rejected for the same reasons.
Regarding claim 2, Xu teaches all the limitations of claim 1, further comprising:
wherein the at least one processor is configured to process each set of nodes of the one or more sets of nodes of the plurality of nodes to form the aggregate value by processing neighbour nodes of each node of that set using a permutation invariant function (Xu 1, the neighbor aggregation in GNNs can be thought of as an aggregation function over the multiset; Figure 1: An overview of our theoretical framework. Middle panel: rooted subtree structures (at the blue node) that the WL test uses to distinguish different graphs. Right panel: if a GNN’s aggregation function captures the full multiset of node neighbors, the GNN can capture the rooted subtrees in a recursive manner and be as powerful as the WL test; 4.2, Figure 2: Ranking by expressive power for sum, mean and max aggregators over a multiset. Left panel shows the input multiset, i.e., the network neighborhood to be aggregated. The next three panels illustrate the aspects of the multiset a given aggregator is able to capture: sum captures the full multiset, mean captures the proportion/distribution of elements of a given type, and the max aggregator ignores multiplicities (reduces the multiset to a simple set); 2, There are two tasks of interest: (1) Node classification, where each node v ∈ V has an associated label yv and the goal is to learn a representation vector hv of v such that v’s label can be predicted as yv = f(hv); (2) Graph classification, where, given a set of graphs {G1, ..., GN } ⊆ G and their labels {y1, ..., yN } ⊆ Y, we aim to learn a representation vector hG that helps predict the label of an entire graph, yG = g(hG); 5, Node embeddings learned by GIN can be directly used for tasks like node classification and link prediction. For graph classification tasks we propose the following “readout” function that, given embeddings of individual nodes, produces the embedding of the entire graph; 5.2, In Figure 3a, every node has the same feature a and f(a) is the same across all nodes (for any function f). When performing neighborhood aggregation, the mean or maximum over f(a) remains f(a) and, by induction, we always obtain the same node representation everywhere. Thus, in this case mean and max-pooling aggregators fail to capture any structural information. In contrast, the sum aggregator distinguishes the structures because 2 · f(a) and 3 · f(a) give different values; 7, We evaluate GINs (Eqs. 4.1 and 4.2) and the less powerful GNN variants. Under the GIN framework, we consider two variants: (1) a GIN that learns in Eq. 4.1 by gradient descent).
Regarding claims 10 and 17, claims 10 and 17 contain substantially similar limitations to those found in claim 2. Consequently, claims 10 and 17 are rejected for the same reasons.
Regarding claim 3, Xu teaches all the limitations of claim 2, further comprising:
wherein the permutation invariant function is one of a sum, a mean, or a maximum (Xu 1, the neighbor aggregation in GNNs can be thought of as an aggregation function over the multiset; Figure 1: An overview of our theoretical framework. Middle panel: rooted subtree structures (at the blue node) that the WL test uses to distinguish different graphs. Right panel: if a GNN’s aggregation function captures the full multiset of node neighbors, the GNN can capture the rooted subtrees in a recursive manner and be as powerful as the WL test; 4.2, Figure 2: Ranking by expressive power for sum, mean and max aggregators over a multiset. Left panel shows the input multiset, i.e., the network neighborhood to be aggregated. The next three panels illustrate the aspects of the multiset a given aggregator is able to capture: sum captures the full multiset, mean captures the proportion/distribution of elements of a given type, and the max aggregator ignores multiplicities (reduces the multiset to a simple set); 2, There are two tasks of interest: (1) Node classification, where each node v ∈ V has an associated label yv and the goal is to learn a representation vector hv of v such that v’s label can be predicted as yv = f(hv); (2) Graph classification, where, given a set of graphs {G1, ..., GN } ⊆ G and their labels {y1, ..., yN } ⊆ Y, we aim to learn a representation vector hG that helps predict the label of an entire graph, yG = g(hG); 5, Node embeddings learned by GIN can be directly used for tasks like node classification and link prediction. For graph classification tasks we propose the following “readout” function that, given embeddings of individual nodes, produces the embedding of the entire graph; 5.2, In Figure 3a, every node has the same feature a and f(a) is the same across all nodes (for any function f). When performing neighborhood aggregation, the mean or maximum over f(a) remains f(a) and, by induction, we always obtain the same node representation everywhere. Thus, in this case mean and max-pooling aggregators fail to capture any structural information. In contrast, the sum aggregator distinguishes the structures because 2 · f(a) and 3 · f(a) give different values; 7, We evaluate GINs (Eqs. 4.1 and 4.2) and the less powerful GNN variants. Under the GIN framework, we consider two variants: (1) a GIN that learns in Eq. 4.1 by gradient descent).
Regarding claims 11 and 18, claims 11 and 18 contain substantially similar limitations to those found in claim 3. Consequently, claims 11 and 18 are rejected for the same reasons.
Regarding claim 6, Xu teaches all the limitations of claim 1, further comprising:
wherein the labels assigned to each of the nodes of that set are vectors (Xu 1, To mathematically formalize the above insight, our framework first represents the set of feature vectors of a given node’s neighbors as a multiset, i.e., a set with possibly repeating elements; 2, There are two tasks of interest: (1) Node classification, where each node v ∈ V has an associated label yv and the goal is to learn a representation vector hv of v such that v’s label can be predicted as yv = f(hv); (2) Graph classification, where, given a set of graphs {G1, ..., GN } ⊆ G and their labels {y1, ..., yN } ⊆ Y, we aim to learn a representation vector hG that helps predict the label of an entire graph, yG = g(hG); 3, A GNN recursively updates each node’s feature vector to capture the network structure and features of other nodes around it, i.e., its rooted subtree structure; feature vectors of a set of neighboring nodes form a multiset (Figure 1): the same element can appear multiple times since different nodes can have identical feature vector; Definition 1 (Multiset). A multiset is a generalized concept of a set that allows multiple instances for its elements; 3, we can assign each feature vector a unique label in {a, b, c . . .}. Then, feature vectors of a set of neighboring nodes form a multiset (Figure 1): the same element can appear multiple times since different nodes can have identical feature vector);
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 5, 12, 13, 19, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Xu in view of Riley et al. (US 20170061276 A1, published 03/02/2017), hereinafter Riley.
Regarding claim 4, Xu teaches all the limitations of claim 1, further comprising:
the parameters of a neural network (Xu abs. Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs; We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler Lehman graph isomorphism test. We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance; 1, We develop a simple neural architecture, Graph Isomorphism Network (GIN), and show that its discriminative/representational power is equal to the power of the WL test; 2, Graph Neural Networks. GNNs use the graph structure and node features Xv to learn a representation vector of a node, hv, or the entire graph; 4.1, We can use multi-layer perceptrons (MLPs) to model and learn f and ϕ in Corollary 6, thanks to the universal approximation theorem (Hornik et al., 1989; Hornik, 1991). In practice, we model f (k+1) ◦ ϕ (k) with one MLP, because MLPs can represent the composition of function).
However, Xu fails to expressly disclose wherein the at least one processor is configured to process the one or more sets of nodes of the plurality of nodes by assigning weights to the nodes of the one or more sets of nodes of the plurality of nodes, wherein the weights are the parameters of a neural network. In the same field of endeavor, Riley teaches:
wherein the at least one processor is configured to process the one or more sets of nodes of the plurality of nodes by assigning weights to the nodes of the one or more sets of nodes of the plurality of nodes, wherein the weights are the parameters of a neural network (Riley Figs. 1-8; [0018], The system 100 includes a neural network 101 and a graph data parser 104; [0022], A subnetwork 106 in the neural network 101 processes the vertex input data 114; [0029], FIG. 2 is a flow diagram of an example method for processing graph data at a neural network; [0040], the first vertex function can be an inverse tangent function. The first vertex function can also be Relu (WX+b), where X is the vertex input data as a vector of length n, W is a weight matrix of size m×n, b is a bias vector of length m, and Relu is an element wise non-linearity function, e.g., a rectifier. Parameters of the first vertex function can be adjusted during training; [0048], the system can sum the first pair output and the second pair output or can generate a weighted sum, with the weights being parameters that can be adjusted during training; [0064], The system applies a non-commutative function (step 808) to three inputs: as a first input, pair input data for the first pair of vertices; as a second input, pair input data for the second pair of vertices; and as a third input, pair input data for the third pair of vertices. For example, for a non-commutative function f and triple XYZ, the system computes f([X,Y], [X,Z], [Y,Z]), where [X,Y] is pair input data for the vertices X and Y, [Y, Z] is pair input data for the vertices Y and Z, and [X,Z] is pair input data for the vertices X and Z. As a result, the non-commutative function generates a first non-commutative output. An example of a non-commutative function is Relu (W(X, Y)+b) , where Relu is a non-linearity function, W is a weight matrix, b is a bias vector, and (X, Y) is a concatenation of input vectors X and Y)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated wherein the at least one processor is configured to process the one or more sets of nodes of the plurality of nodes by assigning weights to the nodes of the one or more sets of nodes of the plurality of nodes, wherein the weights are the parameters of a neural network as suggested in Riley into Xu. Doing so would be desirable because this specification relates to a neural network for processing graph data (see Riley [0001]). A neural network can generate a classification from input data representing an undirected graph of vertices connected by edges. Generally, converting a graph to data to be processed by a neural network requires the graph to be sequenced in a particular order. Some neural networks can output different classifications for different sequences of the graph even though elements of the graph remain the same. The neural network described in this specification allows graphs of vertices to be sequenced in any order while still having the same neural network output classification. For example, the neural network can receive a graph that represents a molecule and effectively process the graph to generate an output, e.g., a classification that defines a likelihood that the molecule will bind to a particular target molecule. As a consequence of having the output of the network be independent of an ordering of the elements of the graph when represented as an input, the neural network has to learn fewer parameters, which leads to a better performing and simpler to use model (see Riley [0006]).
Regarding claims 12 and 19, claims 12 and 19 contain substantially similar limitations to those found in claim 4. Consequently, claims 12 and 19 are rejected for the same reasons.
Regarding claim 5, Xu teaches all the limitations of claim 4, further comprising:
wherein the at least one processor is further configured to iteratively update (Xu abs. Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs; 4, A aggregates and updates node features iteratively; 4.1, We can use multi-layer perceptrons (MLPs) to model and learn f and ϕ in Corollary 6, thanks to the universal approximation theorem (Hornik et al., 1989; Hornik, 1991). In practice, we model f (k+1) ◦ ϕ (k) with one MLP, because MLPs can represent the composition of function; In the first iteration, we do not need MLPs before summation; 4.2, . A sufficient number of iterations is key to achieving good discriminative power).
Riley further teaches:
iteratively update the weights (Riley Figs. 1-8; [0018], The system 100 includes a neural network 101 and a graph data parser 104; [0022], A subnetwork 106 in the neural network 101 processes the vertex input data 114; [0029], FIG. 2 is a flow diagram of an example method for processing graph data at a neural network; [0040], the first vertex function can be an inverse tangent function. The first vertex function can also be Relu (WX+b), where X is the vertex input data as a vector of length n, W is a weight matrix of size m×n, b is a bias vector of length m, and Relu is an element wise non-linearity function, e.g., a rectifier. Parameters of the first vertex function can be adjusted during training; [0048], the system can sum the first pair output and the second pair output or can generate a weighted sum, with the weights being parameters that can be adjusted during training; see also [0064])
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated iteratively update the weights as suggested in Riley into Xu. Doing so would be desirable because this specification relates to a neural network for processing graph data (see Riley [0001]). A neural network can generate a classification from input data representing an undirected graph of vertices connected by edges. Generally, converting a graph to data to be processed by a neural network requires the graph to be sequenced in a particular order. Some neural networks can output different classifications for different sequences of the graph even though elements of the graph remain the same. The neural network described in this specification allows graphs of vertices to be sequenced in any order while still having the same neural network output classification. For example, the neural network can receive a graph that represents a molecule and effectively process the graph to generate an output, e.g., a classification that defines a likelihood that the molecule will bind to a particular target molecule. As a consequence of having the output of the network be independent of an ordering of the elements of the graph when represented as an input, the neural network has to learn fewer parameters, which leads to a better performing and simpler to use model (see Riley [0006]).
Regarding claims 13 and 20, claims 13 and 20 contain substantially similar limitations to those found in claim 5. Consequently, claims 13 and 20 are rejected for the same reasons.
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Xu in view of Al-Rfou et al. (US 20200334495 A1, published 10/22/2020), hereinafter Al-Rfou. Examiner Note: Disclosure in Al-Rfou used for rejection is supported in provisional application 62/835,899 filed on 04/18/2019.
Regarding claim 7, Xu teaches all the limitations of claim 1, further comprising:
wherein the labels assigned to each of the nodes of that set (Xu 1, To mathematically formalize the above insight, our framework first represents the set of feature vectors of a given node’s neighbors as a multiset, i.e., a set with possibly repeating elements; 2, There are two tasks of interest: (1) Node classification, where each node v ∈ V has an associated label yv and the goal is to learn a representation vector hv of v such that v’s label can be predicted as yv = f(hv); (2) Graph classification, where, given a set of graphs {G1, ..., GN } ⊆ G and their labels {y1, ..., yN } ⊆ Y, we aim to learn a representation vector hG that helps predict the label of an entire graph, yG = g(hG); 3, A GNN recursively updates each node’s feature vector to capture the network structure and features of other nodes around it, i.e., its rooted subtree structure; feature vectors of a set of neighboring nodes form a multiset (Figure 1): the same element can appear multiple times since different nodes can have identical feature vector; Definition 1 (Multiset). A multiset is a generalized concept of a set that allows multiple instances for its elements; 3, we can assign each feature vector a unique label in {a, b, c . . .}. Then, feature vectors of a set of neighboring nodes form a multiset (Figure 1): the same element can appear multiple times since different nodes can have identical feature vector; 5, colors denote different node features, and we assume the GNNs aggregate neighbors first before combining them with the central node labeled as v and v0; Let hcolor (r for red, g for green) denote node features transformed by f).
However, Xu fails to expressly disclose wherein the labels assigned to each of the nodes of that set are colours. In the same field of endeavor, Al-Rfou teaches:
wherein the labels assigned to each of the nodes of that set are colours (Al-Rfou Figs. 1-5; [0014], at least one machine-learned multi-label classifier for each of the at least one source graphs to generate a plurality of machine-leaned multi-label classifiers, where each machine-learned multi-label classifier is associated with one source graph; [0034], the graphs can include an attribute such as a label for certain nodes and edges; [0094], the model trainer 160 can train the machine-learned models 120 and/or 140 based on a set of training data 162. The training data 162 can include, for example, a set of source graphs, the graphs including one or more nodes and/or one or more edges. In some implementations, the training data 162 can also include attribute data characterizing the nodes and or edges. As an example, the attribute data can include a label further characterizing the node or edge such as a color (e.g., red, blue, etc.), an element (e.g., hydrogen, carbon, oxygen), or other suitable labels)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated wherein the labels assigned to each of the nodes of that set are colours as suggested in Al-Rfou into Xu. Doing so would be desirable because the present disclosure relates generally to machine learning methods for evaluating graph similarity. More particularly, the present disclosure relates to unsupervised learning techniques for evaluating graph similarity (see Al-Rfou [0002]). Machine Learning (ML) methods have achieved tremendous success in domains where the structure of the data is known a priori (see Al-Rfou [0003]). In contrast, graph learning represents a more general class of problems because the structure of the data is free from any constraints. An ML model (e.g., a neural network) must learn to solve both a desired task at hand (e.g., node classification) and to represent the structure of the problem itself—that of the graph's nodes, edges, attributes, and communities (see Al-Rfou [0004]). The examples and implementations disclosed herein can provide improved technical effects and benefits for learning representations of graphs in an embedding space. As an example, graph structures across varying domains (e.g., biology, chemistry, social networks, transportation networks, etc.) can be represented without requiring explicit feature engineering. Additionally, since the methods are unsupervised, an expert would not be required to label data or model output to determine performance which can lead to decreased costs (see Al-Rfou [0039]).
Regarding claim 14, claim 14 contains substantially similar limitations to those found in claim 7. Consequently, claim 14 is rejected for the same reasons.
Claims 8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Xu in view of Lee et al. (US 20200285944 A1, published 09/10/2020), hereinafter Lee.
Regarding claim 8, Xu teaches all the limitations of claim 1, further comprising:
wherein the labels assigned to each of the nodes of that set are assigned to the determined nodes (Xu 1, To mathematically formalize the above insight, our framework first represents the set of feature vectors of a given node’s neighbors as a multiset, i.e., a set with possibly repeating elements; 2, There are two tasks of interest: (1) Node classification, where each node v ∈ V has an associated label yv and the goal is to learn a representation vector hv of v such that v’s label can be predicted as yv = f(hv); (2) Graph classification, where, given a set of graphs {G1, ..., GN } ⊆ G and their labels {y1, ..., yN } ⊆ Y, we aim to learn a representation vector hG that helps predict the label of an entire graph, yG = g(hG); 3, A GNN recursively updates each node’s feature vector to capture the network structure and features of other nodes around it, i.e., its rooted subtree structure; feature vectors of a set of neighboring nodes form a multiset (Figure 1): the same element can appear multiple times since different nodes can have identical feature vector; Definition 1 (Multiset). A multiset is a generalized concept of a set that allows multiple instances for its elements; 3, we can assign each feature vector a unique label in {a, b, c . . .}. Then, feature vectors of a set of neighboring nodes form a multiset (Figure 1): the same element can appear multiple times since different nodes can have identical feature vector; 5, colors denote different node features, and we assume the GNNs aggregate neighbors first before combining them with the central node labeled as v and v0; Let hcolor (r for red, g for green) denote node features transformed by f).
However, Xu fails to expressly disclose the labels are randomly assigned. In the same field of endeavor, Lee teaches:
the labels are randomly assigned (Lee Figs. 1-5; [0127], In the tests, the architecture of the MCN model which performs the best in previous experiments is used. More specifically, a two-layer MCN with 8 self-attention heads (each with 8 hidden nodes) in the first layer and a softmax binary classification layer in the second layer is used. The model is tested with the following motifs: edge, triangle, and 4-clique motifs, which are shown to give good performance in the experiments described above with K=1 and weighted motif-induced adjacencies. 5% of the total number of nodes is used for training, and equal numbers of nodes are used for validation and testing. Since the graphs do not have corresponding node attributes, 50 features are randomly generated for each node. Random class labels are assigned to the nodes)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to have incorporated the labels are randomly assigned as suggested in Lee into Xu. Doing so would be desirable because this disclosure relates to graph convolutional networks that select the most relevant neighborhoods (including, but not limited to, one-hop edge-connected neighbors) for individual nodes to integrate information from, and, in some cases, thereby more efficiently filtering graph-structured data and achieving high prediction accuracy based on the graph-structured data (see Lee [0001]). In many real-world problems, such as social networks, collaboration networks, citation networks, telecommunication networks, biological networks (e.g., brain connectomes), financial transactions, transportation networks (e.g., traffic/shipping maps or routes), and the like, data may be better represented by graphs, rather than grids (e.g., matrices). Even though many artificial intelligence and machine-learning techniques have been used to extract knowledge and insights from data in various forms in order to understand and analyze actual phenomena with data (e.g., in many vision-related applications), these techniques generally do not perform well for graph-structured data. For example, convolutional neural network (CNN) models have been applied successfully in image classification, object detection and recognition, video action recognition, and the like. CNN models efficiently implements spatial locality and weight-sharing by reusing local filters to extract features from localized regions (also referred to as receptive fields). However, CNN models are designed to process data that is representable by grids, such as videos, images, or audio clips. It is difficult to directly apply CNN models to many real-world problems where the data is better represented by graphs rather than by grids. In addition, CNN models generally do not perform well for graphs that have more irregular structures, such as the various networks described above. Therefore, an improved graph-based convolutional neural network is needed for performing convolution operations on graph-structured data to extract certain features (see Lee [0002]).
Regarding claim 15, claim 15 contains substantially similar limitations to those found in claim 8. Consequently, claim 15 is rejected for the same reasons.
Response to Arguments
The Examiner acknowledges the Applicant’s amendments to claims 1, 2, 4-10, 12, 14-17, and 19. The previous objections to claims 2, 10, and 17 are respectfully withdrawn. Regarding the rejection under 35 U.S.C. 112(b), Applicant believes that the amendments to these claims, and to analogous claims depending from claims 9 and 16, also render the claims definite (see remarks p. 6). Examiner respectfully disagrees.
As discussed in the rejection, the metes and bounds of the claimed invention are unclear. Per the MPEP 2173, the definiteness of claim language is to ensure that the scope of the claims is clear so the public is informed of the boundaries of what constitutes infringement of the patent. During examination, a claim must be given its broadest reasonable interpretation consistent with the specification as it would be interpreted by one of ordinary skill in the art. If the language of a claim, given its broadest reasonable interpretation, is such that a person of ordinary skill in the relevant art would read it with more than one reasonable interpretation, then a rejection under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph is appropriate (see MPEP 2173). As discussed in the rejection, Examiner determined that the language of claims 1, 9, and 16 is indefinite and claims 1-20 stand rejected stand rejected under 35 U.S.C. 112(b).
Regarding the rejection under 35 U.S.C. 101, applicant alleges the specification of the present application does provide an improvement in technology, namely, "[t]he use of labels efficiently separates nodes with the same attributes in a graph neural network. Disambiguation of nodes using this scheme allows for the separation of non-isomorphic graphs and allows the neural network to better identify each node and perform targeted computation." (Paragraph [0013]). The claim has been amended to reflect the disclosed improvement by now reciting "assign a label to each of the nodes of that set so that each node of that set has a different label so as to disambiguate each node of that set." (see remarks p. 7). Examiner respectfully disagrees.
One way to determine integration into a practical application is when the claimed invention improves the functioning of a computer or improves another technology or technical field. To evaluate an improvement to a computer or technical field, the specification must set forth an improvement in technology and the claim itself must reflect the disclosed improvement. See MPEP 2106.04(d)(1) and 2106.05(a).
The consideration of whether the claim as a whole includes an improvement to a computer or to a technological field requires an evaluation of the specification and the claim to ensure that a technical explanation of the asserted improvement is present in the specification, and that the claim reflects the asserted improvement.
While the disclosure states that “using this scheme allows for the separation of non-isomorphic graphs and allows the neural network to better identify each node and perform targeted computation,” there is no improvement to the functioning of a computer nor to any other technology. At best, the claimed combination amounts to an improvement to the abstract idea of “disambiguating nodes” (see instant specification [0011]) rather than to any technology. See MPEP 2106.05(a).
Regarding independent claim 1, the Applicant alleges that Xu fails to teach the amended claim. Examiner respectfully disagrees.
As discussed in the rejection above, Xu teaches a system including receive a plurality of input graphs each having a plurality of nodes, at least some of the nodes of the plurality of nodes having an attribute, the at least one processor being configured to (sections abs., 1, 2): for at least one graph of the plurality of input graphs: determine one or more sets of nodes of the plurality of nodes, the nodes of each set of the one or more sets of nodes of the plurality of nodes having identical attributes; for each set of the one or more sets of nodes of the plurality of nodes, assign a label to each of the nodes of that set so that each node of that set has a different label so as to disambiguate each node of that set (Figure 1 sections 1, 3); process the one or more sets to form an aggregate value; and implement the machine learning process taking as input: (i) the input graphs with the exception of the at least one graph of the plurality of input graphs and (ii) the aggregate value (Figure 1, Figure 2, sections 1, 2, 4.2, 5, 5.2, 7).
Specifically, applicant alleges claim 1 recites a set of nodes. Xu teaches that a multiset is a set with possibly repeating elements (see remarks p. 7). A multiset is not a set of nodes but a set of feature vectors. While those feature vectors are related to nodes, feature vectors are not nodes. Rather, feature vectors are something that nodes can aggregate, as in "each node aggregates feature vectors of its neighbors" (Xu, page 1). Clearly, feature vectors are properties of nodes, and not the nodes themselves (see remarks p. 8). Examiner respectfully disagrees.
Xu discloses GNNs follow a neighborhood aggregation scheme, where the
representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes (see abs.). Figure 1 illustrates our idea. Feature vectors of a set of neighboring nodes form a multiset (see 3). Node embeddings learned by GIN can be directly used for tasks like node classification and link prediction (see 4.2). Examiner notes the claims place no limitations on what the set of nodes must comprise other than at least some of the nodes of the plurality of nodes having an attribute. Thus, Xu’s disclosure of a set of nodes is considered to teach the claimed limitations.
Applicant further alleges, the claim requires that nodes have attributes and within a set of nodes the attributes of all of the nodes are identical. The Examiner alleges that this is met by Xu where the reference teaches "a set with possibly repeating elements." That elements possibly may repeat is not the same thing as that all nodes have the same attribute (see remarks p. 8). Examiner respectfully disagrees.
Xu discloses that each node is represented by the same set of feature attributes (a feature vector). Xu further discloses different nodes can have identical feature vector (see section 3). Xu further discloses for bioinformatics datasets, the nodes have categorical input features. For social networks datasets we create node features as follows: for the REDDIT datasets, we set all node feature vectors to be the same (thus, features here are uninformative); for the other social graphs, we use one-hot encodings of node degrees (see 7). Examiner notes that the claims place no limitations on what the attributes must comprise. The claims do not specify whether the “identical attributes” refers to the type of attributes or the values of the attributes. Thus, Xu’s disclosure of nodes with identical types of attributes and identical values of attributes is considered to teach the claimed limitations.
Applicant further alleges claim 1 also requires assigning a label to each of the nodes of that set so that each node of that set has a different label. Against this limitation the Examiner repeated the definition of a multiset, that GNNs recursively update each node's "feature vector to capture the network structure and features of other nodes around it, i.e., its rooted subtree structure; feature vectors of a set of neighboring nodes form a multiset (Figure 1): the same element can appear multiple times since different nodes can have identical feature vector" and so forth, but did not cite anything where Xu teaches assigning labels (see remarks p. 8). Examiner respectfully disagrees.
Xu discloses for notational simplicity, we can assign each feature vector a unique label in {a, b, c . . .}. Then, feature vectors of a set of neighboring nodes form a multiset (section 3). Examiner notes that the claims place no limitations on what the label must comprise or how the labels must be different. For example, whether the values of each label must be different or whether assigning a label to different nodes would represent different labels. Thus, Xu’s disclosure of different label so as to disambiguate each node is considered to teach the limitation.
Similar arguments have been presented for claims 9 and 16 and thus, Applicant’s arguments are not persuasive for the same reasons.
Applicant states that the dependent claims recite all the limitations of the independent claims, and thus, are allowable in view of the remarks set forth regarding the independent claims. However, as discussed above, Xu is considered to teach the independent claims, and consequently, the dependent claims are rejected.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kumar (US 8280836 B2) see Figs. 1-7 and col. 1 [line 53] – col. 2 [line 55].
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN T REPSHER III whose telephone number is (571)272-7487. The examiner can normally be reached Monday - Friday, 8AM-5PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch can be reached at (571) 272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN T REPSHER III/ Primary Examiner, Art Unit 2143
Prosecution Insights