DETAILED ACTION
This action is responsive to the amendment filed on 07/24/2026. Claims 1-3, 5-9, 11-13, 15-17, 19-20 are pending and have been examined. This action is Final.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Argument 1: The applicant argues that amended claims 1, 12 and 17 are eligible because they no longer recite a judicial exception under Step 2A, Prong 1. In the applicant's view, the claims recite no explicit mathematical operations (such as calculating a matrix) and cannot be done in the mind or with pen and paper. The key limitations are generating a hypergraph from document elements and performing the node hypergraph convolution by generating a plurality of hyperedge-dependent node embeddings. Under Prong 2, the applicant argues that claim 1 improves the functioning of a machine learning system (Enfish, McRO) and the fields of document processing and style recommendation. The problem it identifies is that conventional models learn a single embedding per node and so fail to capture the node's other hyperedges ([0021]). The per-hyperedge embeddings are presented as the fix ([0022]-[0023]), backed by reported gains of 7.72% in hyperedge prediction, 11.37% AUC in node classification, and about 4x better loss at 25 epochs ([0160]-[0163]). For claim 12 (training), the applicant relies on Ex-parte Desjardins and the December 5, 2025 USPTO memo, which added MPEP 2106.05(a)(I) examples xiii/xiv (improved ML training and parameter adjustment), and asks that claim 1 be treated the same way. Under Step 2B, the applicant argues, citing Berkheimer, that the combination is not well-understood, routine or conventional and that the Office gave no factual support for that finding. Dependent claims 2-3, 5-9, 11, 13, 15-16 and 19-20 are argued only through their dependency. The applicant does not rebut the Office's specific findings on the matrix claims (5, 6, 8, 9, 15, 16), where the Office found nothing to evaluate beyond the math.
Response to Argument 1: The applicant’s arguments have been fully considered but are not persuasive. Regarding Step 2A, Prong 1, a claim does not need to recite a formula to recite a mathematical concept; a mathematical calculation may be recited in words, and the claim is read under its broadest reasonable interpretation in light of the specification (see MPEP 2106.04(a)(2)(I)). The specification itself defines the claimed “node hypergraph convolution” and “hyperedge-dependent node embeddings” as the calculations of Eqs. 15-20 and 23-25, i.e., products of the incidence, degree, and random walk transition matrices with learned weight matrices, and a function ψ applied to a node embedding and a hyperedge embedding or their concatenation (SPEC [0118]-[0125]); the words “performing … a node hypergraph convolution … by generating a plurality of hyperedge-dependent node embeddings” therefore recite those calculations, not merely something “based on” them. Further, as claimed for “a document,” generating a hypergraph from its elements is grouping the elements of the document into sets using custom-designed rules (SPEC [0036], [0091], [0093]), and “generating a predicted document element” is recited without any particular technique, both of which are practically performable in the mind or with pen and paper. Regarding Step 2A, Prong 2, the asserted improvement, namely that conventional models learn a “single node embedding per node” whereas the claims generate a plurality of hyperedge-dependent node embeddings (SPEC [0021]-[0023]), is an improvement in the mathematical concept itself, and an improvement in the judicial exception alone cannot provide the improvement relied upon for integration (see MPEP 2106.05(a), explaining that “it is important to keep in mind that an improvement in the abstract idea itself … is not an improvement in technology”). The reported gains in hyperedge prediction, node classification, and data efficiency (SPEC [0160]-[0163]) are more accurate outputs of that mathematics, not an improvement to the functioning of the computer or of any other technology, and the elements that remain once the mathematics and mental steps are set aside (obtaining a document, a generic “hypergraph neural network” and “hypergraph component,” and maintaining the calculated embeddings) are mere data gathering, instructions to apply the exception on generic computing components, and insignificant extra-solution activity (MPEP 2106.05(f), (g)). The HTML style recommendation and marketing email embodiments relied upon by the applicant are not recited in any claim, and the claim itself must reflect the asserted improvement (MPEP 2106.05(a)). Enfish and McRO are distinguishable because those claims recited a specific improvement to computer functionality or specific rules that achieved a technological result, whereas the present claims recite result-oriented calculations and outputs, and Recentive Analytics, Inc. v. Fox Corp. supports the rejection, as the Federal Circuit there held that claims applying machine learning to a new data environment, without claiming an improvement to the machine learning technique itself, are ineligible. Ex-parte Desjardins and the December 5, 2025 memorandum do not compel a different result: the Desjardins claims recited a particular training technique that changed how the model itself learned and retained prior knowledge, whereas claim 12 recites only “training … the hypergraph neural network based on the training data and the predicted node embedding,” without any particular loss, parameter-update rule, or training procedure, and the only specific feature of claim 12 is the mathematical generation of the predicted hyperedge-dependent node embeddings; the claims have been evaluated as a whole consistent with that guidance. Regarding Step 2B and Berkheimer, the well-understood, routine, and conventional findings in the rejection are grounded in activities the courts have recognized as such, including receiving or transmitting data, storing and retrieving information in memory, and performing repetitive calculations, such as repeatedly updating embeddings or model parameters (MPEP 2106.05(d)(II)). Moreover, each additional element is independently evaluated under MPEP 2106.05(f), (g), or (h), and those findings, which do not require Berkheimer evidence, are sufficient on their own to show that the additional elements do not amount to significantly more. The applicant’s assertion that the combination is unconventional rests on the alleged novelty of the hyperedge-dependent embedding calculation, and a new abstract idea is still an abstract idea (see MPEP 2106.05(I), citing SAP America, Inc. v. InvestPic, LLC, 898 F.3d 1161, 1163 (Fed. Cir. 2018)). Dependent claims 2-3, 5-9, 11, 13, 15-16, and 19-20 were not separately argued and, as set forth in the rejection above, add only further mathematical concepts or additional elements that do not integrate the exception or amount to significantly more. Accordingly, the rejection of claims 1-3, 5-9, 11-13, 15-17, and 19-20 under 35 U.S.C. 101 is maintained.
Argument 2: The applicant argues that Sakhinana does not anticipate amended claim 1, on two points. First, Sakhinana's HMPNN combines intra- and inter-hyperedge message aggregations into a single hidden-state vector per node ([0009]-[0011], FIG. 2B). That means it cannot teach generating a plurality of hyperedge-dependent node embeddings, one per hyperedge, with the updated node embedding comprising that plurality. This limitation was carried into claim 1 from canceled claim 4, which the Office had mapped to Sakhinana [0011]. The applicant points to its own FIG. 10 and Eqs. 23-25 ([0117], [0122]-[0125]) for contrast. Second, Sakhinana does not generate an augmented hypergraph. Its “augmented expressiveness of the node embeddings” ([0086]) describes more expressive node vectors, not a hypergraph data structure, and Sakhinana pools node states into a molecular-property prediction. The applicant relies on its own definition of “hypergraph” ([0025]) and on FIG. 3, [0030] and [0070]. Claims 12 and 17 are asserted to be allowable because they include the claim 1 limitations. However, claim 12 recites the predicted hyperedge-dependent embeddings but not the augmented hypergraph, document elements or predicted document element, so only the first argument applies to it. The applicant offers no independent argument against Wang (claims 2, 19), Cheek (claims 7, 11, 12, 20) or Chen (claim 10, now canceled). All dependent claims rest on dependency. Also note that the new “predicted document element” limitation in claims 1 and 17 came from canceled claim 10, which was rejected over Sakhinana + Cheek + Chen. The applicant does not address that combination at all, and it would be the natural basis for a new ground of rejection.
Response to Argument 2: The applicant’s arguments with respect to claims 1, 12, and 17 have been considered but are moot because the new grounds of rejection, necessitated by amendment, do not rely on Sakhinana for any limitation argued; however, to the extent the arguments apply to the new grounds, they are addressed here. First, regarding the alleged “single node embedding per node,” Yadati refines, for each hyperlink e, a separate embedding of each vertex v ∈ e, such that a vertex in two hyperlinks “will have two embeddings” ([Yadati, page 1708, sec. 4.1, Fig. 3]), which is a plurality of hyperedge-dependent node embeddings corresponding to the hyperedges, respectively, and Choe teaches a hypergraph neural network that “represents the same node differently depending on the hyperedges it participates in,” in which the same node has a different output within each hyperedge and those edge-dependent node embeddings are maintained for any downstream task ([Choe, page 1, Abstract, page 4, Fig. 2, page 6, sec. 5.4]), which is an updated node embedding for the node that comprises the plurality of hyperedge-dependent node embeddings. The applicant’s reliance on Eqs. 23-25 and the function ψ of FIG. 10 is not commensurate with the scope of the claims, which recite neither “ψ” nor any particular manner of computing the hyperedge-dependent node embeddings, and limitations from the specification are not read into the claims (see In re Van Geuns, 988 F.2d 1181, 1184 (Fed. Cir. 1993), MPEP 2111.01(II)). Further, although not relied upon in the rejection above, Choe additionally discloses a function “ψ” that “takes the concatenation of node and hyperedge embeddings as its input” ([Choe, page 5, sec. 5.4]). Moreover, the claimed “node hypergraph convolution” does not require any particular convolution operator, as the specification defines it broadly as “a process of extracting high-order data correlation information for representation learning related to nodes in a hypergraph” (SPEC [0028]), which reads on Ding’s HyperGAT layer, which aggregates the nodes of each hyperedge into a hyperedge representation and aggregates those hyperedge representations back into each node ([Ding, pages 4930-4931, sec. 3.3]), and on Yadati’s hyperlink-aware GCN layer, which operates on each hyperlink as a unit “to preserve the higher-order relationships among the vertices in each hyperlink” ([Yadati, page 1707, sec. 4.1]). Second, the examiner agrees that the rejection no longer relies on Sakhinana’s “augmented expressiveness of the node embeddings” for the augmented hypergraph. Instead, Yadati predicts the “missing hyperlinks” of an “incomplete” hypergraph from the hyperlink-aware vertex embeddings and uses them “to complete” the hypergraph ([Yadati, page 1707, sec. 3.1, page 1710, sec. 4.3, page 1711, sec. 5.1]), which is a hypergraph data structure that includes new hyperedges generated based on the updated node embeddings, exactly as Applicant characterizes the augmented hypergraph by relying on the specification’s definition of “hypergraph” and its description of “inferring new unseen hyperedges” (SPEC [0025], [0030]). Third, the newly added limitations of obtaining a document including document elements, generating the hypergraph based on those elements, and generating a predicted document element are taught by Ding, which constructs a hypergraph for each document whose nodes are the words of the document and whose hyperedges are its sentences and topics ([Ding, pages 4929-4930, sec. 3.2]), combined with Yadati’s prediction of new hyperlinks formed of words/phrases ([Yadati, page 1706, Fig. 1, page 1711, sec. 5.1]). Further, the claims do not limit the type of the predicted document element, and the specification’s own style recommendation embodiment likewise predicts a link between a fragment and an entity of the hypergraph (SPEC [0164]). Claim 17 recites substantially the same limitations as claim 1 and is rejected on the same grounds, and claim 12, which does not recite the document, augmented hypergraph, or predicted document element limitations, is rejected over Yadati in view of Choe as set forth above. The applicant has not presented separate arguments for dependent claims 2-3, 5-9, 11, 13, 15-16, and 19-20 beyond their dependency, and those claims remain rejected under 35 U.S.C. 103 for the reasons set forth in the rejection above.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-3, 5-9, 11-13, 15-17, and 19-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
Step 2A Prong 1:
“generating, based on the plurality of document elements, a hypergraph that includes a plurality of nodes and a plurality of hyperedges, wherein a hyperedge of the plurality of hyperedges connects the plurality of nodes” -- This limitation is directed to identifying the elements of a document and grouping the identified elements into sets, where each set (i.e., a hyperedge) connects the elements (i.e., nodes) that belong to it. The specification describes decomposing a document into fragments and fine-grained entities (e.g., button style, text style, words, image), treating the entities as nodes, and encoding the set of entities extracted from a fragment as a hyperedge, using node extraction rules that are custom designed (see specification paragpraphs [0036], [0091], [0093]). As claimed, this step is not tied to any particular machine and is recited for a single document. A person can practically observe the elements of a document, evaluate which of those elements belong together (e.g., within the same section of the document), and record each such group as a hyperedge connecting those elements, with the aid of pen and paper. Therefore, this limitation is a process that can be performed in the human mind using evaluation, observation, and judgment, and therefore is directed to a mental process.
“performing, by a hypergraph neural network, a node hypergraph convolution based on the hypergraph to obtain an updated node embedding for a node of the plurality of nodes by generating a plurality of hyperedge-dependent node embeddings corresponding to the plurality of hyperedges, respectively” -- This limitation is directed to node hypergraph convolution and to the generation of hyperedge-dependent node embeddings, which are mathematical operations performed over hypergraph-structured data. The specification discloses that the node hypergraph convolution computes the updated node embeddings from the incidence matrix, the diagonal degree matrices, the random walk node and hyperedge transition matrices, the hyperedge embeddings, and a learned weight matrix (see Eqs. 15-20 and SPEC [0118]-[0121]). The specification further discloses that each hyperedge-dependent node embedding is computed by applying a function ψ to the node embedding and the hyperedge embedding, or to their concatenation, to obtain a d-dimensional embedding of the node for each hyperedge in which the node participates (see Eqs. 23-25 and SPEC [0122]-[0125]). Although the claim does not recite these equations, a mathematical calculation may be recited in words, and when given its broadest reasonable interpretation in light of the specification, this limitation recites mathematical calculations. Therefore, this limitation recites a mathematical concept, and thus the limitation is directed to math.
“generating a predicted document element based on the augmented hypergraph” -- This limitation is directed to generating a prediction based on analyzed information. The specification describes, for example, generating a style recommendation for a document, such as recommending a button style (e.g., “round-shape corner button”) for a fragment of the document (see SPEC [0038], [0052], [0086]). As claimed, the limitation does not recite how the prediction is generated. A person can practically review the augmented hypergraph, evaluate which document elements are connected to one another, and predict a document element (e.g., a button style) that belongs with a given set of document elements, with the aid of pen and paper. Therefore, this limitation is a process that can be performed in the human mind using evaluation, observation, and judgment, and therefore is directed to a mental process.
Step 2A Prong 2 and Step 2B:
“obtaining a document including a plurality of document elements” -- This limitation is directed to obtaining data (a document and its document elements) for use in the recited judicial exception. Such activity amounts to mere data gathering and insignificant extra-solution activity, and therefore cannot integrate the judicial exception into a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, obtaining (i.e., receiving) data is a well-understood, routine, and conventional activity that the courts have recognized as such (e.g., receiving or transmitting data over a network), and it cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
“by a hypergraph neural network” -- This limitation recites that the node hypergraph convolution and the generation of the hyperedge-dependent node embeddings identified above are performed by a hypergraph neural network. The hypergraph neural network is recited at a high level of generality, as a generic machine learning model that carries out the recited mathematical calculations, and the claim does not recite any particular structure or operation of the network beyond the mathematical concept itself. The limitation amounts to no more than mere instructions to apply the judicial exception using a generic computer component, and it does not integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).
“wherein the updated node embedding for the node comprises the plurality of hyperedge-dependent node embeddings” -- This limitation specifies the form in which the result of the mathematical concept is maintained, namely that the updated node embedding for the node is made up of the plurality of hyperedge-dependent node embeddings. The specification describes this as collecting the hyperedge-dependent embeddings of a node into a hyperedge-dependent embedding matrix for that node (see SPEC [0125]). Specifying how the output of the judicial exception is collected and stored amounts to insignificant extra-solution activity, and therefore cannot integrate the judicial exception into a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, storing and maintaining the results of calculations as a collection of data is a well-understood, routine, and conventional activity that the courts have recognized as such (e.g., storing and retrieving information in memory; electronic recordkeeping), and it cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
“generating, by a hypergraph component, an augmented hypergraph based on the updated node embedding” -- This limitation recites generating an augmented hypergraph from the result of the judicial exception (the updated node embedding) using a hypergraph component. The limitation is recited at a high level of generality and recites only the idea of a solution or outcome (i.e., an augmented hypergraph), without any details of how the augmented hypergraph is generated from the updated node embedding, and the hypergraph component is a generic computer component. The limitation amounts to no more than mere instructions to apply the judicial exception using a generic computer component, and it does not integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).
Therefore, claim 1 is non-patent eligible. Claim 17 is analogous to claim 1, aside from claim type and minute differences, and thus the same rejection applies as above.
Regarding claim 2,
Step 1: The claim is directed to a method, which is one of four statutory categories as a process. The claim satisfies step 1.
Step 2A Prong 1:
“performing, by the hypergraph neural network, a hyperedge hypergraph convolution based on the hypergraph” -- The limitation is directed to performing convolution on a hyperedge/graph based on the original hypergraph, which is directed to a mathematical concept/operation. Thus, the limitation is directed to math.
Step 2A Prong 2 and Step 2B:
“to obtain an updated hyperedge embedding, wherein the augmented hypergraph is based on the updated hyperedge embedding.”-- The limitation recites obtaining an updated hyperedge embedding, wherein the augmented hypergraph is based on these updated embeddings. The limitation is directed to an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of obtaining updating embeddings and basing a hypergraph off of those is directed to a well-understood, routine, and conventional activity (WURC), and cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Thus, claim 2 is non-patent eligible. Claim 19 is analogous to claim 2, aside from claim type, and thus faces the same rejection.
Regarding claim 3,
Step 1: The claim is directed to a method, which is one of four statutory categories. Therefore, the claim satisfies step 1.
Step 2A Prong 1:
“The method of claim 1, further comprising: encoding, by a node encoder, the node and the hyperedge to obtain a preliminary node embedding and a preliminary hyperedge embedding, respectively” -- This limitation is directed to a mathematical concept. In particular, encoding nodes and hyperedges into embeddings involves mathematical transformations of input data into numerical vector representations for subsequent processing.
Step 2A Prong 2 and Step 2B:
“wherein the updated node embedding is based on the preliminary node embedding and the preliminary hyperedge embedding” -- This limitation merely further defines how the mathematical concept is performed using intermediate numerical representations. The limitation is directed to an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of updating node embeddings based on past embeddings (collected data) , is a well-understood, routine, and conventional activity (WURC), and it cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Thus, claim 3 is non-patent eligible. Claim 13 is analogous to claim 3, aside from claim type, and thus faces the same rejection.
Regarding claim 5,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
Step 2A Prong 1:
“The method of claim 1, further comprising: generating a hyperedge transition matrix, wherein the node hypergraph convolution is based on the hyperedge transition matrix.” -- The limitation is directed to generating a matrix representing transition relationships within a hypergraph and using a transition matrix as the basis for convolution over hypergraph data, which involves mathematical operations performed on numerical matrix representations. Thus, the limitation is directed to math.
There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.
Therefore, claim 5 is non-patent eligible. Claim 15 is analogous to claim 5, aside from claim type and minute differences, and thus the same rejection applies as above.
Regarding claim 6,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
Step 2A Prong 1:
“The method of claim 1, further comprising: generating a node transition matrix, wherein the node hypergraph convolution is based on the node transition matrix.” -- The limitation is directed to generating a matrix representing transition relationships within a hypergraph and using a transition matrix as the basis for convolution over hypergraph data, which involves mathematical operations performed on numerical matrix representations. Thus, the limitation is directed to math.
There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.
Therefore, claim 6 is non-patent eligible. Claim 16 is analogous to claim 6, aside from claim type and minute differences, and thus the same rejection applies as above.
Regarding claim 7,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“generating, by the hypergraph component, an additional hyperedge based on the updated node embedding,” -- This limitation recites generating additional data (an additional hyperedge) from the result of the judicial exception (the updated node embedding) using the hypergraph component. The limitation is recited at a high level of generality and recites only the idea of a solution or outcome (i.e., an additional hyperedge) without any details of how the additional hyperedge is generated from the updated node embedding, and the hypergraph component is a generic computer component. The limitation amounts to no more than mere instructions to apply the judicial exception using a generic computer component, and it does not integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).
“wherein the augmented hypergraph includes the additional hyperedge.” -- The limitation recites that the augmented hypergraph will include the additional hyperedge. The limitation amounts to no more than mere further limiting to a field of use/environment, and thus it cannot be integrated to a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(h)).
Therefore, claim 7 is non-patent eligible.
Regarding claim 8,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
Step 2A Prong 1:
“The method of claim 1, further comprising: generating a hyper-incidence matrix based on the hypergraph, wherein a preliminary node embedding for a node of the plurality of nodes and a preliminary hyperedge embedding for the hyperedge are based on the hyper-incidence matrix” -- The limitation is directed to generating an incidence matrix representing relationships between nodes and hyperedges and basing embeddings on a matrix representation, which involves mathematical transformations of data into numerical vector representations. This limitation is directed to a mathematical concept.
There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.
Therefore, claim 8 is non-patent eligible.
Regarding claim 9,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
Step 2A Prong 1:
“The method of claim 1, further comprising: generating a node diagonal degree matrix based on the hypergraph, wherein a preliminary node embedding for a node of the plurality of nodes and a preliminary hyperedge embedding for the hyperedge are based on the node diagonal degree matrix.” -- This limitation is directed to generating a diagonal degree matrix representing node degree relationships within a hypergraph and basing embeddings on a matrix representation involves mathematical transformations of data into numerical vector representations, which involves mathematical structures and operations. This limitation is directed to a mathematical concept.
There are no elements to be evaluated under Step 2A Prong 2 and Step 2B.
Therefore, claim 9 is non-patent eligible.
Regarding claim 11,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The method of claim 1, further comprising: providing a content item to a user based on the augmented hypergraph, wherein the user and the content item are represented by the plurality of nodes” -- This limitation is directed to outputting the result of the judicial exception. Such activity amounts to insignificant post-solution activity and therefore cannot integrate the judicial exception into a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of sending/receiving data over a network is a well-understood, routine, and conventional activity (WURC), and it cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 11 is non-patent eligible.
Regarding claim 12,
Step 1: The claim is directed to a method, which falls under the category of process. The claim satisfies step 1.
Step 2A Prong 1:
“performing, by a hypergraph neural network, a node hypergraph convolution based on the hypergraph to obtain a predicted node embedding for a node of the plurality of nodes by generating a plurality of predicted hyperedge-dependent node embeddings corresponding to the plurality of hyperedges, respectively” -- This limitation is directed to node hypergraph convolution and to the generation of predicted hyperedge-dependent node embeddings, which are mathematical operations performed over hypergraph-structured data. The specification discloses that the node hypergraph convolution computes the node embeddings from the incidence matrix, the diagonal degree matrices, the random walk node and hyperedge transition matrices, the hyperedge embeddings, and a learned weight matrix (see Eqs. 15-20 and SPEC [0118]-[0121]). The specification further discloses that each hyperedge-dependent node embedding is computed by applying a function ψ to the node embedding and the hyperedge embedding, or to their concatenation, to obtain a d-dimensional embedding of the node for each hyperedge in which the node participates (see Eqs. 23-25 and SPEC [0122]-[0125], and SPEC [0060], [0129]). Although the claim does not recite these equations, a mathematical calculation may be recited in words, and when given its broadest reasonable interpretation in light of the specification, this limitation recites mathematical calculations. Therefore, this limitation recites a mathematical concept, and thus the limitation is directed to math.
Step 2A Prong 2 and Step 2B:
“obtaining, by a training component, training data that includes a hypergraph including a plurality of nodes and a plurality of hyperedges, wherein a hyperedge of the plurality of hyperedges connects the plurality of nodes” -- This limitation recites obtaining, by a training component, training data that includes a hypergraph having a plurality of nodes and a plurality of hyperedges for use in the recited judicial exception. Such activity amounts to mere data gathering and insignificant extra-solution activity, and therefore cannot integrate the judicial exception into a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, obtaining (i.e., receiving) data is a well-understood, routine, and conventional activity that the courts have recognized as such (e.g., receiving or transmitting data over a network), and it cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
“by a hypergraph neural network” -- This limitation recites that the node hypergraph convolution and the generation of the predicted hyperedge-dependent node embeddings identified above are performed by a hypergraph neural network. The hypergraph neural network is recited at a high level of generality, as a generic machine learning model that carries out the recited mathematical calculations, and the claim does not recite any particular structure or operation of the network beyond the mathematical concept itself. The limitation amounts to no more than mere instructions to apply the judicial exception using a generic computer component, and it does not integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).
“wherein the predicted node embedding for the node comprises the plurality of predicted hyperedge-dependent node embeddings” -- This limitation specifies the form in which the result of the mathematical concept is maintained, namely that the predicted node embedding for the node is made up of the plurality of predicted hyperedge-dependent node embeddings. The specification describes this as collecting the hyperedge-dependent embeddings of a node into a hyperedge-dependent embedding matrix for that node (see SPEC [0125]). Specifying how the output of the judicial exception is collected and stored amounts to insignificant extra-solution activity, and therefore cannot integrate the judicial exception into a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, storing and maintaining the results of calculations as a collection of data is a well-understood, routine, and conventional activity that the courts have recognized as such (e.g., storing and retrieving information in memory; electronic recordkeeping), and it cannot provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
“training, by the training component, the hypergraph neural network based on the training data and the predicted node embedding” -- This limitation recites training the hypergraph neural network, by the training component, based on the training data and the predicted node embedding. The training is recited at a high level of generality, as the claim does not recite any particular manner in which the training is performed (e.g., a particular loss function or how the parameters of the network are adjusted) or any particular improvement to the training process itself; rather, the claim recites generic training of the network that produces the result of the mathematical concept. The limitation is no more than mere instructions to apply the judicial exception onto a computer in a high-level, generic manner, and it does not integrate the judicial exception into a practical application, nor provide significantly more than the judicial exception (see MPEP 2106.05(f)).
Therefore, claim 12 is non-patent eligible.
Regarding claim 20,
Step 1: The claim is directed to an apparatus, which is one of four statutory categories. Therefore, claim 20 satisfies step 1.
There are no elements to be evaluated under Step 2A Prong 1.
Step 2A Prong 2 and Step 2B:
“The apparatus of claim 17, further comprising: a training component configured to update parameters of the hypergraph neural network based on training data” -- This limitation recites a training component that will update parameters of a hypergraph based on gathered training data. The limitation is directed to an insignificant, extra-solution activity that cannot be integrated to a practical application (see MPEP 2106.05(g)). Furthermore, under Step 2B, the act of updating parameters based on gathered data is a well-understood, routine, and conventional activity (WURC), and it does not provide significantly more than the judicial exception (see MPEP 2106.05(d)(II)).
Therefore, claim 20 is non-patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over the NPL reference “Be More with Less: Hypergraph Attention Networks for Inductive Text Classification” by Ding et al. (referred herein as Ding) in view of the NPL reference “NHP: Neural Hypergraph Link Prediction” by Yadati et al. (referred herein as Yadati) further in view of the NPL reference “Classification of Edge-dependent Labels of Nodes in Hypergraphs” by Choe et al. (referred herein as Choe).
Regarding claim 1, Ding teaches:
A method comprising: obtaining a document including a plurality of document elements; ([Ding, page 4928, sec. 1] “we propose to adopt document-level hypergraph (hypergraph is a generalization of simple graph, in which a hyperedge can connect arbitrary number of nodes) for modeling each text document.” AND [Ding, page 4929, sec. 3.2] “For a text hypergraph, nodes represent words in the document and node attributes could be either one-hot vector or the pre-trained word embeddings (e.g., word2vec, GloVe). In order to model heterogeneous high-order context information within each document”, wherein the examiner interprets “each text document” and the “words in the document” to be the same as a document including a plurality of document elements, because they are both directed to obtaining a document composed of individually identifiable constituent elements for hypergraph modeling.)
generating, based on the plurality of document elements, a hypergraph that includes a plurality of nodes and a plurality of hyperedges, wherein a hyperedge of the plurality of hyperedges connects the plurality of nodes; ([Ding, page 4929, sec. 3.2] “here we consider each sentence as a hyperedge and it connects all the words in this sentence.” AND [Ding, page 4930, sec. 3.2] “Then for each topic, we consider it as a semantic hyperedge that connects the top K words with the largest probabilities in the document.”, wherein the examiner interprets constructing, for the document, a text hypergraph whose nodes are the words of the document and whose sequential (sentence) and semantic (topic) hyperedges each connect a plurality of those words to be the same as generating, based on the plurality of document elements, a hypergraph that includes a plurality of nodes and a plurality of hyperedges, wherein a hyperedge of the plurality of hyperedges connects the plurality of nodes, because they are both directed to deriving a hypergraph from the elements of a document in which each hyperedge groups multiple element nodes.)
performing, by a hypergraph neural network, a node hypergraph convolution based on the hypergraph to obtain an updated node embedding for a node of the plurality of nodes ([Ding, page 4930, sec. 3.3] “Apart from conventional GNN models, HyperGAT learns node representations with two different aggregation functions, allowing to capture heterogeneous high-order context information of words on text hypergraphs.” AND [Ding, page 4931, sec. 3.3] “we again apply an edge-level attention mechanism to highlight the informative hyperedges for learning the next-layer representation of node vi.” AND [Ding, page 4931, sec. 3.3] “where hli is the output representation of node vi”, wherein the examiner interprets HyperGAT, whose layer aggregates node features into hyperedge representations and then aggregates hyperedge representations back into each node to produce the “output representation of node vi,” to be the same as performing, by a hypergraph neural network, a node hypergraph convolution based on the hypergraph to obtain an updated node embedding for a node of the plurality of nodes, because they are both directed to a layer-wise neural aggregation over the node-hyperedge structure of a hypergraph that outputs an updated node representation.)
Ding does not teach by generating a plurality of hyperedge-dependent node embeddings corresponding to the plurality of hyperedges, respectively, wherein the updated node embedding for the node comprises the plurality of hyperedge-dependent node embeddings; generating, by a hypergraph component, an augmented hypergraph based on the updated node embedding; and generating a predicted document element based on the augmented hypergraph.
Yadati teaches:
by generating a plurality of hyperedge-dependent node embeddings corresponding to the plurality of hyperedges, respectively, ([Yadati, page 1708, sec. 4.1] “given a hyperlink e, we refine the embedding of each vertex v ∈ e using the following GCN equation of neural message-passing [13]: [Eq. 1]” AND [Yadati, page 1708, Fig. 3] “A GCN is then used to get hyperlink-aware embeddings of vertices (NaOH will have two embeddings).” AND [Yadati, page 1705, sec. 1] “We note that hyperedge can be used synonymously with hyperlink.”, wherein the examiner interprets the “hyperlink-aware embeddings of vertices” refined “given a hyperlink e” for “each vertex v ∈ e,” such that a vertex belonging to two hyperlinks “will have two embeddings,” to be the same as generating a plurality of hyperedge-dependent node embeddings corresponding to the plurality of hyperedges, respectively, because they are both directed to computing, for a single node, a separate learned embedding for each hyperedge that the node participates in.)
generating, by a hypergraph component, an augmented hypergraph based on the updated node embedding ([Yadati, page 1707, sec. 3.1] “The problem of hyperlink prediction in the incomplete undirected hypergraph H involves predicting missing hyperlinks” AND [Yadati, page 1708, sec. 4.1] “The hyperlink scoring function is precisely what makes NHP handle unseen links at test time.” AND [Yadati, page 1710, sec. 4.3] “Inference: At test time, NHP-U and NHP-D predict a hyperlink e as positive (existing) if its score is higher than the average score of the unobserved links” AND [Yadati, page 1711, sec. 5.1] “The link prediction task would be to complete a canonicalised knowledge graph”, wherein the examiner interprets completing the “incomplete” hypergraph with the “missing hyperlinks” predicted as positive (existing) by the hyperlink scoring layer, whose scores are computed from the hyperlink-aware vertex embeddings, to be the same as generating, by a hypergraph component, an augmented hypergraph based on the updated node embedding, because they are both directed to producing an expanded hypergraph that includes new hyperedges inferred from learned node embeddings.)
generating a predicted document element based on the augmented hypergraph ([Yadati, page 1706, Fig. 1] “d) shows a canonicalised knowledge link modelled as a directed hyperlink in which words/phrases are vertices, canonicalised words/phrases form a hyperlink” AND [Yadati, page 1711, sec. 5.1] “we aim to predict undiscovered canonicalised entities and relationships between pairs.”, wherein the examiner interprets predicting an undiscovered hyperlink formed of words/phrases (i.e., a predicted canonicalised entity) from the completed hypergraph, as applied to the text hypergraph of Ding whose nodes are the words of the document and whose hyperedges are the sentences and topics of the document, to be the same as generating a predicted document element based on the augmented hypergraph, because they are both directed to outputting a new element of textual content (a grouping of words/phrases) predicted from the hyperedges of the augmented hypergraph.)
Ding and Yadati do not teach wherein the updated node embedding for the node comprises the plurality of hyperedge-dependent node embeddings.
Choe teaches wherein the updated node embedding for the node comprises the plurality of hyperedge-dependent node embeddings ([Choe, page 1, Abstract] “we propose WHATsNet, a novel hypergraph neural network that represents the same node differently depending on the hyperedges it participates in by reflecting its varying importance in the hyperedges.” AND [Choe, page 4, Fig. 2] “WithinATT is applied to e1 and e2 independently. Even though the input feature of the node v2 is the same, the output is different within e1 and e2.” AND [Choe, page 6, sec. 5.4] “we can store the node and hyperedge embeddings separately and then concatenate them for any downstream task utilizing edge-dependent node embeddings.”, wherein the examiner interprets WHATsNet representing “the same node differently depending on the hyperedges it participates in,” such that the node v2 has a different output “within e1 and e2” and its edge-dependent node embeddings are maintained for “any downstream task utilizing edge-dependent node embeddings,” to be the same as wherein the updated node embedding for the node comprises the plurality of hyperedge-dependent node embeddings, because they are both directed to a representation of a node that is made up of the set of that node’s per-hyperedge embeddings.)
Ding, Yadati, Choe, and the instant application are analogous art because they are all directed to hypergraph neural networks that learn node representations from the node-hyperedge structure of a hypergraph for downstream prediction.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the document-level hypergraph attention network disclosed by Ding to include the “hyperlink-aware embeddings of vertices” and hyperlink scoring and prediction disclosed by Yadati. One would be motivated to do so to effectively preserve the higher-order relationships among the nodes of each hyperedge and to predict missing and unseen hyperedges of the hypergraph, as suggested by Yadati ([Yadati, page 1707, sec. 4.1] “NHP consists of a trainable hyperlink-aware GCN layer and a hyperlink scoring layer to preserve the higher-order relationships among the vertices in each hyperlink.” AND [Yadati, page 1706, sec. 1] “In contrast to non-neural baselines, NHP can predict unseen hyperlinks at test time (inductive hyperlink prediction).”).
It would have also been obvious to a person of ordinary skill in the art before the effective filing date of the invention to include the “edge-dependent node embeddings” disclosed by Choe. One would be motivated to do so to effectively represent each node according to its varying importance in each of the hyperedges it participates in, as suggested by Choe ([Choe, page 1, Abstract] “represents the same node differently depending on the hyperedges it participates in by reflecting its varying importance in the hyperedges.”).
Claim 17 is analogous to claim 1, aside from claim type and minute differences, and thus the same rejection applies as above.
Claims 2 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Ding in view of Yadati in view of Choe in view of US 20230037388 A1 by Sakhinana et al. (referred herein as Sakhinana) further in view of US 20230342918 A1 by Wang et al. (referred herein as Wang).
Regarding claim 2, Ding, Yadati, and Choe teach The method of claim 1 (see rejection of claim 1).
Ding, Yadati, and Choe do not teach wherein performing, by the hypergraph neural network, a hyperedge hypergraph convolution based on the hypergraph, to obtain an updated hyperedge embedding, wherein the augmented hypergraph is based on the updated hyperedge embedding.
Sakhinana teaches wherein performing, by the hypergraph neural network, a hyperedge hypergraph convolution based on the hypergraph, ([Sakhinana, [0031]] “The message passing phase generates neural messages and update node representations by aggregating encoded information of node's embeddings from confined graph neighborhood”, AND [0069] “performing attention over each node from amongst the set of nodes with a second set feature vectors associated with a set of inter-hyperedges within a global-inter neighborhood of the node to compute a plurality of inter-hyperedge neural-message aggregations”, wherein the examiner interprets “a second set feature vectors associated with a set of inter-hyperedges” and “compute a plurality of inter-hyperedge neural-message aggregations” to be the same as performing, by the hypergraph neural network, a hyperedge hypergraph convolution based on the hypergraph because they are both directed to hypergraph neural processing performed with respect to hyperedges of the hypergraph.)
Ding, Yadati, Choe, and Sakhinana do not teach to obtain an updated hyperedge embedding, wherein the augmented hypergraph is based on the updated hyperedge embedding.
Wang teaches:
to obtain an updated hyperedge embedding, ([Wang, [0008]] “performing vertex convolution calculation on the first hypergraph matrix and the second hypergraph matrix respectively to acquire a first hyperedge feature and a second hyperedge feature; performing hyperedge convolution calculation on the first hyperedge feature and the second hyperedge feature to acquire a fused feature;”, wherein the examiner interprets “acquire a first hyperedge feature and a second hyperedge feature” and “acquire a fused feature” to be the same as to obtain an updated hyperedge embedding because they are both directed to generating an updated learned representation associated with hyperedges after hyperedge-side convolution processing)
wherein the augmented hypergraph is based on the updated hyperedge embedding, ([Wang, [0008]] “performing hypergraph fusion on the non-Euclidean spacial feature and the Euclidean spacial feature to acquire the feature parameters…performing hypergraph data transformation on the non-Euclidean spacial feature and the Euclidean spacial feature respectively to acquire a first hypergraph matrix and a second hypergraph matrix; performing vertex convolution calculation on the first hypergraph matrix and the second hypergraph matrix respectively to acquire a first hyperedge feature and a second hyperedge feature; performing hyperedge convolution calculation on the first hyperedge feature and the second hyperedge feature to acquire a fused feature”, wherein the examiner interprets “performing hypergraph data transformation…acquire a first hypergraph matrix and a second hypergraph matrix…performing hyperedge convolution calculation on the first hyperedge feature and the second hyperedge feature to acquire a fused feature” to be the same as wherein the augmented hypergraph is based on the updated hyperedge embedding because they are both directed to generating or revising a hypergraph-level representation based on updated hyperedge-associated features produced by hyperedge convolution)
Ding, Yadati, Choe, Sakhinana, Wang, and the instant application are analogous art because they are all directed to hypergraph-based neural processing using node-related information, hyperedge-related information, and hypergraph structure.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 1 disclosed by Ding, Yadati, and Choe to include the “hyper-graph convolution using the HMPNN” disclosed by Sakhinana. One would be motivated to do so to efficiently learn embeddings on high-order hypergraph-structured data, as suggested by Sakhinana ([Sakhinana, [0044]] “A Hypergraph attention driven convolution, on molecular hypergraph results in learning efficient embeddings on the high-order molecular graph-structured data.”).
It would have also been obvious to a person of ordinary skill in the art before the effective filing date of the invention to include the “performing hyperedge convolution calculation on the first hyperedge feature and the second hyperedge feature to acquire a fused feature” disclosed by Wang. One would be motivated to do so to effectively generate updated hyperedge-associated representations and use those representations in hypergraph-level processing, as suggested by Wang ([Wang, [0008]] “performing hyperedge convolution calculation on the first hyperedge feature and the second hyperedge feature to acquire a fused feature.”).
Claim 19 is analogous to claim 2, aside from claim type and minute differences, and thus the same rejection applies as above.
Claims 3, 5-6, 8-9, 13, and 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Ding in view of Yadati in view of Choe further in view of Sakhinana.
Regarding claim 3, Ding, Yadati, and Choe teach The method of claim 1 (see rejection of claim 1).
Ding, Yadati, and Choe do not teach further comprising: encoding, by a node encoder, the node and the hyperedge to obtain a preliminary node embedding and a preliminary hyperedge embedding, respectively, wherein the updated node embedding is based on the preliminary node embedding and the preliminary hyperedge embedding.
Sakhinana teaches:
further comprising: encoding, by a node encoder, the node and the hyperedge to obtain a preliminary node embedding and a preliminary hyperedge embedding, respectively, ([Sakhinana. [0031] “The message passing phase generates neural messages and update node representations by aggregating encoded information of node's embeddings from confined graph neighborhood. AND [0041] “Hypergraph convolutions are designed to take G^H = (V^H(|V^H|)) nodes and N^H(|E^H|) hyperedges as input…F_k^{(L)} is the abstract representation of the k-th node in the (L)-th layer”, wherein the examiner interprets the hypergraph convolution taking “nodes and hyperedges as input” and producing “F_k^{(L)}” as the preliminary node representation along with the corresponding hyperedge-level aggregation via H^{HT} as the preliminary hyperedge representation to be the same as encoding, by a node encoder, the node and the hyperedge to obtain a preliminary node embedding and a preliminary hyperedge embedding, respectively because they are both directed to a processing component that jointly encodes both node and hyperedge information to produce initial vector representations for each.)
wherein the updated node embedding is based on the preliminary node embedding and the preliminary hyperedge embedding. ([Sakhinana, [0032] “and the received messages are perceived by the target node by performing mathematical computations to update its hidden representation.” AND [0059] “are the input of the (L)-th and (L+1)-th layer respectively. A symmetric normalization is put in to avoid exploding/vanishing gradient and thus, F (L+1)=σ(D H -1/2 H H W H B H -1 H H T D H -1/2 F (L) P) (Equation 10) and F(L+1) is differentiable with respect to F(L) and P.”, wherein the examiner interprets the updated node embedding “F^{(L+1)}” computed from both the preliminary node embedding “F^{(L)}” and the preliminary hyperedge embedding “H^{HT} F^{(L)}” through the incidence matrix H^H and message passing/node representation updating by aggregating encoded information to be the same as the updated node embedding is based on the preliminary node embedding and the preliminary hyperedge embedding because they are both directed to a formulation in which the updated node-level vector representation is computed by jointly aggregating and transforming both the preliminary node features and the preliminary hyperedge-level results.)
Ding, Yadati, Choe, Sakhinana, and the instant application are analogous art because they are all directed to hypergraph neural networks that learn node and hyperedge representations from the node-hyperedge structure of a hypergraph.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 1 disclosed by Ding, Yadati, and Choe to include the “update node representations by aggregating encoded information of node’s embeddings” disclosed by Sakhinana. One would be motivated to do so to efficiently learn node embeddings on high-order hypergraph-structured data, as suggested by Sakhinana ([Sakhinana, [0044]] “A Hypergraph attention driven convolution, on molecular hypergraph results in learning efficient embeddings on the high-order molecular graph-structured data.”). Claim 13 is analogous to claim 3, aside from claim type and minute differences, except that claim 13 depends from claim 12 rather than claim 1. The limitations of claim 12 are taught by Yadati and Choe as set forth in the rejection of claim 12 below, and Ding is cumulative as to those limitations; the limitations added by claim 13 are taught by Sakhinana as mapped for claim 3 above, and it would have been obvious to combine Sakhinana with Yadati and Choe for the same reasons given for claim 3. Thus, the same rejection applies as above.
Regarding claim 5, Ding, Yadati, and Choe teach The method of claim 1 (see rejection of claim 1).
Ding, Yadati, and Choe do not teach further comprising: generating a hyperedge transition matrix, wherein the node hypergraph convolution is based on the hyperedge transition matrix.
Sakhinana teaches:
further comprising: generating a hyperedge transition matrix, wherein the node hypergraph convolution is based on the hyperedge transition matrix. ([Sakhinana, [0011]] “Furthermore, the method includes, learning, in a plurality of iterations, a dynamic transient incidence matrix through a hypergraph-attention mechanism between a node and a set of hyperedges associated with the node of the hypergraph to perform a hyper-graph convolution using the HMPNN”, wherein the examiner interprets “a dynamic transient incidence matrix” to be the same as a hyperedge transition matrix because they are both directed to a learned matrix structure that encodes the connectivity and weighting relationships between nodes and their associated hyperedges in the hypergraph. The examiner further interprets “to perform a hyper-graph convolution” using the dynamic transient incidence matrix to be the same as the node hypergraph convolution is based on the hyperedge transition matrix because they are both directed to a node-level convolution operation that uses the hyperedge-derived transition matrix as its foundational propagation mechanism.)
Ding, Yadati, Choe, Sakhinana, and the instant application are analogous art because they are all directed to hypergraph neural networks that learn node and hyperedge representations from the node-hyperedge structure of a hypergraph.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 1 disclosed by Ding, Yadati, and Choe to include the “dynamic transient incidence matrix” disclosed by Sakhinana. One would be motivated to do so to effectively augment the scope of hypergraph representation learning, as suggested by Sakhinana ([Sakhinana, [0044]] “By taking into account the transient incidence matrix, the induced inductive bias augments the scope of molecular hypergraph representation learning.”). Claim 15 is analogous to claim 5, aside from claim type and minute differences, except that claim 15 depends from claim 12 rather than claim 1. The limitations of claim 12 are taught by Yadati and Choe as set forth in the rejection of claim 12 below, and Ding is cumulative as to those limitations; the limitations added by claim 15 are taught by Sakhinana as mapped for claim 5 above, and it would have been obvious to combine Sakhinana with Yadati and Choe for the same reasons given for claim 5. Thus, the same rejection applies as above.
Regarding claim 6, Ding, Yadati, and Choe teach The method of claim 1 (see rejection of claim 1).
Ding, Yadati, and Choe do not teach further comprising: generating a node transition matrix, wherein the node hypergraph convolution is based on the node transition matrix.
Sakhinana teaches:
further comprising: generating a node transition matrix, ([Sakhinana, [0067]] “Performing the attention within the local-intra neighborhood of the node includes evaluating a first transient incidence matrix. The first transient incidence matrix fill refers to the transient matrix learnt by computing pairwise attention coefficients between the node and its associated hyperedge.”, wherein the examiner interprets “a first transient incidence matrix” learnt by “computing pairwise attention coefficients between the node and its associated hyperedge” to be the same as a node transition matrix because they are both directed to a node-centric matrix that encodes the weighted transition relationships between each node and its directly connected hyperedge elements.)
wherein the node hypergraph convolution is based on the node transition matrix. ([Sakhinana, [0009]] “updating a set of hidden state vectors for each node of the set of nodes in the hyperedge by utilizing the plurality of intra-hyperedge neural-message aggregations”, wherein the examiner interprets “updating a set of hidden state vectors for each node of the set of nodes in the hyperedge by utilizing the plurality of intra-hyperedge neural-message aggregations” to be the same as the node hypergraph convolution is based on the node transition matrix because they are both directed to performing a node-level convolution operation that uses a node-derived transition matrix, namely the first transient incidence matrix, as the core propagation mechanism by which each node's hidden state representation is aggregated and updated through the hyperedge structure.)
Ding, Yadati, Choe, Sakhinana, and the instant application are analogous art because they are all directed to hypergraph neural networks that learn node and hyperedge representations from the node-hyperedge structure of a hypergraph.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 1 disclosed by Ding, Yadati, and Choe to include the “first transient incidence matrix” disclosed by Sakhinana. One would be motivated to do so to effectively augment the scope of hypergraph representation learning, as suggested by Sakhinana ([Sakhinana, [0044]] “By taking into account the transient incidence matrix, the induced inductive bias augments the scope of molecular hypergraph representation learning.”). Claim 16 is analogous to claim 6, aside from claim type and minute differences, except that claim 16 depends from claim 12 rather than claim 1. The limitations of claim 12 are taught by Yadati and Choe as set forth in the rejection of claim 12 below, and Ding is cumulative as to those limitations; the limitations added by claim 16 are taught by Sakhinana as mapped for claim 6 above, and it would have been obvious to combine Sakhinana with Yadati and Choe for the same reasons given for claim 6. Thus, the same rejection applies as above.
Regarding claim 8, Ding, Yadati, and Choe teach The method of claim 1 (see rejection of claim 1).
Ding, Yadati, and Choe do not teach further comprising: generating a hyper-incidence matrix based on the hypergraph, wherein a preliminary node embedding for a node of the plurality of nodes and a preliminary hyperedge embedding for the hyperedge are based on the hyper-incidence matrix.
Sakhinana teaches:
further comprising: generating a hyper-incidence matrix based on the hypergraph, ([Sakhinana, [0010]] “graph cached in a scaling diagonal matrix. Furthermore learn, in a plurality of iterations, a dynamic transient incidence matrix through a hypergraph-attention mechanism between a node and a set of hyperedges associated with the node of the hypergraph”, wherein the examiner interprets “a dynamic transient incidence matrix” and “a set of hyperedges attached with the node of the hypergraph” to be the same as generating a hyper-incidence matrix based on the hypergraph because they are both directed to generating an incidence matrix from node-hyperedge relationships of the hypergraph.)
wherein a preliminary node embedding for a node of the plurality of nodes and a preliminary hyperedge embedding for the hyperedge are based on the hyper-incidence matrix. ([Sakhinana, [0009]] “performing attention over each node of a set of nodes from amongst the plurality of nodes with a first set of feature vectors associated with a hyperedge within a local-intra neighborhood of the node to compute a plurality of intra-hyperedge neural-message aggregations. Performing the attention within the local-intra neighborhood of the node comprises evaluating a first transient incidence matrix … perform attention over each node from amongst the set of nodes with a second set feature vectors associated with a set of inter-hyperedges within a global-inter neighborhood of the node to compute a plurality of inter-hyperedge neural-message aggregations, wherein performing the attention within the global-inter neighborhood of the node comprises evaluating a second transient incidence matrix;”, wherein the examiner interprets “a first set of feature vectors” used while “evaluating a first transient incidence matrix” to be the same as a preliminary node embedding for a node of the plurality of node ...based on the hyper-incidence matrix because they are both directed to node-associated feature representations that are produced or used on the basis of an incidence matrix derived from the hypergraph. The examiner further interprets “a second set feature vectors associated with a set of inter-hyperedges” used while “evaluating a second transient incidence matrix” to be the same as a preliminary hyperedge embedding for the hyperedge ... based on the hyper-incidence matrix because they are both directed to hyperedge-associated feature representations that are produced or used on the basis of an incidence matrix derived from the hypergraph)
Ding, Yadati, Choe, Sakhinana, and the instant application are analogous art because they are all directed to hypergraph neural networks that learn node and hyperedge representations from the node-hyperedge structure of a hypergraph.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 1 disclosed by Ding, Yadati, and Choe to include the “dynamic transient incidence matrix” disclosed by Sakhinana. One would be motivated to do so to effectively augment the scope of hypergraph representation learning, as suggested by Sakhinana ([Sakhinana, [0044]] “By taking into account the transient incidence matrix, the induced inductive bias augments the scope of molecular hypergraph representation learning.”).
Regarding claim 9, Ding, Yadati, and Choe teach The method of claim 1 (see rejection of claim 1).
Ding, Yadati, and Choe do not teach further comprising: generating a node diagonal degree matrix based on the hypergraph, wherein a preliminary node embedding for a node of the plurality of nodes and a preliminary hyperedge embedding for the hyperedge are based on the node diagonal degree matrix.
Sakhinana teaches:
further comprising: generating a node diagonal degree matrix based on the hypergraph, ([Sakhinana, [0056]] “The node degree of the hypergraph G^H is described as D_{kk}^H = Σ_{j=1}^{M^H} H_{kj}^H. Here, D^H ∈ R^{M^H×M^H} and B^H ∈ R^{N^H×N^ H} are both diagonal matrices”, wherein the examiner interprets “D^H ∈ R^{M^H×M^H}” with entries “D_{kk}^H = Σ_{j=1}^{M^H} H_{kj}^H” derived from the hypergraph G^H to be the same as a node diagonal degree matrix based on the hypergraph because they are both directed to computing a diagonal matrix whose entries encode the degree of each node in the hypergraph, where the node degree is determined by counting the number of hyperedges each node belongs to, yielding a square diagonal matrix derived directly from the hypergraph structure.)
wherein a preliminary node embedding for a node of the plurality of nodes ([Sakhinana, [0057]] “Here, Fk (L) is the abstract representation of the k-th node in the (L)-th layer.”, wherein the examiner interprets “the abstract representation of the k-th node” to be the same as a preliminary node embedding for a node of the plurality of nodes because they are both directed to a node-level representation used in the hypergraph neural processing.)
and a preliminary hyperedge embedding for the hyperedge are based on the node diagonal degree matrix. ([Sakhinana, [0058-0059]] “The hypergraph convolution can be expressed in a matrix form as :.. A symmetric normalization is put in to avoid exploding/vanishing gradient” and [Sakhinana, [0064]] “Here, F^(ε H) denotes the static-hyperedge feature matrix, and a is a weight vector used to output a scalar attention value.”, wherein the examiner interprets “the static-hyperedge feature matrix” to be the same as a preliminary hyperedge embedding for the hyperedge because they are both directed to a hyperedge-associated representation used in the hypergraph neural processing. The examiner further interprets the hypergraph convolution expressed in matrix form using “Dv” to be the same as are based on the node diagonal degree matrix because they are both directed to using the node diagonal degree matrix within the matrix-form hypergraph convolution that operates on the node representation and the hyperedge representation.)
Ding, Yadati, Choe, Sakhinana, and the instant application are analogous art because they are all directed to hypergraph neural networks that learn node and hyperedge representations from the node-hyperedge structure of a hypergraph.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 1 disclosed by Ding, Yadati, and Choe to include the “diagonal matrices” disclosed by Sakhinana. One would be motivated to do so to effectively avoid exploding or vanishing gradients during hypergraph convolution, as suggested by Sakhinana ([Sakhinana, [0059]] “A symmetric normalization is put in to avoid exploding/vanishing gradient”).
Claims 7, 11, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ding in view of Yadati in view of Choe further in view of US 11816618 B1 by Cheek et al. (referred herein as Cheek).
Regarding claim 7, Ding, Yadati, and Choe teach The method of claim 1 (see rejection of claim 1).
Ding, Yadati, and Choe do not teach further comprising: generating, by the hypergraph component, an additional hyperedge based on the updated node embedding, wherein the augmented hypergraph includes the additional hyperedge.
Cheek teaches:
further comprising: generating, by the hypergraph component, an additional hyperedge ([Cheek col 2, lines 17-20] “When the category corresponds to a node, the system may identify a timestamp for the electronic object, and it will update the hypergraph by assigning the electronic object to an edge of the corresponding node”, wherein the examiner interprets “assigning the electronic object to an edge of the corresponding node” to be the same as generating, by the hypergraph component, an additional hyperedge because they are both directed to adding edge structure to the hypergraph in association with a node.)
based on the updated node embedding ([Cheek, col. 2, lines 53-58] “the workflow management system may associate one or more compressed context representations with one or more of the electronic objects that are assigned to hypergraph data, and therefore assign one or more compressed context representations to one or more nodes, edges or pins of the graph”, wherein the examiner interprets “update a set of hidden state vectors for each node of the set of nodes” and “compressed context representations” assigned to “one or more nodes” to be the same as updated node embedding because they are both directed to updated node-associated representations used in subsequent hypergraph processing.)
wherein the augmented hypergraph includes the additional hyperedge. ([Cheek, col. 2, lines 18-20] “it will update the hypergraph by assigning the electronic object to an edge of the corresponding node”, wherein the examiner interprets “update the hypergraph” and “edge of the corresponding node” to be the same as wherein the augmented hypergraph includes the additional hyperedge because they are both directed to a modified hypergraph that includes an added edge structure.)
Ding, Yadati, Choe, Cheek, and the instant application are analogous art because they are all directed to hypergraph-based processing that includes modifying hypergraph structure.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 1 disclosed by Ding, Yadati, and Choe to include the hypergraph updating technique disclosed by Cheek. One would be motivated to do so to effectively update and expand the hypergraph structure by adding edge connections associated with nodes, as suggested by Cheek ([Cheek, col. 2, lines 18-20] “it will update the hypergraph by assigning the electronic object to an edge of the corresponding node.”)
Regarding claim 11, Ding, Yadati, and Choe teach The method of claim 1 (see rejection of claim 1).
Ding, Yadati, and Choe do not teach further comprising: providing a content item to a user based on the augmented hypergraph, wherein the user and the content item are represented by the plurality of nodes.
Cheek teaches:
further comprising: providing a content item to a user ([Cheek, col 2, lines 38-41] “when selected, will cause an electronic device of which the user interface is a component to display the object via an application that generated the electronic object to which the edge associated with the pin is assigned”, wherein the examiner interprets “display the object” to be the same as providing a content item to a user because they are both directed to presenting an item of electronic content to a user through a user interface.)
based on the augmented hypergraph, wherein the user and the content item are represented by the plurality of nodes. ([Cheek, col 2, Lines, 9-24] “The hypergraph data also includes edges that are associated with one or more of the nodes and that correspond to one or more of the electronic objects. The workflow management system saves the hypergraph data to a memory. When the workflow management system receives a new electronic object, it will assign a category to the new electronic object, and it will determine whether the category corresponds to one or more nodes of the hypergraph data. When the category corresponds to a node, the system may identify a timestamp for the electronic object, and it will update the hypergraph by assigning the electronic object to an edge of the corresponding node, optionally with a chronological location that corresponds to the timestamp. A workflow presentation system will cause a display device to output a graphical user interface that includes a hypergraph constructed from the hypergraph data.”, wherein the examiner interprets “update the hypergraph” and “a hypergraph constructed from the hypergraph data” to be the same as based on the augmented hypergraph because they are both directed to presenting information using a hypergraph that has been updated and then used as the basis for display. The examiner further interprets “people” to be the same as the user and “electronic objects” to be the same as the content item because they are both directed to entities represented as nodes in the hypergraph)
Ding, Yadati, Choe, Cheek, and the instant application are analogous art because they are all directed to hypergraph-based processing that includes modifying hypergraph structure.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the method of claim 1 disclosed by Ding, Yadati, and Choe to include the object display technique disclosed by Cheek. One would be motivated to do so to effectively present hypergraph-associated content to users through a user interface based on relationships encoded in the hypergraph, as suggested by Cheek ([Cheek, col 2, lines 38-41] “display the object via an application that generated the electronic object”).
Regarding claim 20, Ding, Yadati, and Choe teach The apparatus of claim 17 (see rejection of claim 1, as claim 17 is analogous to claim 1).
Ding, Yadati, and Choe do not teach further comprising: a training component configured to update parameters of the hypergraph neural network based on training data.
Cheek teaches further comprising: a training component configured to update parameters of the hypergraph neural network based on training data ([Cheek, col 19, lines 40-45] “If the system receives such a 40 reassignment, it may provide the new object-workstream association to the machine learning model (step 504) so that the model may learn from the correction and use the result of the reassignment to make better workstream assignment decisions on future objects.” AND [Cheek, Fig 5] “RECEIVE USER REASSIGNMENT OF OBJECТ 503 … TRAIN MODEL USING REASSIGNMENT OF OBJECТ 504”, wherein the examiner interprets “REASSIGNMENT OF OBJECT” used in “TRAIN MODEL USING REASSIGNMENT OF OBJECT”, to be the same as a training component configured to update parameters of the hypergraph neural network based on training data because they are all directed to training a machine-learning model by a processing component.)
Ding, Yadati, Choe, Cheek, and the instant application are analogous art because they are all directed to hypergraph-based machine learning and training of neural network models.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the apparatus of claim 17 disclosed by Ding, Yadati, and Choe to include the correction and reassignment approach disclosed by Cheek. One would be motivated to do so to effectively improve training of the hypergraph neural network using updated training signals, as suggested by Cheek ([Cheek, col 19, lines 42-45] “so that the model may learn from the correction and use the result of the reassignment to make better workstream assignment decisions.”).
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Yadati in view of Choe.
Regarding claim 12, Yadati teaches:
A method comprising: obtaining, by a training component, training data that includes a hypergraph including a plurality of nodes and a plurality of hyperedges, wherein a hyperedge of the plurality of hyperedges connects the plurality of nodes; ([Yadati, page 1708, sec. 4.1] “Optimisation: Hyperlinks in the input hypergraph represent known relationships among the vertices of the hyperlink.” AND [Yadati, page 1705, sec. 1] “Hyperlink prediction 1 is the problem of predicting missing high-order relationships in a hypergraph.” AND [Yadati, page 1705, sec. 1] “We note that hyperedge can be used synonymously with hyperlink.”, wherein the examiner interprets the “input hypergraph” whose observed “Hyperlinks” are used in the “Optimisation” of NHP to be the same as training data that includes a hypergraph including a plurality of nodes and a plurality of hyperedges, and interprets each hyperlink representing “relationships among the vertices of the hyperlink” to be the same as a hyperedge of the plurality of hyperedges that connects the plurality of nodes, because they are both directed to obtaining, for training a neural network, a hypergraph whose hyperedges each connect multiple nodes.)
performing, by a hypergraph neural network, a node hypergraph convolution based on the hypergraph to obtain a predicted node embedding for a node of the plurality of nodes ([Yadati, page 1705, Abstract] “NHP adapts GCNs for link prediction in hypergraphs.” AND [Yadati, page 1707, sec. 4.1] “NHP consists of a trainable hyperlink-aware GCN layer and a hyperlink scoring layer to preserve the higher-order relationships among the vertices in each hyperlink.” AND [Yadati, page 1708, sec. 4.1] “We propose to address the above issue by refining the embeddings with a GCN layer on the subgraph obtained from the clique expansion of each hyperlink”, wherein the examiner interprets NHP, a graph convolutional network adapted to hypergraphs, to be the same as a hypergraph neural network, and interprets the “hyperlink-aware GCN layer” that refines the vertex embeddings over each hyperlink of the input hypergraph to be the same as performing a node hypergraph convolution based on the hypergraph to obtain a predicted node embedding for a node, because they are both directed to a neural convolution over the node-hyperedge structure of a hypergraph that outputs learned node representations (see also SPEC [0028], defining node hypergraph convolution as a process of extracting high-order data correlation information for representation learning related to nodes in a hypergraph).)
by generating a plurality of predicted hyperedge-dependent node embeddings corresponding to the plurality of hyperedges, respectively, ([Yadati, page 1708, sec. 4.1] “given a hyperlink e, we refine the embedding of each vertex v ∈ e using the following GCN equation of neural message-passing” AND [Yadati, page 1708, Fig. 3] “A GCN is then used to get hyperlink-aware embeddings of vertices (NaOH will have two embeddings).” AND [Yadati, page 1705, sec. 1] “We note that hyperedge can be used synonymously with hyperlink.”, wherein the examiner interprets the “hyperlink-aware embeddings of vertices” refined “given a hyperlink e” for “each vertex v ∈ e,” such that a vertex belonging to two hyperlinks “will have two embeddings,” to be the same as generating a plurality of predicted hyperedge-dependent node embeddings corresponding to the plurality of hyperedges, respectively, because they are both directed to computing, for a single node, a separate learned embedding for each hyperedge that the node participates in.)
and training, by the training component, the hypergraph neural network based on the training data and the predicted node embedding. ([Yadati, page 1708, sec. 4.1] “Following prior work [26], we rely on a ranking objective as follows:” AND [Yadati, page 1709, sec. 4.1] “It ranks the observed hyperlinks above the unobserved ones. All weights of NHP-U … are learned end-to-end using stochastic gradient descent.” AND [Yadati, page 1708, sec. 4.1] “Intuitively, the score, Ie , for hyperlink e, ideally, needs to be higher than that for any set of vertices that does not form a hyperlink in the hypergraph.”, wherein the examiner interprets learning all weights of NHP-U end-to-end by stochastic gradient descent on the ranking objective, which ranks the scores of the observed hyperlinks of the input hypergraph (computed from the hyperlink-aware vertex embeddings) above those of sampled non-hyperlinks, to be the same as training, by the training component, the hypergraph neural network based on the training data and the predicted node embedding, because they are both directed to updating the parameters of the hypergraph neural network using a loss computed from the input hypergraph and the node embeddings output by the network.)
Yadati does not teach wherein the predicted node embedding for the node comprises the plurality of predicted hyperedge-dependent node embeddings.
Choe teaches wherein the predicted node embedding for the node comprises the plurality of predicted hyperedge-dependent node embeddings ([Choe, page 1, Abstract] “we propose WHATsNet, a novel hypergraph neural network that represents the same node differently depending on the hyperedges it participates in by reflecting its varying importance in the hyperedges.” AND [Choe, page 4, Fig. 2] “WithinATT is applied to e1 and e2 independently. Even though the input feature of the node v2 is the same, the output is different within e1 and e2.” AND [Choe, page 6, sec. 5.4] “we can store the node and hyperedge embeddings separately and then concatenate them for any downstream task utilizing edge-dependent node embeddings.”, wherein the examiner interprets WHATsNet representing “the same node differently depending on the hyperedges it participates in,” such that the node v2 has a different output “within e1 and e2” and its edge-dependent node embeddings are maintained for “any downstream task utilizing edge-dependent node embeddings,” to be the same as wherein the predicted node embedding for the node comprises the plurality of predicted hyperedge-dependent node embeddings, because they are both directed to a representation of a node that is made up of the set of that node’s per-hyperedge embeddings.)
Yadati, Choe, and the instant application are analogous art because they are all directed to hypergraph neural networks that learn node representations from the node-hyperedge structure of a hypergraph.
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to modify the hyperlink-aware GCN disclosed by Yadati to include the “edge-dependent node embeddings” disclosed by Choe. One would be motivated to do so to effectively represent each node according to its varying importance in each of the hyperedges it participates in, as suggested by Choe ([Choe, page 1, Abstract] “represents the same node differently depending on the hyperedges it participates in by reflecting its varying importance in the hyperedges.”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEVAN KAPOOR whose telephone number is (703)756-1434. The examiner can normally be reached Monday - Friday: 9:00AM - 5:00 PM EST (times may vary).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DEVAN KAPOOR/Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights