Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendments filed 03/17/2026 have been entered. They provide amendments to claims 1, 4, 7, 11-16, 18-20, cancel claims 5-6, 8-10, 17 and adds new claims 21-16. Claims 1-4, 7, 11-16, 18-26 are pending. The amendments to the claims overcomes the 101 rejection.
Response to Arguments
Applicant’s arguments with respect to 35 U.S.C § 102 and 103 filed 03/17/2026 (pages 24-26 of applicant’s arguments) have been fully considered but they are not persuasive (specifically that the cited prior art does not teach that the neighbor nodes are neighbors of the source entity).
Applicant argues (page 25) that Bayram (US 20240256917 A1) does not teach “receiving a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to a neighbor entity, the neighbor entity being a neighbor to the source entity”). The examiner respectfully disagrees. In Bayram they identify neighboring nodes of the target node. The target node is the node that is being embedded which can either be a subject node or an object node. The subject node is equivalent to the claimed source entity and those Bayram does teach identifying the neighbors of the source entity.
The rest of applicant’s arguments with respect to 35 U.S.C § 102 and 103 filed 03/17/2026 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7, 11-16, 18-26 are rejected under 35 U.S.C. 103 as being unpatentable over Nguyen Thi (NPL: ‘Knowledge Base Completion with transfer learning using BERT and fastText’ (accessed through applicant IDS)) in view of Bayram (US 20240256917 A1) and Markowitz (NPL: ‘StATIK: Structure and Text for Inductive Knowledge Graph Completion’ (2022)).
Regarding claim 1, Nguyen Thi Teaches:
A computer-implemented method for supplementing a knowledge graph, comprising: (Abstract)
receiving a source-entity data item that provides a passage of source-entity text pertaining to a source entity that is connected to a yet-to-be-determined target entity via a source-target relation, wherein predicate encoded information encodes the source-target relation; (Figure. 3 triple Section II. KBC With Transfer Learning “An input of the word embedding model is a triple (h, r, t)”).
in an input encoder of a first neural network, mapping the source-entity data item to a language-based representation of the source-entity data item; (Section 1 Introduction “First, the authors use the transfer learning method to learn maps from entities and relations to their embeddings. They use the GloVe word embedding method to map each “word” (entity or relation) to a vector, then use them as input data of the entity encoder.”)
in a source entity encoder of the first neural network, mapping the language-based representation of the source-entity data item to source-entity encoded information; (Figure 3 Encoder. Section II. KBC With Transfer Learning “We construct the encoder model using a combination of GRU and FC layers to learn the representation of a triple embedding. FC layers in a neural network are layers where all inputs from one layer are connected to every activation unit of the next layer. When an input vector embedding (e_h,e_r,e_t) is crossed GRU, some essential features will be extracted. After that, these features are filtered again by FC layers, and an output of this model is the triple vector embedding (v_h, v_r, v_t)”).
Nguyen This does not teach:
yet-to-be-determined target entity
receiving a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to a neighbor entity, the neighbor entity being a neighbor to the source entity that is connected to the source entity by a neighbor relation, wherein neighbor-relation encoded information encodes the neighbor relation;
in another input encoder of the first neural network, mapping the neighbor-entity data item to a language-based representation of the neighbor-entity data item;
in a neighbor entity encoder of the first neural network, mapping the language- based representation of the neighbor-entity data item to neighbor-entity encoded information, wherein the language-based representation of the source-entity data item and the language-based representation of the neighbor-entity data item are each formed using a vocabulary of tokens of a natural language;
in an input encoder of a second neural network, assembling input information that includes a sequence of embeddings based on the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information;
in a neighbor encoder of the second neural network, mapping the input information to neighbor-aware source-entity information; and
producing output information that identifies the target entity by identifying a vector that is closest to the neighbor-aware source-entity information.
Bayram does teach:
receiving a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to a neighbor entity, the neighbor entity being a neighbor to the source entity that is connected to the source entity by a neighbor relation, wherein neighbor-relation encoded information encodes the neighbor relation; ([0067] “For each identified target node, the subgraph generator 302 is configured identify a list of k-hop neighboring nodes of the target node, e.g., using the knowledge graph triples 218.”).
in another input encoder of the first neural network, mapping the neighbor-entity data item to a language-based representation of the neighbor-entity data item; ([0069]-[0070] describes the subgraph encoder which takes the neighborhood embeddings and produces and encoding. “The subgraph encoder 214 is also configured to receive triples for the k-hop subgraph 312 from the subgraph generator 302, as well as the entity types 224 and the ontology embeddings 222. The subgraph encoder 214 is configured to process these inputs using multiple relational message-passing layers 316 and a relational attention mechanism to generate, as output, an embedding of the target node”)
in a neighbor entity encoder of the first neural network, mapping the language- based representation of the neighbor-entity data item to neighbor-entity encoded information, wherein the language-based representation of the source-entity data item and the language-based representation of the neighbor-entity data item are each formed using a vocabulary of tokens of a natural language; ([0100] “For example, the knowledge completion task can be the task of predicting a link between a node that was observed in the triples used to train the ontology-driven neural link prediction system and another node that was not observed in the triples used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a new knowledge graph triple, where the new knowledge graph triple includes a subject node or an object node that was not included in the set of triples. As another example, the knowledge completion task can be the task of predicting links from a new knowledge graph that is different to the knowledge graph used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a triple from a new knowledge graph, where the new knowledge graph has a same ontology (e.g., entity types and relation types) as the knowledge graph represented by the set of triples.”)
Nguyen Thi and Bayram are considered analogous art to the claimed invention because they are in the same field of endeavor being knowledge graph completion. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the system and model of Nguyen Thi with the method of implementing neighboring nodes of Bayram. One would want to do this for a more effective knowledge graph completion.
Neither Nguyen Thi nor Bayram teaches:
yet-to-be-determined target entity
in an input encoder of a second neural network, assembling input information that includes a sequence of embeddings based on the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information;
in a neighbor encoder of the second neural network, mapping the input information to neighbor-aware source-entity information; and
producing output information that identifies the target entity by identifying a vector that is closest to the neighbor-aware source-entity information.
However, Markowitz does:
yet-to-be-determined target entity (Page 4 section 4.2.2 “For each query of the form (h, r,?) or (?,r,t), let s be the “query entity”, meaning the head or tail that is part of the query (or the target entity if encoding target candidates). We then form a subgraph around s using the edges, E, connecting s to its neighbors, N(s).”)
in an input encoder of a second neural network, assembling input information that includes a sequence of embeddings based on the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information; (Page 5 section 4.2.5 “We combine the MPNN and the language model in a sequential manner, in which the output of the language model is used to replace the features for the entities being queried (query entities and target entities). These are the same entities whose representations will eventually be used by the decoder.”)
in a neighbor encoder of the second neural network, mapping the input information to neighbor-aware source-entity information; and (Page 5 section 4.2.5 “We combine the MPNN and the language model in a sequential manner, in which the output of the language model is used to replace the features for the entities being queried (query entities and target entities). These are the same entities whose representations will eventually be used by the decoder.”)
producing output information that identifies the target entity by identifying a vector that is closest to the neighbor-aware source-entity information. (page 5 section 4.2.2 “For the m edges in the subgraph, let i_h and i_t be E^m vectors indicating the heads and tails respectively of each edge.” And section 4.3 describes calculating the predictions score”)
Nguyen Thi, Bayram and Markowitz are considered analogous art to the claimed invention because they are in the same field of endeavor being knowledge graph completion. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the system and model of Nguyen Thi with the method of implementing neighboring nodes of Bayram with the embedding of Markowitz. One would want to do this to be able to easily encode all the source, relation and neighbor information.
Regarding claim 2, Nguyen Thi in view of Bayram and Markowitz teaches claim 1 as outlined above. Nguyen Thi further teaches:
at least the target entity is not yet represented by the knowledge graph, and wherein the computer- implemented method further includes adding a node associated with the target entity to the knowledge graph. (Section II. KBV With Transfer learning “Then the embeddings are the input of a KBC algorithm to predict their score using its loss function, and the input triple is missing if the result score is positive”).
Regarding claim 3, Nguyen Thi in view of Bayram and Markowitz teaches claim 1 as outlined above. Nguyen Thi further teaches:
the target entity is represented by the knowledge graph, and wherein the computer-implemented method is performed in a course of training the machine-trained encoder model. (Section III. Experimental Results and Evaluation they describe using complete datasets for training)
Regarding claim 4, Nguyen Thi in view of Bayram and Markowitz teaches claim 1 as outlined above. Nguyen Thi further teaches:
the first neural network and the second neural network include a set of weights that are trained in a first training operation that involves a first knowledge-graph completion task using examples mined from a first knowledge graph, (Page 4 left column “1) Word embedding model With the BERT scenario, we use BERT_large for the pre trained model to initialize triples embedding. The model is pre-trained by using the MLM. In this case, the size of vector embeddings is 100. With the fastText scenario, we use the trained model of 1M words from Wikipedia English language 2017, data warehouse web-based UMBC and dataset statmt.org. There are 16 billion tokens. All entities and relations trained by fastText will be initialized by corresponding word embeddings in fastText. All remaining words not in the pre-trained model will be randomly initialized. In this case, all entities and relations will be mapped into a vector space embedding whose dimension is 300.”)
the set of weights of the first neural network and the second neural network are refined in a second training operation that follows the first training operation and that involves a second knowledge-graph completion task using examples mined from a second knowledge graph, and (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different).
a set of entities associated with the first knowledge graph differs from a set of entities associated with the second knowledge graph, and/or a set of relations associated with the first knowledge graph differs from a set of relations associated with the second knowledge graph. (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different. See also Table 1 for information on different datasets).
Regarding claim 7, Nguyen Thi in view of Bayram and Markowitz teaches claim 4 as outlined above. Nguyen Thi further teaches:
at a start of the first training operation, the machine-trained encoder model includes a set weights that are trained with respect to a language-modeling task. (Section III. Experimental Results and Evaluation “To evaluate the performance of the proposed model, we conduct experiments with two following scenarios. The first scenario uses a pre-trained model to initialize parameters and embeddings, then fine-tune them.”)
Regarding claim 11, Nguyen Thi in view of Bayram and Markowitz teaches claim 1 as outlined above. Bayram further teaches:
the first neural network maps plural neighbor-entity data items to plural instances of neighbor-entity encoded information, and wherein the second neural network uses the plural instances of neighbor-entity encoded information to produce the neighbor-aware source-entity information. ([0068] “The subgraph generator 302 is configured to provide the neighboring node set of the k-hop subgraph 308 to the node embedding initializer 304. The node embedding initializer 304 is configured to receive the neighboring node set of the k-hop subgraph 308, as well as the set of entity types 224 for the knowledge graph and the ontology embeddings 222.”)
Regarding claim 12, Nguyen Thi in view of Bayram and Markowitz teaches claim 1 as outlined above. Nguyen Thi further teaches:
the first neural network and the second neural network use instances of attention-based logic, wherein each instance of attention-based logic interprets input information fed to the instance of attention-based logic by considering relations among different parts of the input information fed to the instance of attention-based logic. (Section II. KBC With Transfer Learning subsection B. The proposed KBC Model describes the BERT model used which is an attention based model.)
Regarding claim 13, Nguyen Thi in view of Bayram and Markowitz teaches claim 1 as outlined above. Nguyen Thi further teaches:
each of the first neural network and the second neural network is a transformer-based neural network. (Section II. KBC With Transfer Learning subsection B. The proposed KBC Model describes the BERT model used which is a transformer based model.)
Regarding claim 14, Nguyen Thi Teaches:
A computing system for providing content, comprising: a memory for storing computer-readable instructions; a store for storing a knowledge graph; a processing system, including a hardware processor, for executing the computer-readable instructions to perform operations that include: (They run experiments and simulations of their model implies a computing system is needed)
receiving a source-entity data item that provides a passage of source-entity text pertaining to a source entity that is connected to a yet-to-be-determined target entity via a source-target relation, wherein predicate encoded information encodes the source-target relation; (Figure. 3 triple Section II. KBC With Transfer Learning “An input of the word embedding model is a triple (h, r, t)”).
in an input encoder of a first neural network, mapping the source-entity data item to a language-based representation of the source-entity data item; (Section 1 Introduction “First, the authors use the transfer learning method to learn maps from entities and relations to their embeddings. They use the GloVe word embedding method to map each “word” (entity or relation) to a vector, then use them as input data of the entity encoder.”)
in a source entity encoder of the first neural network, mapping the language-based representation of the source-entity data item to source-entity encoded information; (Figure 3 Encoder. Section II. KBC With Transfer Learning “We construct the encoder model using a combination of GRU and FC layers to learn the representation of a triple embedding. FC layers in a neural network are layers where all inputs from one layer are connected to every activation unit of the next layer. When an input vector embedding (e_h,e_r,e_t) is crossed GRU, some essential features will be extracted. After that, these features are filtered again by FC layers, and an output of this model is the triple vector embedding (v_h, v_r, v_t)”).
Nguyen This does not teach:
yet-to-be-determined target entity
receiving a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to a neighbor entity, the neighbor entity being a neighbor to the source entity that is connected to the source entity by a neighbor relation, wherein neighbor-relation encoded information encodes the neighbor relation;
in another input encoder of the first neural network, mapping the neighbor-entity data item to a language-based representation of the neighbor-entity data item;
in a neighbor entity encoder of the first neural network, mapping the language- based representation of the neighbor-entity data item to neighbor-entity encoded information, wherein the language-based representation of the source-entity data item and the language-based representation of the neighbor-entity data item are each formed using a vocabulary of tokens of a natural language;
in an input encoder of a second neural network, assembling input information that includes a sequence of embeddings based on the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information;
in a neighbor encoder of the second neural network, mapping the input information to neighbor-aware source-entity information; and
producing output information that identifies the target entity by identifying a vector that is closest to the neighbor-aware source-entity information.
Bayram does teach:
receiving a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to a neighbor entity, the neighbor entity being a neighbor to the source entity that is connected to the source entity by a neighbor relation, wherein neighbor-relation encoded information encodes the neighbor relation; ([0067] “For each identified target node, the subgraph generator 302 is configured identify a list of k-hop neighboring nodes of the target node, e.g., using the knowledge graph triples 218.”).
in another input encoder of the first neural network, mapping the neighbor-entity data item to a language-based representation of the neighbor-entity data item; ([0069]-[0070] describes the subgraph encoder which takes the neighborhood embeddings and produces and encoding. “The subgraph encoder 214 is also configured to receive triples for the k-hop subgraph 312 from the subgraph generator 302, as well as the entity types 224 and the ontology embeddings 222. The subgraph encoder 214 is configured to process these inputs using multiple relational message-passing layers 316 and a relational attention mechanism to generate, as output, an embedding of the target node”)
in a neighbor entity encoder of the first neural network, mapping the language- based representation of the neighbor-entity data item to neighbor-entity encoded information, wherein the language-based representation of the source-entity data item and the language-based representation of the neighbor-entity data item are each formed using a vocabulary of tokens of a natural language; ([0100] “For example, the knowledge completion task can be the task of predicting a link between a node that was observed in the triples used to train the ontology-driven neural link prediction system and another node that was not observed in the triples used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a new knowledge graph triple, where the new knowledge graph triple includes a subject node or an object node that was not included in the set of triples. As another example, the knowledge completion task can be the task of predicting links from a new knowledge graph that is different to the knowledge graph used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a triple from a new knowledge graph, where the new knowledge graph has a same ontology (e.g., entity types and relation types) as the knowledge graph represented by the set of triples.”)
Nguyen Thi and Bayram are considered analogous art to the claimed invention because they are in the same field of endeavor being knowledge graph completion. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the system and model of Nguyen Thi with the method of implementing neighboring nodes of Bayram. One would want to do this for a more effective knowledge graph completion.
Neither Nguyen Thi nor Bayram teaches:
yet-to-be-determined target entity
in an input encoder of a second neural network, assembling input information that includes a sequence of embeddings based on the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information;
in a neighbor encoder of the second neural network, mapping the input information to neighbor-aware source-entity information; and
producing output information that identifies the target entity by identifying a vector that is closest to the neighbor-aware source-entity information.
However, Markowitz does:
yet-to-be-determined target entity (Page 4 section 4.2.2 “For each query of the form (h, r,?) or (?,r,t), let s be the “query entity”, meaning the head or tail that is part of the query (or the target entity if encoding target candidates). We then form a subgraph around s using the edges, E, connecting s to its neighbors, N(s).”)
in an input encoder of a second neural network, assembling input information that includes a sequence of embeddings based on the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information; (Page 5 section 4.2.5 “We combine the MPNN and the language model in a sequential manner, in which the output of the language model is used to replace the features for the entities being queried (query entities and target entities). These are the same entities whose representations will eventually be used by the decoder.”)
in a neighbor encoder of the second neural network, mapping the input information to neighbor-aware source-entity information; and (Page 5 section 4.2.5 “We combine the MPNN and the language model in a sequential manner, in which the output of the language model is used to replace the features for the entities being queried (query entities and target entities). These are the same entities whose representations will eventually be used by the decoder.”)
producing output information that identifies the target entity by identifying a vector that is closest to the neighbor-aware source-entity information. (page 5 section 4.2.2 “For the m edges in the subgraph, let i_h and i_t be E^m vectors indicating the heads and tails respectively of each edge.” And section 4.3 describes calculating the predictions score”)
Nguyen Thi, Bayram and Markowitz are considered analogous art to the claimed invention because they are in the same field of endeavor being knowledge graph completion. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the system and model of Nguyen Thi with the method of implementing neighboring nodes of Bayram with the embedding of Markowitz. One would want to do this to be able to easily encode all the source, relation and neighbor information.
Regarding claim 15, Nguyen Thi in view of Bayram and Markowitz teaches claim 14 as outlined above. Nguyen Thi further teaches:
the first neural network and the second neural network include a set of weights that are trained in a first training operation that involves a first knowledge-graph completion task using examples mined from a first knowledge graph, (Page 4 left column “1) Word embedding model With the BERT scenario, we use BERT_large for the pre trained model to initialize triples embedding. The model is pre-trained by using the MLM. In this case, the size of vector embeddings is 100. With the fastText scenario, we use the trained model of 1M words from Wikipedia English language 2017, data warehouse web-based UMBC and dataset statmt.org. There are 16 billion tokens. All entities and relations trained by fastText will be initialized by corresponding word embeddings in fastText. All remaining words not in the pre-trained model will be randomly initialized. In this case, all entities and relations will be mapped into a vector space embedding whose dimension is 300.”)
the set of weights of the first neural network and the second neural network are refined in a second training operation that follows the first training operation and that involves a second knowledge-graph completion task using examples mined from a second knowledge graph, and (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different).
a set of entities associated with the first knowledge graph differs from a set of entities associated with the second knowledge graph, and/or a set of relations associated with the first knowledge graph differs from a set of relations associated with the second knowledge graph. (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different. See also Table 1 for information on different datasets).
Regarding claim 16, Nguyen Thi in view of Bayram and Markowitz teaches claim 15 as outlined above. Nguyen Thi further teaches:
at a start of the first training operation, the machine-trained encoder model includes a set weights that are trained with respect to a language-modeling task. (Section III. Experimental Results and Evaluation “To evaluate the performance of the proposed model, we conduct experiments with two following scenarios. The first scenario uses a pre-trained model to initialize parameters and embeddings, then fine-tune them.”)
Regarding claim 18, Nguyen Thi teaches
receiving a source-entity data item that provides a passage of source-entity text pertaining to a source entity that is connected to a yet-to-be-determined target entity via a source-target relation, wherein predicate encoded information encodes the source-target relation; (Figure. 3 triple Section II. KBC With Transfer Learning “An input of the word embedding model is a triple (h, r, t)”).
in an input encoder of a first neural network, mapping the source-entity data item to a language-based representation of the source-entity data item; (Section 1 Introduction “First, the authors use the transfer learning method to learn maps from entities and relations to their embeddings. They use the GloVe word embedding method to map each “word” (entity or relation) to a vector, then use them as input data of the entity encoder.”)
in a source entity encoder of the first neural network, mapping the language-based representation of the source-entity data item to source-entity encoded information; (Figure 3 Encoder. Section II. KBC With Transfer Learning “We construct the encoder model using a combination of GRU and FC layers to learn the representation of a triple embedding. FC layers in a neural network are layers where all inputs from one layer are connected to every activation unit of the next layer. When an input vector embedding (e_h,e_r,e_t) is crossed GRU, some essential features will be extracted. After that, these features are filtered again by FC layers, and an output of this model is the triple vector embedding (v_h, v_r, v_t)”).
Nguyen This does not teach:
A non-transitory computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer- readable instructions to perform operations
yet-to-be-determined target entity
receiving a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to a neighbor entity, the neighbor entity being a neighbor to the source entity that is connected to the source entity by a neighbor relation, wherein neighbor-relation encoded information encodes the neighbor relation;
in another input encoder of the first neural network, mapping the neighbor-entity data item to a language-based representation of the neighbor-entity data item;
in a neighbor entity encoder of the first neural network, mapping the language- based representation of the neighbor-entity data item to neighbor-entity encoded information, wherein the language-based representation of the source-entity data item and the language-based representation of the neighbor-entity data item are each formed using a vocabulary of tokens of a natural language;
in an input encoder of a second neural network, assembling input information that includes a sequence of embeddings based on the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information;
in a neighbor encoder of the second neural network, mapping the input information to neighbor-aware source-entity information; and
producing output information that identifies the target entity by identifying a vector that is closest to the neighbor-aware source-entity information.
Bayram does teach:
A non-transitory computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer- readable instructions to perform operations ([0103] “The computer readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them.”)
receiving a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to a neighbor entity, the neighbor entity being a neighbor to the source entity that is connected to the source entity by a neighbor relation, wherein neighbor-relation encoded information encodes the neighbor relation; ([0067] “For each identified target node, the subgraph generator 302 is configured identify a list of k-hop neighboring nodes of the target node, e.g., using the knowledge graph triples 218.”).
in another input encoder of the first neural network, mapping the neighbor-entity data item to a language-based representation of the neighbor-entity data item; ([0069]-[0070] describes the subgraph encoder which takes the neighborhood embeddings and produces and encoding. “The subgraph encoder 214 is also configured to receive triples for the k-hop subgraph 312 from the subgraph generator 302, as well as the entity types 224 and the ontology embeddings 222. The subgraph encoder 214 is configured to process these inputs using multiple relational message-passing layers 316 and a relational attention mechanism to generate, as output, an embedding of the target node”)
in a neighbor entity encoder of the first neural network, mapping the language- based representation of the neighbor-entity data item to neighbor-entity encoded information, wherein the language-based representation of the source-entity data item and the language-based representation of the neighbor-entity data item are each formed using a vocabulary of tokens of a natural language; ([0100] “For example, the knowledge completion task can be the task of predicting a link between a node that was observed in the triples used to train the ontology-driven neural link prediction system and another node that was not observed in the triples used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a new knowledge graph triple, where the new knowledge graph triple includes a subject node or an object node that was not included in the set of triples. As another example, the knowledge completion task can be the task of predicting links from a new knowledge graph that is different to the knowledge graph used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a triple from a new knowledge graph, where the new knowledge graph has a same ontology (e.g., entity types and relation types) as the knowledge graph represented by the set of triples.”)
Nguyen Thi and Bayram are considered analogous art to the claimed invention because they are in the same field of endeavor being knowledge graph completion. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the system and model of Nguyen Thi with the method of implementing neighboring nodes of Bayram. One would want to do this for a more effective knowledge graph completion.
Neither Nguyen Thi nor Bayram teaches:
yet-to-be-determined target entity
in an input encoder of a second neural network, assembling input information that includes a sequence of embeddings based on the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information;
in a neighbor encoder of the second neural network, mapping the input information to neighbor-aware source-entity information; and
producing output information that identifies the target entity by identifying a vector that is closest to the neighbor-aware source-entity information.
However, Markowitz does:
yet-to-be-determined target entity (Page 4 section 4.2.2 “For each query of the form (h, r,?) or (?,r,t), let s be the “query entity”, meaning the head or tail that is part of the query (or the target entity if encoding target candidates). We then form a subgraph around s using the edges, E, connecting s to its neighbors, N(s).”)
in an input encoder of a second neural network, assembling input information that includes a sequence of embeddings based on the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information; (Page 5 section 4.2.5 “We combine the MPNN and the language model in a sequential manner, in which the output of the language model is used to replace the features for the entities being queried (query entities and target entities). These are the same entities whose representations will eventually be used by the decoder.”)
in a neighbor encoder of the second neural network, mapping the input information to neighbor-aware source-entity information; and (Page 5 section 4.2.5 “We combine the MPNN and the language model in a sequential manner, in which the output of the language model is used to replace the features for the entities being queried (query entities and target entities). These are the same entities whose representations will eventually be used by the decoder.”)
producing output information that identifies the target entity by identifying a vector that is closest to the neighbor-aware source-entity information. (page 5 section 4.2.2 “For the m edges in the subgraph, let i_h and i_t be E^m vectors indicating the heads and tails respectively of each edge.” And section 4.3 describes calculating the predictions score”)
Nguyen Thi, Bayram and Markowitz are considered analogous art to the claimed invention because they are in the same field of endeavor being knowledge graph completion. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the system and model of Nguyen Thi with the method of implementing neighboring nodes of Bayram with the embedding of Markowitz. One would want to do this to be able to easily encode all the source, relation and neighbor information.
Regarding claim 19, Nguyen Thi in view of Bayram and Markowitz teaches claim 18 as outlined above. Nguyen Thi further teaches:
the first neural network and the second neural network include a set of weights that are trained in a first training operation that involves a first knowledge-graph completion task using examples mined from a first knowledge graph, (Page 4 left column “1) Word embedding model With the BERT scenario, we use BERT_large for the pre trained model to initialize triples embedding. The model is pre-trained by using the MLM. In this case, the size of vector embeddings is 100. With the fastText scenario, we use the trained model of 1M words from Wikipedia English language 2017, data warehouse web-based UMBC and dataset statmt.org. There are 16 billion tokens. All entities and relations trained by fastText will be initialized by corresponding word embeddings in fastText. All remaining words not in the pre-trained model will be randomly initialized. In this case, all entities and relations will be mapped into a vector space embedding whose dimension is 300.”)
the set of weights of the first neural network and the second neural network are refined in a second training operation that follows the first training operation and that involves a second knowledge-graph completion task using examples mined from a second knowledge graph, and (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different).
a set of entities associated with the first knowledge graph differs from a set of entities associated with the second knowledge graph, and/or a set of relations associated with the first knowledge graph differs from a set of relations associated with the second knowledge graph. (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different. See also Table 1 for information on different datasets).
Regarding claim 20, Nguyen Thi in view of Bayram and Markowitz teaches claim 19 as outlined above. Nguyen Thi further teaches:
at a start of the first training operation, the machine-trained encoder model includes a set weights that are trained with respect to a language-modeling task. (Section III. Experimental Results and Evaluation “To evaluate the performance of the proposed model, we conduct experiments with two following scenarios. The first scenario uses a pre-trained model to initialize parameters and embeddings, then fine-tune them.”)
Regarding claim 21, Nguyen Thi in view of Bayram and Markowitz teaches claim 1 as outlined above. Nguyen Thi further teaches:
the source-entity text includes a name of the source entity and other words, and wherein the neighbor-entity text includes a name of the neighbor entity and other words. (Page 2 left column “Open knowledge base completion (OKBC) is to predict unseen facts even when real-world entities and relations may appear under several different names.”)
Regarding claim 22, Nguyen Thi in view of Bayram and Markowitz teaches claim 21 as outlined above. Markowitz further teaches:
the input information produced by the input encoder associated with the source entity encoder includes a sequence of embeddings associated with tokens in the source-entity text, and wherein the input information produced by the input encoder associated with the neighbor entity encoder includes a sequence of embeddings associated with tokens in the neighbor- entity text. (Page 5 section 4.2.5 “We combine the MPNN and the language model in a sequential manner, in which the output of the language model is used to replace the features for the entities being queried (query entities and target entities). These are the same entities whose representations will eventually be used by the decoder”)
Regarding claim 23, Nguyen Thi in view of Bayram and Markowitz teaches claim 1 as outlined above. Markowitz further teaches:
the sequence of embeddings in the input information produced by the input encoder of the second neural network includes: the source-entity encoded information produced by the source-entity encoder; the predicate encoded information; and a sum of the neighbor-entity encoded information and neighbor-relation encoded information that represents an encoded version of neighbor relation between the neighbor entity and the source entity. (Page 5 section 4.2.5 “We combine the MPNN and the language model in a sequential manner, in which the output of the language model is used to replace the features for the entities being queried (query entities and target entities). These are the same entities whose representations will eventually be used by the decoder” where Nguyen Thi teaches the source-entity encoded information and Bayram teaches the neighborhood encoded information)
Regarding claim 24, Nguyen Thi in view of Bayram and Markowitz teaches claim 23 as outlined above. Nguyen Thi further teaches:
input encoder of the second neural network also adds segment information to each token that is produced. (Page 3. Start of left column “We construct the encoder model using a combination of GRU and FC layers to learn the representation of a triple embedding. FC layers in a neural network are layers where all inputs from one layer are connected to every activation unit of the next layer. When an input vector embedding (e_h, e_r, e_t) is crossed GRU, some essential features will be extracted. After that, these features are filtered again by FC layers, and an output of this model is the triple vector embedding (v_h, v_r, v_t) see Figure 4 (b).”
Regarding claim 25, Nguyen Thi in view of Bayram and Markowitz teaches claim 14 as outlined above. Nguyen Thi further teaches:
the source-entity text includes a name of the source entity and other words, and wherein the neighbor-entity text includes a name of the neighbor entity and other words. (Page 2 left column “Open knowledge base completion (OKBC) is to predict unseen facts even when real-world entities and relations may appear under several different names.”)
Regarding claim 26, Nguyen Thi in view of Bayram and Markowitz teaches claim 14 as outlined above. Markowitz further teaches:
the sequence of embeddings in the input information produced by the input encoder of the second neural network includes: the source-entity encoded information produced by the source-entity encoder; the predicate encoded information; and a sum of the neighbor-entity encoded information and neighbor-relation encoded information that represents an encoded version of neighbor relation between the neighbor entity and the source entity. (Page 5 section 4.2.5 “We combine the MPNN and the language model in a sequential manner, in which the output of the language model is used to replace the features for the entities being queried (query entities and target entities). These are the same entities whose representations will eventually be used by the decoder” where Nguyen Thi teaches the source-entity encoded information and Bayram teaches the neighborhood encoded information)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL P GRUSZKA whose telephone number is (571)272-5259. The examiner can normally be reached M-F 9:00 AM - 6:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL GRUSZKA/Examiner, Art Unit 2121
/Li B. Zhen/Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights