Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 14-17 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter, software per se.
Claim 14 recites a “computing system” which comprises “a store” and “a processing system”. Given their broadest reasonable interpretations, “a store” and “a processing system” are not limited to hardware. The claim does not recite any tangible components such as a processor, memory, or computer-readable medium, and instead merely describes an arrangement of functional elements at an abstract, conceptual level. Thus, the claim amounts to a description of software per se.
Claims 15-17 are rejected for the same reasons as claim 14 because they do not add any limitations that would overcome the rejection.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
101 Subject Matter Eligibility Analysis
Step 1: Claims 1-13, 18-20 are within the four statutory (a process, machine, manufacture or composition of matter.) Claims 1-13 describe a process and 18-20 describes a machine. Claims 14-17 are directed to software per se as mentioned above.
With respect to claim 1:
Step 2A Prong 1: The claim recites an abstract idea enumerated in the 2019 PEG
identifying a source entity having a source-target relation that connects the source entity to a yet-to-be-determined target entity; (This is an abstract idea of a "Mental Process." The "identifying" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The identification could be made manually by an individual.)
identifying a source-entity data item that provides a passage of source-entity text pertaining to the source entity; (This is an abstract idea of a "Mental Process." The "identifying" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The identification could be made manually by an individual.)
Step 2A Prong 2: The judicial exception is not integrated into a practical application
Additional elements:
mapping, using a machine-trained encoder model, a language-based representation of the source-entity data item to source-entity encoded information; and (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
predicting an identity of the target entity based on the source-entity encoded information, and based on predicate encoded information that encodes the source-target relation. (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional elements are recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 1 is ineligible.
With respect to claim 2:
Step 2A Prong 1: claim 2, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
at least the target entity is not yet represented by the knowledge graph (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
adding a node associated with the target entity to the knowledge graph. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional elements add insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 2 is ineligible.
With respect to claim 3:
Step 2A Prong 1: claim 3, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the target entity is represented by the knowledge graph (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
performed in a course of training the machine-trained encoder model. (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element “the target entity…” adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
The additional element “performed…” is recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 3 is ineligible.
With respect to claim 4:
Step 2A Prong 1: claim 4, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the machine-trained encoder model is trained in a training operation (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
at a start of the training operation, the machine-trained encoder model includes a set weights that are trained with respect to a language-modeling task. (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional elements are recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 4 is ineligible.
With respect to claim 5:
Step 2A Prong 1: claim 5, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the knowledge graph is a first knowledge graph (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
the machine-trained encoder model is trained in a first training operation using the first knowledge graph, and wherein at a start of the first training operation, the machine-trained encoder model includes a set weights that are trained with respect to a second training operation that precedes the first training operation and which uses a second knowledge graph, the second knowledge graph being different than the first knowledge graph. (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element “the knowledge graph is a first knowledge graph” adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
The additional element “the machine-trained encoder model…” is recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 5 is ineligible.
With respect to claim 6:
Step 2A Prong 1: claim 6, which incorporates the rejection of claim 5, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
a set of entities associated with the first knowledge graph differs from a set of entities associated with the second knowledge graph (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
a set of relations associated with the first knowledge graph differs from a set of relations associated with the second knowledge graph. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional elements add insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 6 is ineligible.
With respect to claim 7:
Step 2A Prong 1: claim 7, which incorporates the rejection of claim 5, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
at a start of the second training operation, the machine-trained encoder model includes a set weights that are trained with respect to a language-modeling task. (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element is recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 7 is ineligible.
With respect to claim 8:
Step 2A Prong 1: claim 8, which incorporates the rejection of claim 1, recites an additional abstract idea:
identifying a neighbor entity that is a neighbor to the source entity, and is connected to the source entity via a neighbor relation; and (This is an abstract idea of a "Mental Process." The "identifying" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The identification could be made manually by an individual.)
identifying a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to the neighbor entity. (This is an abstract idea of a "Mental Process." The "identifying" step under its broadest reasonable interpretation, covers concepts that can be practically performed in the human mind. The identification could be made manually by an individual.)
Step 2A Prong 2: claim 8 does not recite any additional elements and thus cannot be integrated into a practical application.
Step 2B: claim 8 does not recite an additional element.
Therefore, claim 8 is ineligible.
With respect to claim 9:
Step 2A Prong 1: claim 9, which incorporates the rejection of claim 8, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
in a first-stage mapping, in addition to producing the source-entity encoded information, using the machine-trained encoder model to map a language-based representation of the neighbor-entity data item to neighbor-entity encoded information; (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
and in a second-stage mapping, mapping the source-entity encoded information, the neighbor-entity encoded information, and the predicate encoded information to neighbor-aware source-entity information, wherein the predicting includes predicting the identity of the target entity based on the neighbor-aware source-entity information. (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional elements are recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 9 is ineligible.
With respect to claim 10:
Step 2A Prong 1: claim 10, which incorporates the rejection of claim 9, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the second-stage mapping also operates on neighbor-relation encoded information, the neighbor-relation encoded information being produced by encoding the neighbor relation. (this limitation amounts to adding insignificant extra-solution activity to the judicial exception).
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element adds insignificant extra-solution activity to the judicial exception and cannot provide an inventive concept. Storing and retrieving information in memory is directed to a well understood routine conventional activity of data transmission (MPEP 2106.05(d)(II)(iv)).
Therefore, claim 10 is ineligible.
With respect to claim 11:
Step 2A Prong 1: claim 11, which incorporates the rejection of claim 9, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the first-stage mapping involves mapping plural neighbor-entity data items to plural instances of neighbor-entity encoded information, and wherein the second-stage mapping uses the plural instances of neighbor-entity encoded information to produce the neighbor-aware source-entity information. (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element is recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 11 is ineligible.
With respect to claim 12:
Step 2A Prong 1: claim 12, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the machine-trained encoder model uses attention-based logic that interprets input information fed to the attention-based logic by considering relations among different parts of the input information. (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element is recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 12 is ineligible.
With respect to claim 13:
Step 2A Prong 1: claim 13, which incorporates the rejection of claim 1, does not recite an abstract idea.
Step 2A Prong 2: The judicial exception is not integrated into a practical application.
the machine-trained encoder model is a transformer-based neural network. (This amounts to no more than mere instructions to “apply” the exception using a generic computer component.)
Step 2B: the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception
The additional element is recited in a generic level and they represent generic computer components to apply the abstract idea. Mere instructions to apply an exception cannot provide an inventive concept (MPEP 2106.05(f)).
Therefore, claim 13 is ineligible.
With respect to claim 14:
The claim recites similar limitations as corresponding to claim 1. Therefore, the same subject matter analysis that was utilized for claim 1, as described above, is equally applicable to claim 14. Therefore, claim 14 is ineligible.
With respect to claim 15:
The claim recites similar limitations as corresponding to claim 5 & 6. Therefore, the same subject matter analysis that was utilized for claim 5 & 6, as described above, is equally applicable to claim 15. Therefore, claim 15 is ineligible.
With respect to claim 16:
The claim recites similar limitations as corresponding to claim 7. Therefore, the same subject matter analysis that was utilized for claim 7, as described above, is equally applicable to claim 16. Therefore, claim 16 is ineligible.
With respect to claim 17:
The claim recites similar limitations as corresponding to claim 8, 9, 10 & 11. Therefore, the same subject matter analysis that was utilized for claim 8, 9, 10 & 11, as described above, is equally applicable to claim 17. Therefore, claim 17 is ineligible.
With respect to claim 18:
The claim recites similar limitations as corresponding to claim 1, 8, 9 & 10. Therefore, the same subject matter analysis that was utilized for claim 1, 8, 9 & 10, as described above, is equally applicable to claim 18. Therefore, claim 18 is ineligible.
With respect to claim 19:
The claim recites similar limitations as corresponding to claim 5 & 6. Therefore, the same subject matter analysis that was utilized for claim 5 & 6, as described above, is equally applicable to claim 19. Therefore, claim 19 is ineligible.
With respect to claim 20:
The claim recites similar limitations as corresponding to claim 7. Therefore, the same subject matter analysis that was utilized for claim 7, as described above, is equally applicable to claim 20. Therefore, claim 20 is ineligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-7, 12-16 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Nguyen Thi (NPL: ‘Knowledge Base Completion with transfer learning using BERT and fastText’ accessed through applicant’s information disclose statement (IDS)).
Regarding claim 1, Nguyen Thi Teaches:
A computer-implemented method for supplementing a knowledge graph, comprising: (Abstract)
identifying a source entity having a source-target relation that connects the source entity to a yet-to-be-determined target entity; (Figure. 3 triple Section II. KBC With Transfer Learning “An input of the word embedding model is a triple (h, r, t)”).
identifying a source-entity data item that provides a passage of source-entity text pertaining to the source entity; (Figure. 3 word embedding model Section II. KBC With Transfer Learning “the first part is a word embedding model that is used to map from a word space to a vector space”)
mapping, using a machine-trained encoder model, a language-based representation of the source-entity data item to source-entity encoded information; and (Figure 3 Encoder. Section II. KBC With Transfer Learning “We construct the encoder model using a combination of GRU and FC layers to learn the representation of a triple embedding. FC layers in a neural network are layers where all inputs from one layer are connected to every activation unit of the next layer. When an input vector embedding (e_h,e_r,e_t) is crossed GRU, some essential features will be extracted. After that, these features are filtered again by FC layers, and an output of this model is the triple vector embedding (v_h, v_r, v_t)”).
predicting an identity of the target entity based on the source-entity encoded information, and based on predicate encoded information that encodes the source-target relation. (Figure. 3 KBC Model. Section II. KBC With Transfer Learning “the last part is a KBC model whose input is a triple embedding, and then its score is predicted to determine whether it is a correct triple or not”)
Regarding claim 2, Nguyen Thi Teaches:
at least the target entity is not yet represented by the knowledge graph, and wherein the computer- implemented method further includes adding a node associated with the target entity to the knowledge graph. (Section II. KBV With Transfer learning “Then the embeddings are the input of a KBC algorithm to predict their score using its loss function, and the input triple is missing if the result score is positive”).
Regarding claim 3, Nguyen Thi Teaches:
the target entity is represented by the knowledge graph, and wherein the computer-implemented method is performed in a course of training the machine-trained encoder model. (Section III. Experimental Results and Evaluation they describe using complete datasets for training)
Regarding claim 4, Nguyen Thi Teaches:
the machine-trained encoder model is trained in a training operation, and wherein at a start of the training operation, the machine-trained encoder model includes a set weights that are trained with respect to a language-modeling task. (Section III. Experimental Results and Evaluation “To evaluate the performance of the proposed model, we conduct experiments with two following scenarios. The first scenario uses a pre-trained model to initialize parameters and embeddings”)
Regarding claim 5, Nguyen Thi Teaches:
the knowledge graph is a first knowledge graph, wherein the machine-trained encoder model is trained in a first training operation using the first knowledge graph, and wherein at a start of the first training operation, the machine-trained encoder model includes a set weights that are trained with respect to a second training operation that precedes the first training operation, and which uses a second knowledge graph, the second knowledge graph being different than the first knowledge graph. (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different).
Regarding claim 6, Nguyen Thi Teaches:
a set of entities associated with the first knowledge graph differs from a set of entities associated with the second knowledge graph, and/or wherein a set of relations associated with the first knowledge graph differs from a set of relations associated with the second knowledge graph. (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different. See also Table 1 for information on different datasets).
Regarding claim 7, Nguyen Thi Teaches:
at a start of the second training operation, the machine-trained encoder model includes a set weights that are trained with respect to a language-modeling task. (Section III. Experimental Results and Evaluation “To evaluate the performance of the proposed model, we conduct experiments with two following scenarios. The first scenario uses a pre-trained model to initialize parameters and embeddings, then fine-tune them.”)
Regarding claim 12, Nguyen Thi Teaches:
the machine-trained encoder model uses attention-based logic that interprets input information fed to the attention-based logic by considering relations among different parts of the input information. (Section II. KBC With Transfer Learning subsection B. The proposed KBC Model describes the BERT model used which is an attention based model.)
Regarding claim 13, Nguyen Thi Teaches:
the machine-trained encoder model is a transformer-based neural network. (Section II. KBC With Transfer Learning subsection B. The proposed KBC Model describes the BERT model used which is a transformer based model.)
Regarding claim 14, Nguyen Thi Teaches:
A computing system for providing content, comprising: a store for storing computer-readable instructions; a store for storing a knowledge graph; a processing system for executing the computer-readable instructions to perform operations that include: (They run experiments and simulations of their model implies a computing system is needed)
identifying a source entity having a source-target relation that connects the source entity to a yet-to-be-determined target entity via a source-target relation; (Figure. 3 triple Section II. KBC With Transfer Learning “An input of the word embedding model is a triple (h, r, t)”).
identifying a source-entity data item that provides a passage of source-entity text pertaining to the source entity; (Figure. 3 word embedding model Section II. KBC With Transfer Learning “the first part is a word embedding model that is used to map from a word space to a vector space”)
mapping, using a machine-trained encoder model, a language-based representation of the source-entity data item to source-entity encoded information; and (Figure 3 Encoder. Section II. KBC With Transfer Learning “We construct the encoder model using a combination of GRU and FC layers to learn the representation of a triple embedding. FC layers in a neural network are layers where all inputs from one layer are connected to every activation unit of the next layer. When an input vector embedding (e_h,e_r,e_t) is crossed GRU, some essential features will be extracted. After that, these features are filtered again by FC layers, and an output of this model is the triple vector embedding (v_h, v_r, v_t)”).
predicting an identity of the target entity based on the source-entity encoded information, and based on predicate encoded information that encodes the source-target relation. (Figure. 3 KBC Model. Section II. KBC With Transfer Learning “the last part is a KBC model whose input is a triple embedding, and then its score is predicted to determine whether it is a correct triple or not”)
Regarding claim 15, Nguyen Thi Teaches:
the knowledge graph is a first knowledge graph, wherein the machine-trained encoder model is trained in a first training operation using the first knowledge graph, wherein at a start of the first training operation, the machine-trained encoder model includes a set weights that are trained with respect to a second training operation that precedes the first training operation, and which uses a second knowledge graph, and (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different).
wherein a set of entities associated with the first knowledge graph differs from a set of entities associated with the second knowledge graph, and/or wherein a set of relations associated with the first knowledge graph differs from a set of relations associated with the second knowledge graph. (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different. See also Table 1 for information on different datasets).
Regarding claim 16, Nguyen Thi Teaches:
at a start of the second training operation, the machine-trained encoder model includes a set weights that are trained with respect to a language-modeling task. (Section III. Experimental Results and Evaluation “To evaluate the performance of the proposed model, we conduct experiments with two following scenarios. The first scenario uses a pre-trained model to initialize parameters and embeddings, then fine-tune them.”)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 8-11 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over Nguyen Thi
in view of Bayram (US 20240256917 A1).
Regarding claim 8, Nguyen Thi teaches claim 1 as outlined above. Nguyen Thi does not teach:
identifying a neighbor entity that is a neighbor to the source entity, and is connected to the source entity via a neighbor relation; and
identifying a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to the neighbor entity.
However Bayram does:
identifying a neighbor entity that is a neighbor to the source entity, and is connected to the source entity via a neighbor relation; and ([0067] “For each identified target node, the subgraph generator 302 is configured identify a list of k-hop neighboring nodes of the target node, e.g., using the knowledge graph triples 218.”).
identifying a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to the neighbor entity. ([0068] “The node embedding initializer 304 is configured to receive the neighboring node set of the k-hop subgraph 308, as well as the set of entity types 224 for the knowledge graph and the ontology embeddings 222.”)
Nguyen Thi and Bayram are considered analogous art to the claimed invention because they are in the same field of endeavor being knowledge graph completion. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the system and model of Nguyen Thi with the method of implementing neighboring nodes of Bayram. One would want to do this for a more effective knowledge graph completion.
Regarding claim 9, Nguyen Thi in view of Bayram teaches claim 8 as outlined above. Bayram further teaches:
in a first-stage mapping, in addition to producing the source-entity encoded information, using the machine-trained encoder model to map a language-based representation of the neighbor-entity data item to neighbor-entity encoded information; ([0069]-[0070] describes the subgraph encoder which takes the neighborhood embeddings and produces and encoding. “The subgraph encoder 214 is also configured to receive triples for the k-hop subgraph 312 from the subgraph generator 302, as well as the entity types 224 and the ontology embeddings 222. The subgraph encoder 214 is configured to process these inputs using multiple relational message-passing layers 316 and a relational attention mechanism to generate, as output, an embedding of the target node”)
and in a second-stage mapping, mapping the source-entity encoded information, the neighbor-entity encoded information, and the predicate encoded information to neighbor-aware source-entity information, wherein the predicting includes predicting the identity of the target entity based on the neighbor-aware source-entity information. ([0100] “For example, the knowledge completion task can be the task of predicting a link between a node that was observed in the triples used to train the ontology-driven neural link prediction system and another node that was not observed in the triples used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a new knowledge graph triple, where the new knowledge graph triple includes a subject node or an object node that was not included in the set of triples. As another example, the knowledge completion task can be the task of predicting links from a new knowledge graph that is different to the knowledge graph used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a triple from a new knowledge graph, where the new knowledge graph has a same ontology (e.g., entity types and relation types) as the knowledge graph represented by the set of triples.”)
Regarding claim 10, Nguyen Thi in view of Bayram teaches claim 9 as outlined above. Bayram further teaches:
the second-stage mapping also operates on neighbor-relation encoded information, the neighbor-relation encoded information being produced by encoding the neighbor relation. ([0037] “The ontology-driven neural link prediction system includes an embedding generation layer that generates target knowledge graph node embeddings using an ontology lookup layer that stores learned ontology embeddings, e.g., entity and relation type embeddings, and a subgraph encoder that operates on a k-hop neighboring subgraph that encloses the target knowledge graph nodes.”)
Regarding claim 11, Nguyen Thi in view of Bayram teaches claim 9 as outlined above. Bayram further teaches:
the first-stage mapping involves mapping plural neighbor-entity data items to plural instances of neighbor-entity encoded information, and wherein the second-stage mapping uses the plural instances of neighbor-entity encoded information to produce the neighbor-aware source-entity information. ([0068] “[0068] The subgraph generator 302 is configured to provide the neighboring node set of the k-hop subgraph 308 to the node embedding initializer 304. The node embedding initializer 304 is configured to receive the neighboring node set of the k-hop subgraph 308, as well as the set of entity types 224 for the knowledge graph and the ontology embeddings 222.”)
Regarding claim 17, Nguyen Thi teaches claim 14 as outlined above. Nguyen Thi does not teach any of the limitations of claim 17. However, Bayram does:
identifying a neighbor entity that is a neighbor to the source entity, and is connected to the source entity via a neighbor relation, wherein neighbor-relation encoded information encodes the neighbor relation; and ([0067] “For each identified target node, the subgraph generator 302 is configured identify a list of k-hop neighboring nodes of the target node, e.g., using the knowledge graph triples 218.”).
identifying a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to the neighbor entity, wherein the mapping includes: ([0068] “The node embedding initializer 304 is configured to receive the neighboring node set of the k-hop subgraph 308, as well as the set of entity types 224 for the knowledge graph and the ontology embeddings 222.”)
in first-stage mapping, in addition to producing the source-entity encoded information, using to the machine-trained encoder model to map a language-based representation of the neighbor-entity data item to neighbor-entity encoded information; ([0069]-[0070] describes the subgraph encoder which takes the neighborhood embeddings and produces and encoding. “The subgraph encoder 214 is also configured to receive triples for the k-hop subgraph 312 from the subgraph generator 302, as well as the entity types 224 and the ontology embeddings 222. The subgraph encoder 214 is configured to process these inputs using multiple relational message-passing layers 316 and a relational attention mechanism to generate, as output, an embedding of the target node”)
and in second-stage mapping, mapping the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information to neighbor-aware source-entity information, wherein the predicting includes predicting the identity of the target entity based on the neighbor-aware source-entity information. ([0100] “For example, the knowledge completion task can be the task of predicting a link between a node that was observed in the triples used to train the ontology-driven neural link prediction system and another node that was not observed in the triples used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a new knowledge graph triple, where the new knowledge graph triple includes a subject node or an object node that was not included in the set of triples. As another example, the knowledge completion task can be the task of predicting links from a new knowledge graph that is different to the knowledge graph used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a triple from a new knowledge graph, where the new knowledge graph has a same ontology (e.g., entity types and relation types) as the knowledge graph represented by the set of triples.”)
Nguyen Thi and Bayram are considered analogous art to the claimed invention because they are in the same field of endeavor being knowledge graph completion. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the system and model of Nguyen Thi with the method of implementing neighboring nodes of Bayram. One would want to do this for a more effective knowledge graph completion.
Regarding claim 18, Nguyen Thi teaches:
identifying a source entity that is connected to a yet-to-determined target entity via a source-target relation, wherein predicate encoded information encodes the source- target relation; (Figure. 3 triple Section II. KBC With Transfer Learning “An input of the word embedding model is a triple (h, r, t)”).
identifying a source data item that provides a passage of source-entity text pertaining to the source entity; (Figure. 3 word embedding model Section II. KBC With Transfer Learning “the first part is a word embedding model that is used to map from a word space to a vector space”)
Bayram teaches:
identifying a neighbor entity that is a neighbor to the source entity, and is connected to the source entity via a neighbor relation, wherein neighbor-relation encoded information encodes the neighbor relation; ([0067] “For each identified target node, the subgraph generator 302 is configured identify a list of k-hop neighboring nodes of the target node, e.g., using the knowledge graph triples 218.”).
identifying a neighbor-entity data item that provides a passage of neighbor-entity text pertaining to the neighbor entity, ([0068] “The node embedding initializer 304 is configured to receive the neighboring node set of the k-hop subgraph 308, as well as the set of entity types 224 for the knowledge graph and the ontology embeddings 222.”)
in a first-stage mapping, mapping using a machine-trained encoder model, a language-based representation of the source-entity data item to source-entity encoded information, and mapping a language-based representation of the neighbor-entity data- item to neighbor-entity encoded information, each language-based representation being formed using a vocabulary of tokens of a natural language; ([0069]-[0070] describes the subgraph encoder which takes the neighborhood embeddings and produces and encoding. “The subgraph encoder 214 is also configured to receive triples for the k-hop subgraph 312 from the subgraph generator 302, as well as the entity types 224 and the ontology embeddings 222. The subgraph encoder 214 is configured to process these inputs using multiple relational message-passing layers 316 and a relational attention mechanism to generate, as output, an embedding of the target node”)
in second-stage mapping, mapping the source-entity encoded information, the neighbor-entity encoded information, the neighbor-relation encoded information, and the predicate encoded information to neighbor-aware source-entity information; and predicting an identity of the target entity based on the neighbor-aware source- entity information. ([0100] “For example, the knowledge completion task can be the task of predicting a link between a node that was observed in the triples used to train the ontology-driven neural link prediction system and another node that was not observed in the triples used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a new knowledge graph triple, where the new knowledge graph triple includes a subject node or an object node that was not included in the set of triples. As another example, the knowledge completion task can be the task of predicting links from a new knowledge graph that is different to the knowledge graph used to train the ontology-driven neural link prediction system. In these implementations the system can receive a request to score a triple from a new knowledge graph, where the new knowledge graph has a same ontology (e.g., entity types and relation types) as the knowledge graph represented by the set of triples.”)
Nguyen Thi and Bayram are considered analogous art to the claimed invention because they are in the same field of endeavor being knowledge graph completion. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the system and model of Nguyen Thi with the method of implementing neighboring nodes of Bayram. One would want to do this for a more effective knowledge graph completion.
Regarding claim 19, Nguyen Thi in view of Bayram teaches claim 18 as outlined above. Nguyen Thi further teaches:
the knowledge graph is a first knowledge graph, wherein the machine-trained encoder model is trained in a first training operation using the first knowledge graph, wherein at a start of the first training operation, the machine-trained encoder model includes a set weights that are trained with respect to a second training operation that precedes the first training operation, and which uses a second knowledge graph, and (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different).
wherein a set of entities associated with the first knowledge graph differs from a set of entities associated with the second knowledge graph, and/or a set of relations associated with the first knowledge graph differs from a set of relations associated with the second knowledge graph. (Section III. Experimental Results and Evaluation they describe a pre-training and a fine-tune training. For these separate trainings they use different datasets implying the knowledge graphs are different. See also Table 1 for information on different datasets).
Regarding claim 20, Nguyen Thi in view of Bayram teaches claim 1 as outlined above. Nguyen Thi further teaches:
at a start of the second training operation, the machine-trained encoder model includes a set weights that are trained with respect to a language-modeling task. (Section III. Experimental Results and Evaluation “To evaluate the performance of the proposed model, we conduct experiments with two following scenarios. The first scenario uses a pre-trained model to initialize parameters and embeddings, then fine-tune them.”)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DANIEL PATRICK GRUSZKA whose telephone number is (571)272-5259. The examiner can normally be reached M-F 9:00 AM - 6:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li Zhen can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DANIEL GRUSZKA/ Examiner, Art Unit 2121
/Li B. Zhen/ Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights