Detailed Action
This action is in response to the RCE filed on 12/08/2025 for the amended claims filed 11/13/2025 for application 17/563,411, in which:
Claims 1, 10, and 16 are independent claims.
Claims 1, 10, and 16 are amended.
Claims 2, 11 and 17 are cancelled.
Claims 1, 3-10, 12-16 and 18-20 are currently pending.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 12/08/2025 has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. CN201910583845.0, filed on 06/29/2019.
Response to Arguments
Applicant's arguments filed 11/13/2025 have been fully considered but they are not persuasive.
Regarding the 35 USC § 101 Rejections:
Applicant's arguments regarding the 35 U.S.C. 101 rejections of the previous office action have been fully considered, but are unpersuasive.
Applicant's disagrees (Page 10) with the previous office action that the claims recite a process that can be performed in the human mind and does not recite additional elements that integrate the alleged judicial exception into a practical application. However, the independent claims have been amended. The claims recite computer-implemented operations that are tied to computing hardware and that cannot be performed in the human mind, and provide a technological improvement to computer-based knowledge-graph processing.
Examiner respectfully disagrees. For the reasons given below and in the 35 U.S.C. § 101 rejections, the claims are directed to an abstract idea (Step 2A Prong 1) and do not integrate the abstract idea into a practical application (Step 2A Prong 2). The claims recites obtaining M entities in a target knowledge graph, wherein the M entities comprise an entity 1, an entity 2, through an entity M, and M is an integer greater than 1 (a human being can mentally apply evaluation to obtain entities within a knowledge graph), obtaining, from a preset knowledge base, N related entities of an entity m in the M entities and K concepts corresponding to a related entity n in the N related entities, wherein the N related entities comprise a related entity 1, a related entity 2, through a related entity N, N and K are integers not less than 1, m = 1, 2, 3, through M, n = 1, 2, 3, through N, the entity m is semantically correlated with the N related entities, and the related entity n is semantically correlated with the K concepts … (a human being can mentally apply evaluation to obtain related entities and concepts of an entity wherein the entity is semantically correlated with the related entities and the concepts), determining a semantic correlation between each of the M entities and each of the N related entities of the entity m, and determining a first entity embedding representation of each of the N related entities based on corresponding K concepts … (a human being can mentally apply evaluation to determine a semantic correlation between entities and related entities; and to determine an embedding representation of an entity based on corresponding concepts), … wherein determining the first entity embedding representation comprises performing vectorization processing on each concept in the K concepts corresponding to the related entity n … (a human being can mentally apply evaluation to determine a embedding representation by performing vectorization on specific concepts in specific entities with the aid of pen and paper), … performing average summation on the word vectors of the K concepts to obtain the first entity embedding representation of the related entity n (a human being can mentally apply evaluation to determine a embedding representation by performing average summation on specific vectors of specific concepts to obtain a specific embedding representation with the aid of pen and paper), modeling, based on the first entity embedding representation and the semantic correlation, an embedding representation of the M entities and an embedding representation of an entity relationship between the M entities, to obtain an embedding representation model (a human being can mentally apply evaluation to model embedding representations to obtain an embedding representation model); where the abstract ideas are evaluations or judgements that can be performed in the human mind, or by a human using pen and paper. The additional elements recited within the independent claim only recites performance of an abstract idea within a computer or restricting the abstract idea to a particular technological environment; thus, as the additional elements fall within MPEP 2106.05 they are unable to integrate the judicial exception as they are unable to provide significantly more. The limitations are unable to provide improvement as they are currently being evaluated as either abstract idea(s) or additional elements that fall within MPEP 2106.05. Therefore, for the reasons given above and in the rejections below, the rejection to all Claims (including Claim 1, similar independent claims, and all dependent Claims) are maintained. More specific details are discussed below within the 35 USC § 101 Rejections.
Applicant asserts (Page 11), the newly added limitations of automatically identifying K concepts, performing vectorization using a word-vector generation model, and previously presented limitation of training an embedding representation model requires intensive machine computation and cannot be practically executed through mental steps. The present claims fall outside the scope of the "mental process" grouping described in the guidance. The claimed computer-implemented operations, noted above, require extensive numerical computation that cannot reasonably be performed mentally; thus, the claims are directed to a computer-implemented process and not an abstract idea.
Examiner respectfully disagrees. The automatically identifying K concepts is being evaluated as an additional element (more information below) where the limitation is restricting the abstract idea; the limitation merely mentions what K concepts comprise of for automatic identification and their representations of specific information. The performing vectorization using a word-vector generation model limitation is being evaluated as two different limitations; where the performing of vectorization is an abstract idea (a human being can mentally apply evaluation to determine a embedding representation by performing vectorization on specific concepts in specific entities with the aid of pen and paper) and using a word-vector generation model limitation is an additional elements as it is merely applying the abstract idea on a computer. The previously presented limitation of training an embedding representation model is merely performing a mental process on a computer; which is no more than instructions to “apply it” on a computer. The extensiveness of numerical computations or amount of computations do not dictate judicial exception from being a mental process as merely invoking computers or machinery as a tool to perform abstract idea are mere instructions to apply it. The claims are directed towards the improvement of an abstract idea. Improvements to an abstract idea are still considered to an abstract idea. The independent claim fails to recite the steps that achieve the improvement.
Applicant asserts (Page 11-12), for sake of argument, even if the claims recite mental processes… the additional elements of each of the claims integrate the alleged abstract ideas into a practical application. Applicant further supports their assertions by noting the specification for the amended claims where the K concepts implement semantic extension of an entity to improve a representation and accuracy/comprehensiveness. Applicant also notes that performing vectorization and average summation, for determining embedding representations, includes the results in embedding capturing contextual and relational semantics cannot be derived from text co-occurrence alone; thus, enhancing the system’s ability for representations. Moreover, performing average summation on multiple concept vectors reduces the computational noise and produces a composite embedding for a representation; thus, providing a technical improvement. The additional elements of the claims, including automatic semantic-descriptor identification, computer-implemented vectorization, and hardware-executed embedding training, integrate any alleged abstract idea into a practical application. The claimed process is not well-understood, routine, or conventional. Rather, it provides a technical improvement due to transforming texts and inputs into embedding representations; which increase computer performance and provides significantly more than the alleged abstract idea by enhancing computer functionality itself (i.e. improving how the computing device acquires, models, and represents complex relational data with higher precision and lower computational cost). Therefore, claims 1, 3-10, 12-16 and 18-20 are patent eligible under 35 U.S.C. § 101. Withdrawal of the rejection under 35 U.S.C. 101 is requested.
Examiner respectfully disagrees. Please note that the automatically identifying K concepts, performing vectorization, and performing average summation limitations are being evaluated under Step 2A Prong 1 as abstract ideas (noted above); where, for example, using a word-vector generation model limitation is an additional element, as it is merely applying the abstract idea (performing of vectorization) on a computer to obtain a vector. Limitations within Step 2A Prong 2 are unable to integrate the abstract idea(s) into a practical application as the additional elements do not amount to significantly more than the judicial exception as the additional elements are merely performing the abstract idea(s) where the process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, restricting the abstract idea to a Particular Technological Environment, or insignificant extra-solution activity of data gathering. Performance “on a computer” or “with a machine learning model” or “using a specialized machine” does not prevent a limitation from being practically performable. Additionally, the Claims do not reflect any improvement in the functioning of a computer or hardware processor rather the additional elements merely use a generic computer component to perform the abstract idea and/or restrict the abstract idea to a specific technological environment. Therefore, the claims do not integrate the judicial exception into a practical application nor amount to significantly more. The claim is not patent eligible. Although the Claims are interpreted in light of the specification, limitations from the specification are not read into the Claims.
MPEP 2106.05(a) recites:
After the examiner has consulted the specification and determined that the disclosed invention improves technology, the claim must be evaluated to ensure the claim itself reflects the disclosed improvement in technology … the claim must include the components or steps of the invention that provide the improvement described in the specification
…
It is important to note, the judicial exception alone cannot provide the improvement. The improvement can be provided by one or more additional elements. See the discussion of Diamond v. Diehr, 450 U.S. 175, 187 and 191-92, 209 USPQ 1, 10 (1981)) in subsection II, below.
Applicant fails to show how any alleged technical improvement would be provided by anything more than the judicial exception on its own. Additionally, applicant fails to show how the claim includes components or steps that would provide the alleged improvement described in the specification. The independent Claim fails to recite any details as to how implementing the training or using the training improves the method for performing the abstract ideas. By MPEP 2106.05(f)(1), "the claim recites only the idea of a solution or outcome, i.e. the claim fails to recite details of how a solution to a problem is accomplished". The independent Claim is no more detailed than merely obtaining a knowledge graph with specific details, implementing training based on specific requirements, and obtaining relationship representations for determining + modeling semantic correlations. Moreover, the examiner maintains that the Claim does not impose any meaningful limits on the judicial exception. As noted in the rejection, the Claim does not include additional elements that are sufficient to amount to an integration of the identified abstract idea into a practical application, thus the claim is directed to an abstract idea. Therefore, for the reasons given above and in the updated rejections below, the rejection to all Claims (including Claim 1 and all dependent Claims) are maintained and updated as necessitated by Claim amendments. More specific details are discussed below within the 35 USC § 101 Rejections.
Regarding the 35 USC § 102 Rejections:
Applicant's arguments regarding the 35 U.S.C. 102 rejections of the previous office action have been fully considered, but are unpersuasive.
Applicant traverses the 102 rejections as the Applicant asserts (Pages 13-14) that Lin fails to disclose the amended independent claim and specifically the automatic identification of textual or semantic descriptors ("K concepts" which represent semantic information that characterizes attributes, categories, or contextual features) from a page linked to a related entity and the generation of a first entity embedding representation based on those K concepts by performing vectorization processing on each concept to obtain a word vector of each concept and performing average summation on the word vectors of the K concepts.
Applicant’s arguments with respect to the independent claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant further supports their traversal of the 102 rejections by asserting (Page 14) that Lin operates only on existing knowledge graphs composed of predefined triples (h, r, t) from datasets such as WordNet or Freebase. Lin does not identify any textual or semantic descriptors automatically identified from linked pages; thus, fails to disclose the particularly claimed K concepts that comprise textual or semantic descriptors automatically identified on a page linked to the related entity n, and represent semantic information that characterizes attributes, categories, or contextual features of the related entity n.
Applicant’s arguments with respect to the independent claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant asserts (Page 14), that Lin also does not disclose the claimed performing vectorization on each concept using a word-vector generation model and performing average summation on the resulting vectors to obtain a first entity embedding representation. Lin's TransR method initializes entity and relation embeddings from TransE, not from word-vector models, and applies relation-specific projection matrices, not averaging operations.
Applicant’s arguments with respect to the independent claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant asserts (Page 14), as noted above Lin at least fails to disclose the amended limitations recited by claim 1 (and similarly claims 10 and 16). Thus, claims 1, 10 and 16 and their dependent claims are not anticipated by Lin at least for this reason. Accordingly, Applicants respectfully request the rejection under 35 U.S.C. § 102 be withdrawn.
Applicant’s arguments with respect to the independent claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Applicant’s arguments regarding the other independent and dependent claims rely upon the same assertions as with respect to Claim 1, and are thus likewise unpersuasive. Therefore, for the reasons given above and in the updated rejections below, the rejection to all Claims (including Claim 1 and all dependent Claims) are maintained and updated as necessitated by Claim amendments.
Regarding the 35 USC § 103 Rejections:
Applicant's arguments regarding the 35 U.S.C. 103 rejections of the previous office action have been fully considered, but are unpersuasive.
Applicant traverses the 103 rejections as the Applicant asserts (Page 15) that Liu fails to remedy the deficiencies of Lin discussed above with respect to claims 1, 10, and 16. Liu relates to sentence-level encoding of textual data in a natural language processing context. However, Liu does not teach modeling entities and relationships within a knowledge graph, nor does it teach automatically identifying semantic descriptors from linked pages associated with entities. Thus, Liu fails to teach identifying K concepts corresponding to a related entity, determining a semantic correlation between entities, or generating a first entity embedding representation based on those K concepts. Liu's sentence averaging merely aggregates linguistic features for text understanding and does not teach performing vectorization processing on each concept in the K concepts corresponding to the related entity n by using a word-vector generation model to obtain a word vector of each concept and performing average summation on the word vectors of the K concepts to obtain the first entity embedding representation of the related entity n.
Applicant’s arguments with respect to the independent claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Applicant asserts (Page 15), Liu does not cure the deficiencies of Lin and fails to teach the amended limitations of claim 1 (and similarly claims 10 and 16). Therefore, claims 1, 10 and 16 and their dependent claims, including claims 2, 5-8, 11, 14, 15, 17 and 20, are patentable over the alleged combination of Lin and Liu. Accordingly, Applicants respectfully request that the rejection be withdrawn.
Applicant’s arguments with respect to the independent claim(s) have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Applicant’s arguments regarding the other independent and dependent claims rely upon the same assertions as with respect to Claim 1, and are thus likewise unpersuasive. Therefore, for the reasons given above and in the updated rejections below, the rejection to all Claims (including Claim 1 and all dependent Claims) are maintained and updated as necessitated by Claim amendments.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding Claim 1:
Claim 1 recites a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 1 further recites the process comprising of:
obtaining M entities in a target knowledge graph, wherein the M entities comprise an entity 1, an entity 2, through an entity M, and M is an integer greater than 1 (a human being can mentally apply evaluation to obtain entities within a knowledge graph)
obtaining, from a preset knowledge base, N related entities of an entity m in the M entities and K concepts corresponding to a related entity n in the N related entities, wherein the N related entities comprise a related entity 1, a related entity 2, through a related entity N, N and K are integers not less than 1, m = 1, 2, 3, through M, n = 1, 2, 3, through N, the entity m is semantically correlated with the N related entities, and the related entity n is semantically correlated with the K concepts … (a human being can mentally apply evaluation to obtain related entities and concepts of an entity wherein the entity is semantically correlated with the related entities and the concepts)
determining a semantic correlation between each of the M entities and each of the N related entities of the entity m, and determining a first entity embedding representation of each of the N related entities based on corresponding K concepts … (a human being can mentally apply evaluation to determine a semantic correlation between entities and related entities; and to determine an embedding representation of an entity based on corresponding concepts)
… wherein determining the first entity embedding representation comprises performing vectorization processing on each concept in the K concepts corresponding to the related entity n … (a human being can mentally apply evaluation to determine a embedding representation by performing vectorization on specific concepts in specific entities with the aid of pen and paper)
… performing average summation on the word vectors of the K concepts to obtain the first entity embedding representation of the related entity n (a human being can mentally apply evaluation to determine a embedding representation by performing average summation on specific vectors of specific concepts to obtain a specific embedding representation with the aid of pen and paper)
modeling, based on the first entity embedding representation and the semantic correlation, an embedding representation of the M entities and an embedding representation of an entity relationship between the M entities, to obtain an embedding representation model (a human being can mentally apply evaluation to model embedding representations to obtain an embedding representation model)
Claim 1 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the additional elements recited consists of:
A knowledge graph embedding representation method, performed by an electronic device including at least one processor and a memory, comprising (to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f))
wherein the K concepts comprise textual or semantic descriptors automatically identified on a page linked to the related entity n, the K concepts representing semantic information that characterizes attributes, categories, or contextual features of the related entity n (which is restricting the abstract idea to a Particular Technological Environment, by MPEP 2106.05(h))
… by using a word-vector generation model to obtain a word vector of each concept … (to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f))
training the embedding representation model to obtain a second entity embedding representation of each entity and a relationship embedding representation of the entity relationship (to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f))
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements recited, alone or in combination, do not provide significantly more than the abstract idea itself. Additional element a, and c-d are merely applying the abstract idea on a computer (MPEP 2106.05(f)) which cannot provide significantly more. Additional element b is only restricting the abstract idea to a Particular Technological Environment (MPEP 2106.05(h)) which cannot provide significantly more. Thus, the claim is subject-matter ineligible.
Regarding Claim 3:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 3 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 3 further recites:
wherein the modeling, based on the first entity embedding representation and the semantic correlation, the embedding representation of the M entities and the embedding representation of the entity relationship between the M entities, to obtain the embedding representation model comprises: (a human being can mentally apply evaluation to model embedding representations to obtain an embedding representation model)
determining, based on the semantic correlation and a first entity embedding representation of the N related entities, a unary text embedding representation corresponding to each entity (a human being can mentally apply evaluation to determine a unary text embedding representation corresponding to each entity based on semantic correlations and an embedding representation of the related entities)
determining, based on the N related entities, a common related entity of every two entities in the M entities (a human being can mentally apply evaluation to determine a common related entity between two entities)
determining, based on the semantic correlation and a first entity embedding representation of the common related entity, a binary text embedding representation corresponding to the every two entities (a human being can mentally apply evaluation to determine a binary text embedding representation corresponding to the every two entities based on semantic correlations and an embedding representation of the related entities)
establishing, based on the unary text embedding representation and the binary text embedding representation, the embedding representation model (a human being can mentally apply evaluation to establish the embedding representation model based on unary and binary text embedding representations)
Claim 3 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 4:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 4 recites the method of Claim 3. Claim 3 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 4 further recites:
wherein the establishing, based on the unary text embedding representation and the binary text embedding representation, the embedding representation model comprises: (a human being can mentally apply evaluation to establish the embedding representation model based on unary and binary text embedding representations)
mapping the unary text embedding representation and the binary text embedding representation to a same vector space, to obtain a semantically enhanced unary text embedding representation and a semantically enhanced binary text embedding representation (a human being can mentally apply evaluation to map unary and binary text embeddings to obtain semantically enhanced text embedding representations)
establishing, based on the semantically enhanced unary text embedding representation and the semantically enhanced binary text embedding representation, the embedding representation model (a human being can mentally apply evaluation to establish the embedding representation model based off the semantically enhanced text embedding representations)
Claim 4 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 5:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 5 recites the method of Claim 3. Claim 3 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 5 further recites:
wherein the determining, based on the semantic correlation and the first entity embedding representation of the N related entities, the unary text embedding representation corresponding to each entity comprises: (a human being can mentally apply evaluation to determine a unary text embedding representation corresponding to each entity based on semantic correlations and an embedding representation of the related entities)
using the semantic correlation as a first weight coefficient of each of the N related entities; and performing, based on the first weight coefficient, weighted summation on the first entity embedding representation of the N related entities, to obtain the unary text embedding representation (a human being can mentally apply evaluation to perform a weighted summation on an embedding representation to obtain a unary text embedding representation using the semantic correlation as the weight coefficient).
Claim 5 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 6:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 6 recites the method of Claim 3. Claim 3 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 6 further recites:
wherein the determining, based on the semantic correlation and a first entity embedding representation of the common related entity, the binary text embedding representation corresponding to the every two entities comprises: (a human being can mentally apply evaluation to determine a binary text embedding representation corresponding to the every two entities based on semantic correlations and an embedding representation of the related entities)
using the common related entity and a minimum semantic correlation of semantic correlations of every two entities as a second weight coefficient of the common related entity; and performing, based on the second weight coefficient, weighted summation on the first entity embedding representation of the common related entity, to obtain the binary text embedding representation (a human being can mentally apply evaluation to perform a weighted summation on an embedding representation to obtain a unary text embedding representation using the semantic correlation as the weight coefficient).
Claim 6 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because there are no new additional elements recited.
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because there are no new additional elements recited. The judicial exception alone does not provide significantly more than the abstract idea itself. Thus, the claim is subject-matter ineligible.
Regarding Claim 7:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 7 recites the method of Claim 5. Claim 5 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 7 further recites determining a loss function of the embedding representation model (a human being can mentally apply evaluation to determine a loss function of the embedding representation model). Claim 7 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the additional elements recited consists of:
wherein the training the embedding representation model to obtain the second entity embedding representation of each entity and the relationship embedding representation of the entity relationship comprises: (which is restricting the abstract idea to a Particular Technological Environment, by MPEP 2106.05(h))
training, according to a preset training method, the embedding representation model to minimize a function value of the loss function, to obtain the second entity embedding representation and the relationship embedding representation (to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f))
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements recited, alone or in combination, do not provide significantly more than the abstract idea itself. Additional element a is only restricting the abstract idea to a Particular Technological Environment (MPEP 2106.05(h)) which cannot provide significantly more. Additional element b is merely applying the abstract idea on a computer (MPEP 2106.05(f)) which cannot provide significantly more. Thus, the claim is subject-matter ineligible.
Regarding Claim 8:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 8 recites the method of Claim 7. Claim 7 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 8 further recites initializing the embedding representation of each entity and the embedding representation of the entity relationship, to obtain an initial entity embedding representation and an initial relationship embedding representation (a human being can mentally apply evaluation to initialize embedding representations and relationship embedding representations to obtain an initial representation). Claim 8 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the additional elements recited consists of:
wherein the function value is associated with an embedding representation of each entity, an embedding representation of the entity relationship, and a unary text embedding representation; the training, according to the preset training method, the embedding representation model to minimize the function value of the loss function, to obtain the second entity embedding representation and the relationship embedding comprises: (which is restricting the abstract idea to a Particular Technological Environment, by MPEP 2106.05(h))
iteratively updating the first weight coefficient according to an attention mechanism to update the unary text embedding representation, and iteratively updating the initial entity embedding representation and the initial relationship embedding representation according to the preset training method (which is an insignificant extra-solution activity, by MPEP 2106.05(g))
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements recited, alone or in combination, do not provide significantly more than the abstract idea itself. Additional element a is only restricting the abstract idea to a Particular Technological Environment (MPEP 2106.05(h)) which cannot provide significantly more. Additional element b falls within MPEP 2106.05(d) as well-understood, routine and conventional activities of performing repetitive calculations (MPEP 2106.05(d)(II)(ii): “Performing repetitive calculations, Flook, 437 U.S. at 594, 198 USPQ2d at 199”). Thus, the claim is subject-matter ineligible.
Regarding Claim 9:
Subject Matter Eligibility Analysis Step 1:
Dependent Claim 9 recites the method of Claim 1. Claim 1 is a method, thus a process, one of the four statutory categories of patentable subject matter.
Subject Matter Eligibility Analysis Step 2A Prong 1:
However, Claim 9 further recites:
replacing the entity relationship comprised in the known fact triplet with another entity relationship between the M entities, or replacing one entity comprised in the known fact triplet with another M entity in the entities, to obtain a predicted fact triplet (a human being can mentally apply evaluation to replace an entity relationship within a knowledge graph with the aid of pen and paper to obtain a predicted fact)
determining a recommended score of the predicted fact triplet based on a second entity embedding representation of an entity in the predicted fact triplet and a relationship embedding representation of the entity relationship (a human being can mentally apply evaluation to determine a recommend score of a predicted fact triplet based on an embedding representation entities)
adding, based on the recommended score, the predicted fact triplet to the target knowledge graph (a human being can mentally apply evaluation to add a predicted triplet to a knowledge graph)
Claim 9 thus recites an abstract idea (that falls into the “mental processes” group of abstract ideas).
Subject Matter Eligibility Analysis Step 2A Prong 2:
This judicial exception is not integrated into a practical application because the sole additional elements recited consists of wherein the target knowledge graph comprises a known fact triplet, and the known fact triplet comprises two entities in the M entities and an entity relationship; the training the embedding representation model to obtain the second entity embedding representation of each entity and the relationship embedding representation of the entity relationship comprises: (which is restricting the abstract idea to a Particular Technological Environment, by MPEP 2106.05(h)).
Subject Matter Eligibility Analysis Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the new sole additional element recited, alone or in combination, does not provide significantly more than the abstract idea itself. The additional element is only restricting the abstract idea to a Particular Technological Environment (MPEP 2106.05(h)) which cannot provide significantly more. Thus, the claim is subject-matter ineligible.
Regarding Claims 10 and 12-15:
Claims 10 and 12-15 incorporate substantively all the limitations of Claims 1 and 3-6 in an apparatus (thus a machine) and further recites a new additional element at least one processor; and one or more memories coupled to the at least one processor and storing executable program instructions that, when executed by the at least one processor, cause the at least one processor to: (these claim limitations appear to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f)) and does not appear to integrate the abstract idea into a particular application; thus, the claim is subject-matter ineligible as it does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself); thus, Claims 10 and 12-15 are rejected for reasons set forth in the rejections of Claims 1 and 3-6, respectively.
Regarding Claims 16 and 18-20:
Claims 16 and 18-20 incorporate substantively all the limitations of Claims 1 and 3-5 in a non-transitory computer-readable storage medium and further recites a new additional element at least one processor; and one or more memories coupled to the at least one processor and storing executable program instructions that, when executed by the at least one processor, cause the at least one processor to: (these claim limitations appear to perform a mental process and the performance of an abstract idea on a computer is no more than instructions to “apply it” on a computer, by MPEP 2106.05(f)) and does not appear to integrate the abstract idea into a particular application; thus, the claim is subject-matter ineligible as it does not include additional elements that are sufficient to amount to significantly more than the judicial exception because the additional elements, alone or in combination, do not provide significantly more than the abstract idea itself); thus, Claims 16 and 18-20 are rejected for reasons set forth in the rejections of Claims 1 and 3-5, respectively.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-4, 9-10, 12-13, 16 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al., “Learning Entity and Relation Embeddings for Knowledge Graph Completion”, in view of Xie et al., “Representation Learning of Knowledge Graphs with Entity Descriptions”.
Regarding Claim 1:
Lin teaches:
A knowledge graph embedding representation method, comprising, performed by an electronic device including at least one processor and a memory, comprising:
(Lin, Page 2183, Column 2, Paragraph 5, “In this paper, we evaluate our methods with two typical knowledge graphs”; Page 2183, Column 1, Paragraph 4, “… we propose a new method, which models entities and relations in … entity space and relation spaces, and performs translation in relation space, hence named as TransR … each triple, entities embeddings are … and relation embedding …”; Page 2181, Abstract, “…source code … can be obtained from https: //github.com/mrlyk423/relation extraction”. The method taught by Lin utilizes a TransR model to represent the knowledge graph embeddings; where the method contains source code to handle the complexity of the TransR model for training, testing/evaluation which is interpreted as an device with at least one processor, memory, and a storage to run the source code within the electronic device).
obtaining M entities in a target knowledge graph, wherein the M entities comprise an entity 1, an entity 2, through an entity M, and M is an integer greater than 1;
(Lin, Table 1. Table 1 shows the data sets used for the target knowledge graphs within the methods/experiments within Lin comprising M entities where M is greater than 1).
obtaining, from a preset knowledge base,
(Lin, Table 1; Page 2183, Column 2, Paragraph 5, “In this paper, we evaluate our methods with two typical knowledge graphs, built with WordNet … and Freebase …”. Table 1 shows the WordNet and Freebase data set versions used to build the knowledge graphs (WordNet and Freebase is interpreted by the examiner as a preset knowledge base for the methods/experiments of embedding knowledge graphs for link prediction)).
N related entities of an entity m in the M entities and K concepts corresponding to a related entity n in the N related entities, wherein the N related entities comprise a related entity 1, a related entity 2, through a related entity N, N and K are integers not less than 1, m = 1, 2, 3, through M, n = 1, 2, 3, through N, the entity m is semantically correlated with the N related entities, and the related entity n is semantically correlated with the K concepts …
(Lin, Table 1; (Lin, Figure 1; Page 2183, Column 1, Paragraph 5, “In TransR, for each triple, entities embeddings are (h, r, t) set as h, t ∈ ℝk and relation embedding is set as r ∈ ℝd …”. Table 1 comprises #Ent (interpreted as the M entities) and #Rel (interpreted as relations/links for the N related entities where N is greater than 1). Knowledge graphs consist of two primary concepts which are nodes (entities or concepts) and edges (relationships). Triples (h, r, t) are used within Lin to represent the knowledge graph data where h = head (entity or concept), r = relationship (edge), t = tail (entity or concept); where r is the relationship between h and t. Thus, the K concepts are interpreted as entity nodes within the knowledge graphs and are linked (related) from entity to entity (node to node) based off the semantic correlation between an entity and a concept (entity)).
determining a semantic correlation between each of the M entities and each of the N related entities of the entity m, and determining a first entity embedding representation of each of the N related entities based on corresponding K concepts …
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “we propose TransR, which represent entities and relations in distinct semantic space bridged by relation-specific matrices … which models entities and relations in distinct spaces, i.e., entity space and relation spaces, and performs translation in relation space … In TransR, for each triple, entities embeddings are (h, r, t) set as h, t ∈ ℝk and relation embedding is set as r ∈ ℝd … For each relation r, we set a projection matrix Mr ∈ ℝkxd, which may projects entities from entity space to relation space. With the mapping matrix, we define the projected vectors of entities as
PNG
media_image1.png
27
251
media_image1.png
Greyscale
The score function is correspondingly defined as
PNG
media_image2.png
28
248
media_image2.png
Greyscale
”; Page 2184, Column 2, Paragraph 2, “… TransR achieves great improvement consistently on all mapping categories of relations … TransR provides more precise representation for both entities and relation and their complex correlations, as illustrated in Fig. 1; … which shows the ability of TransR to discriminate relevant from irrelevant entities via relation-specific projection”. The TransR model models entities as vectors in the entity space and models each relation as a vector in the relation space as a projection matrix. The score function is used to rank entities in descending order of similarity scores. As the modelling represents entities and relations in a semantic space, the model determines a semantic correlation between each entity and each related entity (which are based on the concepts (related nodes)) through the entity/relation spaces. Each triple (entity embedding vectors which are interpreted as entity embedding representations) will be projected with each relation vector; thus, a first entity embedding representation of each of the N related entities based on corresponding K concepts will be determined; where the scores with the highest ranking will be the most relevant semantically correlated entities for link prediction/triple classification between entity to related concepts (based on entities and relations). Fig. 1 shows a simple illustration of the mapping which indicates the precise representation for both entities and relations and their complex correlations used for prediction).
modeling, based on the first entity embedding representation and the semantic correlation, an embedding representation of the M entities and an embedding representation of an entity relationship between the M entities, to obtain an embedding representation model; and
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “we propose TransR, which represent entities and relations in distinct semantic space bridged by relation-specific matrices … which models entities and relations in distinct spaces, i.e., entity space and relation spaces, and performs translation in relation space … In TransR, for each triple, entities embeddings are (h, r, t) set as h, t ∈ ℝk and relation embedding is set as r ∈ ℝd … For each relation r, we set a projection matrix Mr ∈ ℝkxd, which may projects entities from entity space to relation space. With the mapping matrix, we define the projected vectors of entities as
PNG
media_image1.png
27
251
media_image1.png
Greyscale
The score function is correspondingly defined as
PNG
media_image2.png
28
248
media_image2.png
Greyscale
”; Page 2184, Column 2, Paragraph 2, “… TransR achieves great improvement consistently on all mapping categories of relations … TransR provides more precise representation for both entities and relation and their complex correlations, as illustrated in Fig. 1; … which shows the ability of TransR to discriminate relevant from irrelevant entities via relation-specific projection”. The TransR model models entities as vectors in the entity space and models each relation as a vector in the relation space as a projection matrix. The score function is used to rank entities in descending order of similarity scores. As the modelling represents entities and relations in a semantic space, the model determines a semantic correlation between each entity and each related entity (which are based on the concepts (related nodes)) through the entity/relation spaces. Each triple (entity embedding vectors which are interpreted as entity embedding representations) will be projected with each relation vector; thus, a first entity embedding representation of each of the N related entities based on corresponding K concepts will be determined; where the scores with the highest ranking will be the most relevant semantically correlated entities for link prediction between entity to related concepts (based on entities and relations). Fig. 1 shows a simple illustration of the mapping which indicates the precise representation for both entities and relations and their complex correlations used for prediction).
training the embedding representation model to obtain a second entity embedding representation of each entity and a relationship embedding representation of the entity relationship.
(Lin, Table 1; Page 2183, Column 2, Paragraph 4, “The learning process of TransR and CTransR is carried out using stochastic gradient descent (SGD) … we initialize entity and relation embeddings with results of TransE, and initialize relation matrices as identity matrices”; Page 2184, Column 1, Paragraph 1, “Link prediction aims to predict the missing h or t for a relation fact triple (h, r, t),”; Page 2184, Column 2, Paragraph 4, “Triple classification aims to judge whether a given triple (h, r, t) is correct or not” Table 1 shows the training data set partition within the preset knowledge bases for the target knowledge graphs used within the method. The training and learning of the model is used to predict missing links and make judgements on whether a triple is correct or not to obtain a new entity embedding representation. Thus, the model TransR with the initial values for the first entity/relation embedding representation being from the TransE model are trained to obtain the second entity embedding representations).
While Lin teaches the determining of entity embedding representations with performing vectorization to obtain word vectors… Lin does not explicitly disclose the K concepts comprising textual or semantic descriptors representing semantic information.
However, Xie explicitly discloses:
… wherein the K concepts comprise textual or semantic descriptors automatically identified on a page linked to the related entity n, the K concepts representing semantic information that characterizes attributes, categories, or contextual features of the related entity n;
(Xie, Page 2660, Column 1, Paragraph 1, “… DKRL model can build representations for those novel entities automatically from their descriptions”; Page 2660, Column 2, Paragraph 6, “… we propose two encoders to build description-based representations … a continuous bag-of-words encoder for entity construction, then we propose a deep convolutional neural network encoder for a better understanding of textual information”; Page 2661, Column 1, Paragraph 3, “In … (CBOW), we select top n keywords in the description for each entity as the input (some classical textual features ”; Figures 2-3. The K concepts in Xie are interpreted as the descriptions of the nodes (which are entities comprising contextual features that characterize the entities description). Xie’s DKRL model creates knowledge graph representations where nodes are automatically linked to related entities based on descriptions for a better understanding of textual information (interpreted by the examiner as textual descriptors); where textual descriptors are handled with the CBOW encoder (Fig. 2 ) and the semantic descriptors are handled via CNN encoder (Fig. 3)).
… wherein determining the first entity embedding representation comprises performing vectorization processing on each concept in the K concepts corresponding to the related entity n by using a word-vector generation model to obtain a word vector of each concept and performing average summation on the word vectors of the K concepts to obtain the first entity embedding representation of the related entity n;
(Xie, Page 2661, Figure 2, Column 1, Paragraph 2, “From each short description, we can generate a set of keywords … similar entities should have similar descriptions, and … similar keywords … CBOW … we select top n keywords in the description for each entity as the input … TF-IDF … Then we simply sum up the embeddings of keywords to get the entity embedding … Equation (4)”.; Page 2661, Column 2, Paragraph 2, “In our experiments, we use the word embeddings trained on Wikipedia by word2vec … as inputs for the CNN Encoder”. The DKRL model taught by Xie comprises two types of encoders (CBOW: continuous bag of words for entity construction; CNN: convolutional neural network for understanding textual information) for parsing textual inputs and outputting entity embeddings. The CNN encoder utilizes Word2Vec (word vector generation) which teaches performing vectorization on the textual inputs for each concept using the DKRL model. The CBOW encoder teaches the adding up the embeddings from the bag of words; thus, interpreted by the examiner as performing summation via Equation (4). The embeddings are keywords from a bag of words where the input is like TF-IDF which is weighted averaging; thus, interpreted by the examiner as teaching performing average summation on the word embedded vectors).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to utilize the TransR methodology of Lin for entity/relational generation in respective separate spaces, with the encoders taught within Xie’s methodology for Knowledge Graphs with entity descriptions to illustrate the performance and effectiveness of adding together different weighed components of the entity representations to consider all entities equally to obtain the representation as Xie notes the explicit extensions of the TransR and similar models (see Xie, Page 2660, Column 1, Paragraph 1, “…indicates the good generalization ability and robustness of
the DKRL model, which is … important for largescale KGs and their applications in Web domain”; Page 2665, Column 1, Paragraph 1, “… We verify the effectiveness of description-based representations only with TransE, and it is not difficult for further explorations with more sophisticated extension models of TransE …”).
Regarding Claim 3:
Lin/Xie teach the method of Claim 1 and Lin further teaches:
wherein the modeling, based on the first entity embedding representation and the semantic correlation, the embedding representation of the M entities and the embedding representation of the entity relationship between the M entities, to obtain the embedding representation model comprises:
determining, based on the semantic correlation and a first entity embedding representation of the N related entities,
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “we propose TransR, which represent entities and relations in distinct semantic space bridged by relation-specific matrices … which models entities and relations in distinct spaces, i.e., entity space and relation spaces, and performs translation in relation space …”; Page 2184, Column 2, Paragraph 2, “… TransR achieves great improvement consistently on all mapping categories of relations … TransR provides more precise representation for both entities and relation and their complex correlations, as illustrated in Fig. 1; … which shows the ability of TransR to discriminate relevant from irrelevant entities via relation-specific projection”. TransR models entities as vectors in the entity space and models each relation as a vector in the relation space as a projection matrix. The score function is used to rank entities in descending order of similarity scores. As the modelling represents entities and relations in a semantic space, the model determines a semantic correlation between each entity and each related entity through translation (which are based on the concepts (related nodes)) through the entity/relation spaces. Each triple (entity embedding vectors which are interpreted as entity embedding representations) will be projected with each relation vector; thus, a first entity embedding representation of each of the N related entities based on corresponding K concepts will be determined; where the scores with the highest ranking will be the most relevant semantically correlated entities for link prediction between entity to related concepts (based on entities and relations). Fig. 1 shows a simple illustration of the mapping which indicates the precise representation for both entities and relations and their complex correlations used for prediction).
a unary text embedding representation corresponding to each entity;
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “In TransR, for each triple, entities embeddings are (h, r, t) set as h, t ∈ ℝk …”. The head and tail entities within a triple are interpreted as unary text embedding representations as the respective embedded vector representations are involving a single component. The unary text embedding representations of the entities are based on semantic correlations due to being projected into distinct semantic space which is bridged by relation-specific matrix which allows the determination of scores to find the most relevant entities for link prediction/classification).
determining, based on the N related entities, a common related entity of every two entities in the M entities; determining, based on the semantic correlation and a first entity embedding representation of the common related entity, a binary text embedding representation corresponding to the every two entities; and
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “In TransR … relation embedding is set as r ∈ ℝd …”; Page 2184, Column 1, Paragraph 1, “Link prediction aims to predict the missing h or t for a relation fact triple (h, r, t)”. The relation vectors are considered binary as it relates the head and tail vectors which is interpreted as a binary text embedding representation as the vector involves two components. The relation between the every two entities is based on the N related entities (which is based on the concepts (entities) that can be related to an entity). The relationship will be the edge between the two nodes (entity to entity (based on concept)); thus, a common related entity of every two entities in the M entities as the knowledge graph is built to predict/create links between all missing relationships for h or t entities to have a complete related triple).
establishing, based on the unary text embedding representation and the binary text embedding representation, the embedding representation model.
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “With the mapping matrix, we define the projected vectors of entities as
PNG
media_image1.png
27
251
media_image1.png
Greyscale
…”. The unary and binary text embedding representations are used for the TransR knowledge graph (embedding representation model) with the use of mapping the unary and binary embedding vectors in the same vector space to construct the knowledge graph).
Regarding Claim 4:
Lin/Xie teach the method of Claim 3 and Lin further teaches:
wherein the establishing, based on the unary text embedding representation and the binary text embedding representation, the embedding representation model comprises:
mapping the unary text embedding representation and the binary text embedding representation to a same vector space, to obtain a semantically enhanced unary text embedding representation and a semantically enhanced binary text embedding representation; and
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “With the mapping matrix, we define the projected vectors of entities as
PNG
media_image1.png
27
251
media_image1.png
Greyscale
… ”; Page 2184, Column 2, Paragraph 2, “… TransR achieves great improvement consistently on all mapping categories of relations … TransR provides more precise representation for both entities and relation and their complex correlations, as illustrated in Fig. 1; … which shows the ability of TransR to discriminate relevant from irrelevant entities via relation-specific projection”; Page 2181, Column 2, Paragraph 5, “The relation-specific projection can make the head/tail entities that actually hold the relation (denoted as colored circles) close with each other, and also get far away from those that do not hold the relation (denoted as colored triangles)”. The TransR model is mapping the unary text embedding representation and the binary text embedding representation in distinct spaces and then translates them to the same vector space (relation space) and can be shown in Figure 1. The mapping creates a semantically enhanced representation for both embedding representations as Figure 1 shows the related entities will get closer within the relation space if there is more of a relation and farther apart if there is less of a relation. As the modelling represents entities and relations in a semantic space, the model determines a semantic correlation between each entity and each related entity (which are based on the concepts (related nodes)) through the entity/relation spaces. The scores with the highest ranking will be the most relevant semantically correlated entities for link prediction between entity to related concepts (based on entities and relations). Fig. 1 shows a simple illustration of the mapping which indicates the precise representation for both entities and relations and their complex correlations used for prediction).
establishing, based on the semantically enhanced unary text embedding representation and the semantically enhanced binary text embedding representation, the embedding representation model.
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “With the mapping matrix, we define the projected vectors of entities as
PNG
media_image1.png
27
251
media_image1.png
Greyscale
…”. The semantically enhanced unary and binary text embedding representations are used for the TransR knowledge graph (embedding representation model) with the use of mapping the unary and binary embedding vectors in the same vector space to construct the knowledge graph).
Regarding Claim 9:
Lin/Xie teach the method of Claim 1 and Lin further teaches:
wherein the target knowledge graph comprises a known fact triplet, and the known fact triplet comprises two entities in the M entities and an entity relationship; the training the embedding representation model to obtain the second entity embedding representation of each entity and the relationship embedding representation of the entity relationship comprises:
replacing the entity relationship comprised in the known fact triplet with another entity relationship between the M entities, or replacing one entity comprised in the known fact triplet with another entity in the M entities, to obtain a predicted fact triplet;
(Lin, Page 2184, Column 1, Paragraph 2, “… for each test triple (h, r, t), we replace the head/tail entity by all entities in the knowledge graph …”. The entity relationships are replaced by the best results/scoring relationships for the triplets to obtain a predicted fact triplet).
determining a recommended score of the predicted fact triplet based on a second entity embedding representation of an entity in the predicted fact triplet and a relationship embedding representation of the entity relationship; and
(Lin, Page 2184. Column 1, Paragraph 2, “… and rank these entities in descending order of similarity scores calculated by score function fr”; Page 2183, Column 1, Paragraph 6, “… The score function is correspondingly defined as
PNG
media_image2.png
28
248
media_image2.png
Greyscale
”. The score function is utilized to determine a recommended score for the predicted fact triplet).
adding, based on the recommended score, the predicted fact triplet to the target knowledge graph.
(Lin, Page 2184, Column 1, Paragraph 2, “… for each test triple (h, r, t), we replace the head/tail entity by all entities in the knowledge graph, and rank these entities in descending order of similarity scores calculated by score function fr”; Page 2184, Column 1, Paragraph 1, “Link prediction aims to predict the missing h or t for a relation fact triple (h, r, t)”. For missing links/relations between entities or sub-optimal links, Lin teaches the replacing of the head/tail entities with the highest similarity scored triple based off the score function. Thus, adding the predicted fact triplet is taught by Lin by replacing or adding links for sub-optimal or missing links, respectively, to optimize the knowledge graph).
Regarding Claims 10 and 12-13:
Claims 10 and 12-13 incorporate substantively all the limitations of Claims 1 and 3-4 in an apparatus and further recites a new additional element at least one processor; and one or more memories coupled to the at least one processor and storing executable program instructions that, when executed by the at least one processor, cause the at least one processor to: (Lin, Abstract, “The source code of this paper can be obtained from https: //github.com/mrlyk423/relation extraction”; Page 2184, Paragraph 1, “… the system is asked to rank…”; Page 2185, Paragraph 2, “The computation complexity of TransR is higher than both TransE and TransH, which takes about 3 hours for training”. The system of Lin utilizes a computer to handle the complexity of the TransR model for training, testing/evaluation which is interpreted as an apparatus with at least one processor, memory, and a storage to run the source code within the system); thus, Claims 10 and 12-13 are rejected for reasons set forth in the rejections of Claims 1 and 3-4, respectively.
Regarding Claims 16 and 18-19:
Claims 16 and 18-19 incorporate substantively all the limitations of Claims 1 and 3-4 in a non-transitory computer-readable storage medium and further recites a new additional element at least one processor; and one or more memories coupled to the at least one processor and storing executable program instructions that, when executed by the at least one processor, cause the at least one processor to: (Lin, Abstract, “The source code of this paper can be obtained from https: //github.com/mrlyk423/relation extraction”; Page 2184, Paragraph 1, “… the system is asked to rank…”; Page 2185, Paragraph 2, “The computation complexity of TransR is higher than both TransE and TransH, which takes about 3 hours for training”. The system of Lin utilizes a computer to handle the complexity of the TransR model for training, testing/evaluation which is interpreted as a manufacture with at least one processor, memory, and a storage to run the source code); thus, Claims 16 and 18-19 are rejected for reasons set forth in the rejections of Claims 1 and 3-4, respectively.
Claims 5-8, 14-15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al., “Learning Entity and Relation Embeddings for Knowledge Graph Completion”, in view of Xie et al., “Representation Learning of Knowledge Graphs with Entity Descriptions”, in view of Liu et al., “Deep Learning in Natural Language Processing: [Chapter 5] Deep Learning in Knowledge Graph”, Pages 117–145.
Regarding Claim 5:
Lin/Xie teach the method of Claim 3 and Lin further teaches:
wherein the determining, based on the semantic correlation and the first entity embedding representation of the N related entities, the unary text embedding representation corresponding to each entity comprises:
using the semantic correlation as a … of each of the N related entities; and
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “In TransR … relation embedding is set as r ∈ ℝd …”; Page 2184, Column 1, Paragraph 1, “Link prediction aims to predict the missing h or t for a relation fact triple (h, r, t)”. The relation between the every two entities is based on the N related entities (which is based on the concepts (entities) that can be related to an entity). As the modelling represents entities and relations in a semantic space, the model determines a semantic correlation between each entity and each related entity (which are based on the concepts (related nodes)) through the entity/relation spaces. The scores with be ranked will and are used for the semantic correlation for link prediction between entity to related concepts (based on entities and relations)).
performing, based on … the first entity embedding representation of the N related entities, to obtain the unary text embedding representation.
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “In TransR, for each triple, entities embeddings are (h, r, t) set as h, t ∈ ℝk and relation embedding is set as r ∈ ℝd …”; Page 2184, Paragraph 1, “Link prediction aims to … rank a set of candidate entities from the knowledge graph, instead of only giving one best result … With the mapping matrix, we define the projected vectors of entities as
PNG
media_image1.png
27
251
media_image1.png
Greyscale
…”. The semantically enhanced unary and binary text embedding representations are obtained for the TransR knowledge graph (embedding representation model) with the use of mapping the unary and binary embedding vectors in the same vector. The head and tail vectors are considered unary as they involve one component which is interpreted as a unary text embedding representation and is related to the N related entities via equation 7).
While Lin/Xie teaches the use of semantic correlation of each of the N related entities. Lin/Xie does not explicitly disclose the weight coefficient.
However, Liu explicitly discloses:
first weight coefficient and weighted summation
(Liu, Page 132, Paragraph 1, “Attentive Encoder… Formally, the document representation is defined as a weighted sum of sentence vectors:
PNG
media_image3.png
45
307
media_image3.png
Greyscale
where αi is defined as
PNG
media_image4.png
50
329
media_image4.png
Greyscale
where A is a diagonal matrix and r is the representation vector of relation r”. αi is interpreted as a weight coefficient that is updated based on the iterative step which is understood from i. This is utilized for the weighted summation on the embedding representations. αi is the first weight coefficient when i & j = 1 which causes the exponential function to be for the first word vector (x1) considering the diagonal matrix and representation vector of r).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to utilize the determining of a unary text embedding representation utilizing semantic correlation by Lin/Xie, with the weight coefficients and attention mechanisms taught by Liu to illustrate the importance of being able to handle incorrect labels and embeddings with no related information by applying attentive weights (see Liu, Page 132, Paragraph 5, “Attentive Encoder. Due to the wrong label issue brought by distant supervision assumption inevitably, the performance of average encoder will be influenced by those sentences that contain no related information. To address this issue… employ a selective attention to de-emphasize…”).
Regarding Claim 6:
Lin/Xie teach the method of Claim 3 and Lin further teaches:
wherein the determining, based on the semantic correlation and a first entity embedding representation of the common related entity, the binary text embedding representation corresponding to the every two entities comprises:
using the common related entity and a minimum semantic correlation of semantic correlations of semantic correlations of every two entities
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “In TransR … relation embedding is set as r ∈ ℝd …”; Page 2184, Paragraph 1, “Link prediction aims to … rank a set of candidate entities from the knowledge graph, instead of only giving one best result”. The relation vectors are considered binary as it relates the head and tail vectors which is interpreted as a binary text embedding representation as the vector involves two components which are the common related entities (head and tails related by a relation). The ranking is done for finding the highest -> lowest semantically correlated entities which is done so not only the best result is found but also the least optimal result (which is interpreted as the minimum semantic correlation between every two entities)).
performing, based on the … on the first entity embedding representation of the common related entity, to obtain the binary text embedding representation.
(Lin, Figure 1; Page 2183, Column 1, Paragraph 3, “In TransR … relation embedding is set as r ∈ ℝd …”; Page 2184, Paragraph 1, “Link prediction aims to … rank a set of candidate entities from the knowledge graph, instead of only giving one best result … With the mapping matrix, we define the projected vectors of entities as
PNG
media_image1.png
27
251
media_image1.png
Greyscale
…”. The semantically enhanced unary and binary text embedding representations are used for the TransR knowledge graph (embedding representation model) with the use of mapping the unary and binary embedding vectors in the same vector space to construct the knowledge graph. The relation vectors are considered binary as it relates the head and tail vectors which is interpreted as a binary text embedding representation as the vector involves two components which are the common related entities (head and tails related by a relation). The ranking is done for finding the highest -> lowest semantically correlated entities which is done so not only the best result is found but also the least optimal result (which is interpreted as the minimum semantic correlation between every two entities)).
While Lin/Xie teaches the use of determining, based on semantic correlation, a binary text embedding representation related to every two entities. Lin/Xie does not explicitly disclose a minimum semantic correlation and a second weight coefficient.
However, Liu explicitly discloses:
second weight coefficient of the common related entity and weighted summation
(Liu, Page 132, Paragraph 1, “Attentive Encoder… Formally, the document representation is defined as a weighted sum of sentence vectors:
PNG
media_image3.png
45
307
media_image3.png
Greyscale
where αi is defined as
PNG
media_image4.png
50
329
media_image4.png
Greyscale
where A is a diagonal matrix and r is the representation vector of relation r”. αi is interpreted as the weight coefficient which utilizes a weighted summation and is considered the second weight coefficient when i & j are updated accordingly to the second weight (which was noted as the minimum semantic correlation and taught above within Lin) with the exponential function containing the diagonal matrix and r for the common related entity).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to utilize the determining of a binary text embedding representation utilizing semantic correlation by Lin/Xiu, with the weight coefficients taught by Liu to illustrate the importance of being able to handle incorrect labels and embeddings with no related information by applying attentive weights (see Liu, Page 132, Paragraph 5, “Attentive Encoder. Due to the wrong label issue brought by distant supervision assumption inevitably, the performance of average encoder will be influenced by those sentences that contain no related information. To address this issue… employ a selective attention to de-emphasize…”).
Regarding Claim 7:
Lin/Xie/Liu teach the method of Claim 5 and Lin further teaches:
wherein the training the embedding representation model to obtain the second entity embedding representation of each entity and the relationship embedding representation of the entity relationship comprises:
determining a loss function of the embedding representation model; and
(Lin, Page 2183, Column 2, Paragraph 2, “Training Method and Implementation Details: We define the following margin-based score function as objective for training
PNG
media_image5.png
64
375
media_image5.png
Greyscale
where max(x, y) aims to get the maximum between x and y, γ is the margin, S is the set of correct triples and S’ is the set of incorrect triples”; L was determined as a loss function of the embedding representation model).
training, according to a preset training method, the embedding representation model to minimize a function value of the loss function, to obtain the second entity embedding representation and the relationship embedding representation.
(Lin, Page 2183, Column 2, Paragraph 4, “The learning process of TransR and CTransR is carried out using stochastic gradient descent (SGD)”; Section [Training method and Implementation Details] teaches the training based on a preset training method to minimize loss and obtain the second embedding representations).
Regarding Claim 8:
Lin/Xie/Liu teach the method of Claim 7 and Lin further teaches:
wherein the function value is associated with an embedding representation of each entity, an embedding representation of the entity relationship, and a unary text embedding representation; the training, according to the preset training method, the embedding representation model to minimize the function value of the loss function, to obtain the second entity embedding representation and the relationship embedding comprises:
initializing the embedding representation of each entity and the embedding representation of the entity relationship, to obtain an initial entity embedding representation and an initial relationship embedding representation; and
(Lin, Page 2183, Column 2, Paragraph 4, “To avoid overfitting, we initialize entity and relation embeddings with results of TransE, and initialize relation matrices as identity matrices”. The entity relationships are initialized with initial values and then iterated through the training methodology).
iteratively updating
(Lin, “The learning process of TransR and CTransR is carried out using stochastic gradient descent (SGD) …”. The preset training method of Lin teaches iteratively updating via gradient descent).
the first weight coefficient according to an attention mechanism to update the unary text embedding representation, and iteratively updating the initial entity embedding representation and the initial relationship embedding representation according to the preset training method.
(Liu, Fig. 5.6; Page 128, Paragraph 1, “… word attention can also combine all local feature vectors together. It uses attention mechanism … to learn attention weights on each step. Suppose H = [h1, h2, . . . , hm] is the matrix consisting of all output vectors that produced by the recurrent layer, the whole sentence’s feature vector x is formed by a weighted sum of each step’s output”; Page 132, Paragraph 1, “Attentive Encoder… Formally, the document representation is defined as a weighted sum of sentence vectors:
PNG
media_image3.png
45
307
media_image3.png
Greyscale
where αi is defined as
PNG
media_image4.png
50
329
media_image4.png
Greyscale
where A is a diagonal matrix and r is the representation vector of relation r”. αi is interpreted as the first weight coefficient which gets updated iteratively through the training method of Lin. Each step within the iterative process as noted by Liu, will form a sentence vector (interpreted as a triple: h, r, t (subject, predicate, and object) by a weighted sum which is shown within Fig. 5.6 for the shortest path between target entities for effective relations).
The motivation of Claim 5’s combination of Lin/Xie/Liu is still maintained.
Regarding Claims 14-15:
Claims 14-15 incorporate substantively all the limitations of Claims 5-6 in an apparatus and further recites a new additional element at least one processor; and one or more memories coupled to the at least one processor and storing executable program instructions that, when executed by the at least one processor, cause the at least one processor to: (Lin, Abstract, “The source code of this paper can be obtained from https: //github.com/mrlyk423/relation extraction”; Page 2184, Paragraph 1, “… the system is asked to rank…”; Page 2185, Paragraph 2, “The computation complexity of TransR is higher than both TransE and TransH, which takes about 3 hours for training”. The system of Lin utilizes a computer to handle the complexity of the TransR model for training, testing/evaluation which is interpreted as an apparatus with at least one processor, memory, and a storage to run the source code within the system); thus, Claims 14-15 are rejected for reasons set forth in the rejections of Claims 5-6, respectively.
Regarding Claim 20:
Claim 20 incorporates substantively all the limitations of Claim 5 in a non-transitory computer-readable storage medium and further recites a new additional element at least one processor; and one or more memories coupled to the at least one processor and storing executable program instructions that, when executed by the at least one processor, cause the at least one processor to: (Lin, Abstract, “The source code of this paper can be obtained from https: //github.com/mrlyk423/relation extraction”; Page 2184, Paragraph 1, “… the system is asked to rank…”; Page 2185, Paragraph 2, “The computation complexity of TransR is higher than both TransE and TransH, which takes about 3 hours for training”. The system of Lin utilizes a computer to handle the complexity of the TransR model for training, testing/evaluation which is interpreted as a manufacture with at least one processor, memory, and a storage to run the source code); thus, Claim 20 is rejected for reasons set forth in the rejections of Claim 5.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IBRAHIM RAHMAN whose telephone number is (703)756-1646. The examiner can normally be reached M-F 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571) 272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/I.R./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122