DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is responsive to the Application filed on 04/19/2023. Claims 1-20 are pending in the case. Claims 1, 9, and 17 are independent claims.
Claim Rejections - 35 U.S.C. § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant are advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1-5, 9-13, and 17-19 are rejected under 35 U.S.C. § 103 as being unpatentable over Kipf et al. (Kipf, Thomas N., and Max Welling. "Semi-supervised classification with graph convolutional networks." arXiv preprint arXiv:1609.02907 (2016)., hereinafter Kipf) in view of Gal et al. (Gal, Yarin, and Zoubin Ghahramani. "Dropout as a bayesian approximation: Representing model uncertainty in deep learning." In international conference on machine learning, pp. 1050-1059. PMLR, 2016., hereinafter Gal) and Hamilton et al. (Hamilton, William L., Rex Ying, and Jure Leskovec. "Inductive Representation Learning on Large Graphs." arXiv preprint arXiv:1706.02216 (2017)., hereinafter Hamilton).
As to independent claim 1, Kipf teaches:
A computing device for quantifying certainty for a prediction based on a knowledge graph, the computing device comprising (Title and abstract):
a reception circuitry configured to receive a target triple and a knowledge graph comprising a set of structured data… (Page 6, "relation nodes r1 and r2 for each entity pair (e1; r; e2) as (e1; r1) and (e2; r2)." Page 1, "graph structure." Graph structured data representing relationships among entities);
a knowledge graph embedding generation circuitry configured to convert the target triple to an embeddings space… (Section A.2, semi-supervised node embeddings. Learns low-dimensional embeddings for graph nodes);
a scoring circuitry configured to generate a plausibility prediction for the target triple using a scoring function (Page 6, "GCN as described in Section 3.1 and evaluate prediction accuracy." Outputs predictions (e.g., link existence probability));….
Kipf does not appear to expressly teach a set of certainty scores for the structured data; a control circuitry configured to repeat the acts of the knowledge graph embedding generation circuitry and the scoring circuitry N times with dropouts to obtain N plausibility scores for the target triple, wherein N is an integer larger than one; and an output circuitry configured to generate a predicted plausibility score and a certainty score for the target triple based on the N plausibility scores, and output the predicted plausibility score and the certainty score.
Gal teaches a set of certainty scores for the structured data (Figure 2, predictive mean and uncertainties. The Monte Carlo dropout generates predictive mean and variance, i.e., uncertainty scores associated with predictions); a control circuitry configured to repeat the acts of the knowledge graph embedding generation circuitry and the scoring circuitry N times with dropouts to obtain N plausibility scores for the target triple, wherein N is an integer larger than one (Page 4, "performing T stochastic forward passes." Section 3, dropout as a Bayesian approximation); and an output circuitry configured to generate a predicted plausibility score and a certainty score for the target triple based on the N plausibility scores, and output the predicted plausibility score and the certainty score (Page 4, "performing T stochastic forward passes." Page 5, "standard deviation expresses the models’ uncertainty about the point").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
Kipf does not appear to expressly teach according to neighborhood sampling, wherein the embeddings space includes a set of point coordinates representing the set of structured data in the embeddings space.
Hamilton teaches according to neighborhood sampling, wherein the embeddings space includes a set of point coordinates representing the set of structured data in the embeddings space (Page 3, "learned the parameters of K aggregator functions..., which aggregate information from node neighbors." Page 1, "Low-dimensional vector embeddings of nodes in large graphs." Node embeddings are vectors in continuous space (i.e., point coordinates)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the graph techniques of Hamilton to learn structural information about a node’s role in a graph (see Hamilton at page 2).
As to dependent claim 2, Hamilton further teaches the knowledge graph embedding generation circuitry is configured to convert the target triple to the embeddings space according to the neighborhood sampling, by: selecting K neighboring nodes of the target triple based on certainty scores of the neighboring nodes, wherein K is an integer larger than one (Page 3, "we assume that we have learned the parameters of K aggregator functions... which aggregate information from node neighbors"); generating an embedding vector for each of the K neighboring nodes by an encoder circuitry; and aggregating the K embedding vectors to obtain an aggregated embedding vector for the target triple (Algorithm 1).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the graph techniques of Hamilton to learn structural information about a node’s role in a graph (see Hamilton at page 2).
As to dependent claim 3, Hamilton further teaches selecting the K neighboring nodes of the target triple based on… the neighboring nodes, is carried out by: sampling the neighboring nodes of the target triple according to a sampling with replacement algorithm to obtain the K neighboring nodes… (Page 4, "sample a fixed-size set of neighbors." Algorithm 2).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the graph techniques of Hamilton to learn structural information about a node’s role in a graph (see Hamilton at page 2).
Gal further teaches based on certainty scores and highest certainty scores (Figure 2, predictive mean and uncertainties. The Monte Carlo dropout generates predictive mean and variance, i.e., uncertainty scores associated with predictions).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
As to dependent claim 4, Gal further teaches the encoder circuitry in the knowledge graph embedding generation circuitry comprises a neural network with dropouts after every weight layer (Page 4, "performing T stochastic forward passes through the network and averaging the results").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
As to dependent claim 5, Gal further teaches aggregating the K embedding vectors to obtain the aggregated embedding vector for the target triple is carried out by summarizing, averaging or taking the weighted average of the corresponding point coordinate of the K embedding vectors to determine each point of the aggregate embedding vector (Page 4, "sample variance of T stochastic forward passes through the NN plus the inverse model precision").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
As to independent claim 9, Kipf teaches
A method for quantifying certainty for a prediction based on a knowledge graph, the method comprising (Title and abstract):
receiving, by a device comprising a memory storing instructions and a processing circuitry in communication with the memory, a target triple and a knowledge graph comprising a set of structured data… (Page 6, "relation nodes r1 and r2 for each entity pair (e1; r; e2) as (e1; r1) and (e2; r2)." Page 1, "graph structure." Graph structured data representing relationships among entities);
converting, by the device, the target triple to an embeddings space… (Section A.2, semi-supervised node embeddings. Learns low-dimensional embeddings for graph nodes);
generating, by the device, a plausibility prediction for the target triple using a scoring function (Page 6, "GCN as described in Section 3.1 and evaluate prediction
accuracy." Outputs predictions (e.g., link existence probability));….
Kipf does not appear to expressly teach a set of certainty scores for the structured data; repeating, by the device, converting the target triple to the embedding space and generating another plausibility prediction for the target triple N times with dropouts to obtain N plausibility scores for the target triple, wherein N is an integer larger than one; and generating, by the device, a predicted plausibility score and a certainty score for the target triple based on the N plausibility scores, and outputting for display the predicted plausibility score and the certainty score.
Gal teaches a set of certainty scores for the structured data (Figure 2, predictive mean and uncertainties. The Monte Carlo dropout generates predictive mean and variance, i.e., uncertainty scores associated with predictions); repeating, by the device, converting the target triple to the embedding space and generating another plausibility prediction for the target triple N times with dropouts to obtain N plausibility scores for the target triple, wherein N is an integer larger than one (Page 4, "performing T stochastic forward passes." Section 3, dropout as a Bayesian approximation.); and generating, by the device, a predicted plausibility score and a certainty score for the target triple based on the N plausibility scores, and outputting for display the predicted plausibility score and the certainty score (Page 4, "performing T stochastic forward passes." Page 5, "standard deviation expresses the models’ uncertainty about the point.").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
Kipf does not appear to expressly teach according to neighborhood sampling by a neural network, wherein the embeddings space includes a set of point coordinates representing the set of structured data in the embeddings space.
Hamilton teaches according to neighborhood sampling by a neural network, wherein the embeddings space includes a set of point coordinates representing the set of structured data in the embeddings space (Page 3, "learned the parameters of K aggregator functions..., which aggregate information from node neighbors." Page 1, "Low-dimensional vector embeddings of nodes in large graphs." Node embeddings are vectors in continuous space (i.e., point coordinates)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the graph techniques of Hamilton to learn structural information about a node’s role in a graph (see Hamilton at page 2).
As to dependent claim 10, Hamilton further teaches converting the target triple to the embeddings space according to the neighborhood sampling comprises: selecting K neighboring nodes of the target triple based on certainty scores of the neighboring nodes, wherein K is an integer larger than one (Page 3, "we assume that we have learned the parameters of K aggregator functions... which aggregate information from node neighbors"); generating an embedding vector for each of the K neighboring nodes by an encoder; and aggregating the K embedding vectors to obtain an aggregated embedding vector for the target triple (Algorithm 1).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the graph techniques of Hamilton to learn structural information about a node’s role in a graph (see Hamilton at page 2).
As to dependent claim 11, Hamilton further teaches selecting the K neighboring nodes of the target triple based on certainty scores of the neighboring nodes comprises: sampling the neighboring nodes of the target triple according to a sampling with replacement algorithm to obtain the K neighboring nodes… (Page 4, "sample a fixed-size set of neighbors." Algorithm 2).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the graph techniques of Hamilton to learn structural information about a node’s role in a graph (see Hamilton at page 2).
Gal further teaches highest certainty scores (Figure 2, predictive mean and uncertainties. The Monte Carlo dropout generates predictive mean and variance, i.e., uncertainty scores associated with predictions).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
As to dependent claim 12, Gal further teaches the encoder in the knowledge graph embedding generation circuitry comprises a neural network with dropouts after every weight layer (Page 4, "performing T stochastic forward passes through the network and averaging the results").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
As to dependent claim 13, Gal further teaches aggregating the K embedding vectors to obtain the aggregated embedding vector for the target triple comprises: summarizing, averaging or taking the weighted average of the corresponding point coordinate of the K embedding vectors to determine each point of the aggregate embedding vector (Page 4, "sample variance of T stochastic forward passes through the NN plus the inverse model precision").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
As to independent claim 17, Kipf teaches
A non-transitory computer-readable storage medium storing computer-readable instructions, wherein, the computer-readable instructions, when executed by a processing circuitry, are configured to cause the processing circuitry to perform (Title and abstract):
receiving a target triple and a knowledge graph comprising a set of structured data… (Page 6, "relation nodes r1 and r2 for each entity pair (e1; r; e2) as (e1; r1) and (e2; r2)." Page 1, "graph structure." Graph structured data representing relationships among entities);
converting the target triple to an embeddings space… (Section A.2, semi-supervised node embeddings. Learns low-dimensional embeddings for graph nodes);
generating a plausibility prediction for the target triple using a scoring function (Page 6, "GCN as described in Section 3.1 and evaluate prediction accuracy." Outputs predictions (e.g., link existence probability));….
Kipf does not appear to expressly teach a set of certainty scores for the structured data; repeating converting the target triple to the embedding space and generating another plausibility prediction for the target triple N times with dropouts to obtain N plausibility scores for the target triple, wherein N is an integer larger than one; and generating a predicted plausibility score and a certainty score for the target triple based on the N plausibility scores, and outputting for display the predicted plausibility score and the certainty score.
Gal teaches a set of certainty scores for the structured data (Figure 2, predictive mean and uncertainties. The Monte Carlo dropout generates predictive mean and variance, i.e., uncertainty scores associated with predictions); repeating converting the target triple to the embedding space and generating another plausibility prediction for the target triple N times with dropouts to obtain N plausibility scores for the target triple, wherein N is an integer larger than one (Page 4, "performing T stochastic forward passes." Section 3, dropout as a Bayesian approximation.); and generating a predicted plausibility score and a certainty score for the target triple based on the N plausibility scores, and outputting for display the predicted plausibility score and the certainty score (Page 4, "performing T stochastic forward passes." Page 5, "standard deviation expresses the models’ uncertainty about the point.").
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
Kipf does not appear to expressly teach according to neighborhood sampling by a neural network, wherein the embeddings space includes a set of point coordinates representing the set of structured data in the embeddings space.
Hamilton teaches according to neighborhood sampling by a neural network, wherein the embeddings space includes a set of point coordinates representing the set of structured data in the embeddings space (Page 3, "learned the parameters of K aggregator functions..., which aggregate information from node neighbors." Page 1, "Low-dimensional vector embeddings of nodes in large graphs." Node embeddings are vectors in continuous space (i.e., point coordinates)).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the graph techniques of Hamilton to learn structural information about a node’s role in a graph (see Hamilton at page 2).
As to dependent claim 18, Hamilton further teaches converting the target triple to the embeddings space according to the neighborhood sampling, the computer-readable instructions are configured to cause the processing circuitry to perform: selecting K neighboring nodes of the target triple based on certainty scores of the neighboring nodes, wherein K is an integer larger than one (Page 3, "we assume that we have learned the parameters of K aggregator functions... which aggregate information from node neighbors"); generating an embedding vector for each of the K neighboring nodes by an encoder; and aggregating the K embedding vectors to obtain an aggregated embedding vector for the target triple (Algorithm 1).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the graph techniques of Hamilton to learn structural information about a node’s role in a graph (see Hamilton at page 2).
As to dependent claim 19, Hamilton further teaches selecting the K neighboring nodes of the target triple based on… the neighboring nodes, the computer-readable instructions are configured to cause the processing circuitry to perform: sampling the neighboring nodes of the target triple according to a sampling with replacement algorithm to obtain the K neighboring nodes… (Page 4, "sample a fixed-size set of neighbors." Algorithm 2).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the graph techniques of Hamilton to learn structural information about a node’s role in a graph (see Hamilton at page 2).
Gal further teaches based on certainty scores and with highest certainty scores (Figure 2, predictive mean and uncertainties. The Monte Carlo dropout generates predictive mean and variance, i.e., uncertainty scores associated with predictions).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having the graph convolutional networks of Kipf to include the uncertainty techniques of Gal to improve predictive log-likelihood and RMSE (see Gal at abstract).
Allowable Subject Matter
Claims 6-8, 14-16, and 20 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Citation of Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Rossiello et al. (U.S. Pat. App. Pub. No. 2022/0327356) teaches improving knowledge graph (KG) link prediction using transformer-based artificial neural networks. A first topic model is leveraged against a first dataset derived from a KG containing a plurality of first triples. The first triples include first entities and first edges connecting the first entities to represent relationships between the first connected entities. A first similarity function is applied to the first connected entities of the first triples to provide respective first similarity scores. A first subset of one of more first triples is selected from the plurality of first triples based upon the first similarity scores. An artificial neural network is trained using the selected first subset of one or more first triples.
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. A reference is relevant for all it contains and may be relied upon for all that it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 1331, 1332-33, 216 U.S.P.Q. 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 1006, 1009, 158 U.S.P.Q. 275, 277 (C.C.P.A. 1968)).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Casey R. Garner whose telephone number is 571-272-2467. The examiner can normally be reached Monday to Friday, 8am to 5pm, Eastern Time.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached on 571-270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/Casey R. Garner/Primary Examiner, Art Unit 2123