Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Regarding claim 11, the phrase "for example" renders the claim indefinite because it is unclear whether the limitation(s) following the phrase are part of the claimed invention. See MPEP § 2173.05(d).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-19 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
MPEP 2106 (III) sets out steps for evaluating whether a claim is drawn to patent-eligible subject matter. The analysis of claims 1-19, in accordance with these steps, follows.
Step 1 Analysis:
Claim 19 is directed to a method (process). Claims 1-18 are directed to a data processing apparatus (machine). Therefore, claims 1-19 fall into one of four statutory categories (i.e., process, machine).
As to claim 1,
Step 2A Prong 1: this claim recites the following abstract ideas:
determine variances associated with the variables ...; (the limitation describes calculating variance values associated with each of the variables, which is a mental process implemented using a pen and paper.)
determine an order of the variables based on the determined variances, including iteratively removing at least one node or variable represented by said at least one node thereby to determine the order. (the limitation describes devising an ordering of the variables by working through the calculated variance values and successively removing variables or nodes, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
a memory configured to store a trained model; and processing circuitry configured to; (This limitation is directed to mere instruction to store the abstract idea on a generic memory and apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
receive at least one dataset that comprises d variables and n samples; (this limitation describes data collection/receiving, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
by processing the dataset using the model; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 19,
Step 2A Prong 1: this claim recites the following abstract ideas:
determining variances associated with the variables ...; (the limitation describes calculating variance values associated with each of the variables, which is a mental process implemented using a pen and paper.)
determining an order of the variables based on the determined variances including iteratively removing variables and/or nodes thereby to determine the order. (the limitation describes devising an ordering of the variables by working through the calculated variance values and successively removing variables or nodes, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
storing a trained model; (Describes a generic computer function of saving data, see MPEP 2106.05(d)(II)(iv) Storing and retrieving information in memory.)
receiving at least one dataset that comprises d variables and n samples; (this limitation describes data collection/receiving, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
by processing the dataset using the model; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 2,
Step 2A Prong 1: this claim recites the following abstract ideas:
wherein the order of the variables represents an order of causation relationship(s), the variances comprise second order derivatives, and wherein a respective one of the variances is associated with each variable. (the limitation describes the content of the order and the variances being considered, which merely specifies the type of data evaluated and is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 3,
Step 2A Prong 1: this claim recites the following abstract ideas:
wherein the determining of the variances comprises determining a variance of a second order gradient of a distribution associated with the variables. (the limitation describes calculating a variance of a second order gradient of a distribution, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 4,
Step 2A Prong 1: this claim recites the following abstract ideas:
wherein the determining of the variances comprises ... to estimate the variances, which are second order derivatives of a distribution associated with the variables; (the limitation describes estimating variance values as second order derivatives of a distribution associated with the variables, which is a mental process implemented using a pen and paper.)
the determining of the order of the variables includes ... after each iterative removal of at least one node or variable to re-determine the variances; (the limitation describes re-calculating the variance values after each removal of a variable or node in order to determine the ordering, which is a mental process implemented using a pen and paper.)
each iterative removal of at least one node or variable is based on a score, and the scores for new distributions resulting from the iterative removals of at least one node or variable are determined based on the re-determined variances. (the limitation describes calculating scores from the re-calculated variance values and basing each removal on the calculated scores, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
applying a neural network, which is trained with denoising diffusion, to the dataset; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
re-applying the same neural network, without retraining, to the dataset; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 5,
Step 2A Prong 1: this claim recites the following abstract ideas:
select parent variable(s) for each variable from preceding variable(s) in the order, wherein the selecting comprises an inference procedure and is followed by a pruning procedure to remove incorrect causal relationships. (the limitation describes selecting parent variables for each variable from the preceding variables and removing incorrect causal relationships, which is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
wherein the processing circuity is configured to; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 6,
Step 2A Prong 1: this claim recites the following abstract ideas:
wherein the variances comprise variances of derivatives of a score in respect of each variable. (the limitation describes the content of the variances being considered, which merely specifies the type of data evaluated and is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 7,
Step 2A Prong 1: this claim recites the following abstract ideas:
wherein the score represents a gradient of a data distribution of the data set. (the limitation describes the content of the score being considered, which merely specifies the type of data evaluated and is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 8,
Step 2A Prong 1: this claim recites the following abstract ideas:
wherein the variances comprise or are represented by a Jacobian or Hessian. (the limitation describes using mathematical variables to represent data, which is a mental process implemented in the human mind.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 9,
Step 2A Prong 1: this claim recites the following abstract ideas:
determine the variances based on a score, wherein the score is calculated ...; (the limitation describes calculating a score and determining the variance values based on the calculated score, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
wherein the processing circuitry is further configured to; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
by processing the at least one dataset by the model; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 10,
Step 2A Prong 1: this claim recites the following abstract ideas:
wherein the score comprises, represents or is determined from gradients generated ...; (the limitation describes the content of the score as comprising, representing, or being determined from gradients, which merely specifies the type of data evaluated and is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
by processing the dataset by the model; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 11,
Step 2A Prong 1: this claim recites the following abstract ideas:
wherein the order of the variables comprises or is represented by an order of nodes of a causal graph, for example a directed acyclic graph (DAG). (the limitation describes representing the order of the variables as an ordering of nodes of a graph, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
As to claim 12,
Step 2A Prong 1: this claim recites the following abstract ideas:
determine a leaf node that corresponds to a peripheral one of the variables based on the score. (the limitation describes identifying a leaf node corresponding to a peripheral one of the variables based on the score, which is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
wherein the processing circuitry is further configured to; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 13,
Step 2A Prong 1: this claim recites the following abstract ideas:
determine an order subsequent to the leaf node by masking the leaf node and determining the variances without using variable(s) represented by the leaf node. (the limitation describes determining the subsequent positions in the order by crossing out the leaf node and re-calculating the variance values without the crossed-out variable, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
wherein the processing circuitry is further configured to; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 14,
Step 2A Prong 1: this claim recites the following abstract ideas:
remove at least one node or variable represented by said at least one node, and to re-determine the variances thereby to determine the next node or variable in the order. (the limitation describes removing a node or variable and re-calculating the variance values to determine the next node or variable in the order, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
wherein the processing circuity is configured to; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 15,
Step 2A Prong 1: this claim recites the following abstract ideas:
perform an iterative procedure that comprises removing successive variable(s) or node(s) and re-determining variances to determine the next one(s) of the variable(s) or node(s) in the order. (the limitation describes iteratively removing successive variables or nodes and re-calculating the variance values to determine the next variables or nodes in the order, which is a mental process implemented using a pen and paper.)
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
wherein the processing circuitry is configured to; (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 16,
Step 2A Prong 1: This claim does not recite an additional abstract idea, but the claim depends on claim 15.
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
wherein the re-determining of variances is performed using the same trained model. (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 17,
Step 2A Prong 1: This claim does not recite an additional abstract idea, but the claim depends on claim 1.
Step 2A Prong 2 and 2B: the claim recited the following additional elements:
wherein the model comprises at least one of a neural network, a generative model, a generative neural network, a diffusion model, a diffusion probabilistic model, a non-linear additive noise model. (This limitation is directed to mere instruction to apply the abstract idea on a generic computer to process, which is a well-understood, routine, conventional activity, see MPEP 2106.05(d)(II)(i))
The additional element does not integrate the judicial exception into practical application and does not amount to significantly more than the Judicial exception.
As to claim 18,
Step 2A Prong 1: this claim recites the following abstract ideas:
wherein the data set comprises data that includes or represents at least one of: drug dosages given to a patient or other subject; physiological or other measurements performed on the patient; one or more of blood pressure, temperature, heart rate, blood oxygenation, electrocardiograph or other electrical measurements; vision or hearing-related measurements; measurements of any of a patient's or other subject's senses or reactions or any other measurements; or at least one of age, height, weight or other patient data; data relating to an imaging or other procedure. (the limitation describes the content of the data being considered, which merely specifies the type of data evaluated and is an evaluation and judgment activity that can be performed as a mental process in the human mind.)
Step 2A Prong 2 and 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B. Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) 1.), failing step 2A prong 2. The claim is ineligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-17 and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rolland et al. (SCORE MATCHING ENABLES CAUSAL DISCOVERY OF NONLINEAR ADDITIVE NOISE MODELS, 8 Mar 2022) in view of Kreis et al. (US 20230109379 A1).
As to claim 1, Rolland teaches a data processing apparatus comprising:
receive at least one dataset that comprises d variables and n samples; (see Rolland Section [2.2] "The goal of score matching is to learn the score function s(x) ≡ ∇ log p(x) of a distribution with density p(x) given an i.i.d. sample {x^k}_(k=1,...,n).", and see Rolland Section [3.1] "Algorithm 1 ... Input: Data matrix X ∈ ℝ^(n×d).")
Examiner note: the data matrix X ∈ ℝ^(n×d) is a dataset of d variables observed over n samples.
determine variances associated with the variables by processing the dataset using the model; (see Rolland Section [3.1] "This hence provides a way to identify a leaf of the causal graph from the knowledge of the variance of the score's Jacobian diagonal elements.", and see Rolland Section [3.1] Algorithm 1 "Estimate the score function s_nodes = ∇ log p_nodes (for example using Algorithm 1). Estimate V_j = Var_(X_nodes) [∂s_j(X)/∂x_j].")
determine an order of the variables based on the determined variances, including iteratively removing at least one node or variable represented by said at least one node thereby to determine the order. (see Rolland Section [3.1] "By repeating this method and always removing the identified leaves, we can estimate a full topological order." and see Rolland section [3.1] "Algorithm 1 ... l ← nodes[arg min_j V_j]; π ← [l, π]; nodes ← nodes − {l}; Remove l-th column of X")
Rolland does not explicitly teach "a memory configured to store a trained model; and", and "processing circuitry configured to"
However, Kreis teaches
a memory configured to store a trained model; and(see Kreis paragraph [0058] "inference and/or training logic 815 may include, without limitation, code and/or data storage 801 to store forward and/or output weight and/or input/output data, and/or other parameters to configure neurons or layers of a neural network trained and/or used for inferencing in aspects of one or more embodiments.")
processing circuitry configured to: (see Kreis paragraph [0081] "computer system 1000 may include, without limitation, processor 1002 that may include, without limitation, one or more execution units 1008 to perform machine learning model training and/or inferencing according to techniques described herein...")
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the causal discovery method of Rolland by implementing it on the apparatus of Kreis comprising a memory storing a trained neural network score model and processing circuitry to perform machine learning inferencing, because Kreis teaches that a neural network can be utilized to learn the score function, i.e., the gradient of the log probability density, of the data (see Kreis paragraph [0027]) and that such a generative model, once trained, can advantageously be reused for a variety of different applications (see Kreis paragraph [0024]), and one of ordinary skill would have been motivated to make this combination to obtain an efficient and scalable implementation of Rolland's score-based causal ordering, with a reasonable expectation of success and predictable results.
As to claim 2, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
wherein the order of the variables represents an order of causation relationship(s), (see Rolland Section [3.1] "This hence provides a way to identify a leaf of the causal graph from the knowledge of the variance of the score's Jacobian diagonal elements. By repeating this method and always removing the identified leaves, we can estimate a full topological order.")
Examiner note: a topological order of the causal graph's nodes is an ordering of the causation relationships among the variables.
the variances comprise second order derivatives, and (see Rolland Section [3.2] "in order to identify a leaf of the causal graph, we are only interested in estimating the diagonal elements of the score's Jacobian at the sample points, i.e., J ≡ (diag(∇² log p(x¹)), . . . , diag(∇² log p(xⁿ)))ᵀ ∈ ℝ^(n×d).")
Examiner note: the quantities whose variance is computed are the diagonal elements of ∇² log p, which are second-order derivatives.
wherein a respective one of the variances is associated with each variable. (see Rolland Section [3.1], "Algorithm 1 ... Estimate V_j = Var_(X_nodes) [∂s_j(X)/∂x_j].")
Examiner note: a variance V_j is computed for each variable index j.
As to claim 3, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
wherein the determining of the variances comprises determining a variance of a second order gradient of a distribution associated with the variables. (see Rolland Section [3.1] "Lemma 1 ... j is a leaf ⇔ ∀x, ∂s_j(x)/∂x_j = c, with c ∈ ℝ independent of x, i.e., Var_X [∂s_j(X)/∂x_j] = 0.")
As to claim 4, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
to estimate the variances, which are second order derivatives of a distribution associated with the variables, (see Rolland Section [3.2] "what we need for identifying the graph is an estimate of the Jacobian of the score at all samples, in order to estimate its variance. … J ≡ (diag(∇² log p(x¹)), . . . , diag(∇² log p(xⁿ)))ᵀ ∈ ℝ^(n×d).")
each iterative removal of at least one node or variable is based on a score, and (see Rolland Section [3.1] "Algorithm 1... Estimate the score function s_nodes = ∇ log p_nodes . . . l ← nodes[arg min_j V_j] . . . nodes ← nodes − {l}; Remove l-th column of X")
Examiner note: the removed node l is selected from the variance of the score's Jacobian diagonal, so each removal is based on the score.
the scores for new distributions resulting from the iterative removals of at least one node or variable are determined based on the re-determined variances. (see Rolland Section [3.1] "Algorithm 1 for k = 1, . . . , d do Estimate the score function s_nodes = ∇ log p_nodes . . . Estimate V_j = Var_(X_nodes) [∂s_j(X)/∂x_j]. l ← nodes[arg min_j V_j] . . . nodes ← nodes − {l}; Remove l-th column of X end for" and see Section [3.2] "Note that, when using Algorithm 2 for causal discovery in Algorithm 1, the kernel lengthscale is recomputed each time a node is removed from the data matrix X.")
Rolland does not explicitly teach "wherein the determining of the variances comprises applying a neural network, which is trained with denoising diffusion, to the dataset", and "the determining of the order of the variables includes re-applying the same neural network, without retraining, to the dataset after each iterative removal of at least one node or variable to re-determine the variances"
However, Kreis teaches
wherein the determining of the variances comprises applying a neural network, which is trained with denoising diffusion, to the dataset (see Kreis paragraph [0027] "SGMs use a diffusion process to gradually or iteratively add noise to input data, transforming a complex data distribution to an analytically tractable prior distribution. A neural network can be utilized to learn the score function—such as, in example non-limiting embodiments, the gradient of the log probability density—of the perturbed data.")
the determining of the order of the variables includes re-applying the same neural network, without retraining, to the dataset after each iterative removal of at least one node or variable to re-determine the variances, (see Kreis paragraph [0024] "Such a generative model, once trained, can advantageously generate content for a variety of different applications and use cases.", and see paragraph [0027] "These learnt scores can be used to solve a stochastic differential equation (SDE) to synthesize one or more new samples. This corresponds to an iterative denoising process, effectively inverting the forward diffusion process.", and see paragraph [0048] "If at least one such end criterion has been satisfied then this trained score-based generative model can be provided 520 for synthesizing new image content.")
As to claim 5, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
wherein the processing circuity is configured to select parent variable(s) for each variable from preceding variable(s) in the order, (see Rolland Section [3.1] "Algorithm 1 ... Get the final DAG by pruning the full DAG associated with the topological order π.")
wherein the selecting comprises an inference procedure and is followed by a pruning procedure to remove incorrect causal relationships. (see Rolland Section 4.1 "For all order-based causal discovery methods, we always apply the same pruning procedure, i.e., CAM with the same cutoff parameter of 0.001.")
As to claim 6, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
wherein the variances comprise variances of derivatives of a score in respect of each variable. (see Rolland Section [3.1] "Algorithm 1 ... Estimate V_j = Var_(X_nodes) [∂s_j(X)/∂x_j].")
As to claim 7, Rolland as modified by Kreis teaches the data processing apparatus according to claim 6,
wherein the score represents a gradient of a data distribution of the data set. (see Rolland Section [2.2] "The goal of score matching is to learn the score function s(x) ≡ ∇ log p(x) of a distribution with density p(x) given an i.i.d. sample {x^k}_(k=1,...,n).")
As to claim 8, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
wherein the variances comprise or are represented by a Jacobian or Hessian. (see Rolland Section [3.2] "what we need for identifying the graph is an estimate of the Jacobian of the score at all samples, in order to estimate its variance.")
As to claim 9, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
wherein the processing circuitry is further configured to determine the variances based on a score, wherein the score is calculated by processing the at least one dataset by the model. (see Rolland Section [2.2] "we present a method developed in [20] for estimating the score at the sample points, i.e., approximating G ≡ (∇ log p(x¹), . . . , ∇ log p(xⁿ))ᵀ ∈ ℝ^(n×d) .... Hence, by using ridge regression, the Stein gradient estimator is defined as: Ĝ^Stein ≡ arg min . . .")
As to claim 10, Rolland as modified by Kreis teaches the data processing apparatus according to claim 9,
wherein the score comprises, represents or is determined from gradients generated by processing the dataset by the model. (see Rolland Section [2.2] "approximating G ≡ (∇ log p(x¹), . . . , ∇ log p(xⁿ))ᵀ ∈ ℝ^(n×d).")
Examiner note: the estimated score G is a matrix of gradients (∇ log p) generated by processing the sample data through the estimator.
As to claim 11, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
wherein the order of the variables comprises or is represented by an order of nodes of a causal graph, for example a directed acyclic graph (DAG). (see Rolland Section [2.1] "This model is known to be identifiable from observational data [3], meaning that it is possible to recover the DAG underlying the generative model (1) from the knowledge of the joint probability distribution of X.", and see Section [3.1] Discussion "By repeating this method and always removing the identified leaves, we can estimate a full topological order.")
As to claim 12, Rolland as modified by Kreis teaches the data processing apparatus according to claim 9,
wherein the processing circuitry is further configured to determine a leaf node that corresponds to a peripheral one of the variables based on the score. (see Rolland Section [3.1] "Lemma 1 shows that, for non-linear additive Gaussian noise models, leaf nodes (and only leaf nodes) have the property that the associated diagonal element in the score's Jacobian is a constant. This hence provides a way to identify a leaf of the causal graph from the knowledge of the variance of the score's Jacobian diagonal elements.")
As to claim 13, Rolland as modified by Kreis teaches the data processing apparatus according to claim 12,
wherein the processing circuitry is further configured to determine an order subsequent to the leaf node by masking the leaf node and determining the variances without using variable(s) represented by the leaf node. (see Rolland Section 3.1 "Algorithm 1 ... nodes ← nodes − {l}; Remove l-th column of X", and see Section [3.2] "Note that, when using Algorithm 2 for causal discovery in Algorithm 1, the kernel lengthscale is recomputed each time a node is removed from the data matrix X.")
As to claim 14, Rolland as modified by Kreis teaches the data processing apparatus according to claim 11,
wherein the processing circuity is configured to remove at least one node or variable represented by said at least one node, and to re-determine the variances thereby to determine the next node or variable in the order. (see Rolland Section [3.1] "Algorithm 1 ... for k = 1, . . . , d do Estimate the score function s_nodes = ∇ log p_nodes . . . Estimate V_j = Var_(X_nodes) [∂s_j(X)/∂x_j]. l ← nodes[arg min_j V_j]; π ← [l, π]; nodes ← nodes − {l}; Remove l-th column of X end for")
As to claim 15, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
wherein the processing circuitry is configured to perform an iterative procedure that comprises removing successive variable(s) or node(s) and re-determining variances to determine the next one(s) of the variable(s) or node(s) in the order. (see Rolland Section [3.1] "By repeating this method and always removing the identified leaves, we can estimate a full topological order. This procedure is summarized in Algorithm 1.")
As to claim 16, Rolland as modified by Kreis teaches the data processing apparatus according to claim 15,
Rolland does not explicitly teach "wherein the re-determining of variances is performed using the same trained model"
However, Kreis teaches
wherein the re-determining of variances is performed using the same trained model. (see Kreis paragraph [0024] "Such a generative model, once trained, can advantageously generate content for a variety of different applications and use cases." and paragraph [0048] "If at least one such end criterion has been satisfied then this trained score-based generative model can be provided 520 for synthesizing new image content.")
As to claim 17, Rolland as modified by Kreis teaches the data processing apparatus according to claim 1,
wherein the model comprises at least one of a neural network, a generative model, a generative neural network, a diffusion model, a diffusion probabilistic model, a non-linear additive noise model. (see Kreis paragraph [0027] "An SGM 102 or denoising diffusion probabilistic model can be used for a variety of generative and synthesis operations . . . A neural network can be utilized to learn the score function—such as, in example non-limiting embodiments, the gradient of the log probability density—of the perturbed data.")
As to claim 19, this is directed to an apparatus that corresponds to the method of claim 1, See the rejection for claim 1 above, which also applies to claim 19.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rolland et al. (SCORE MATCHING ENABLES CAUSAL DISCOVERY OF NONLINEAR ADDITIVE NOISE MODELS, 8 Mar 2022) in view of Kreis et al. (US 20230109379 A1) and Weissbrod et al. (US 20190096102 A1).
As to claim 18, Rolland-kreis teaches the data processing apparatus according to claim 1,
Rolland does not explicitly teaches "wherein the data set comprises data that includes or represents at least one of: drug dosages given to a patient or other subject; physiological or other measurements performed on the patient; one or more of blood pressure, temperature, heart rate, blood oxygenation, electrocardiograph or other electrical measurements; vision or hearing-related measurements; measurements of any of a patient's or other subject's senses or reactions or any other measurements; or at least one of age, height, weight or other patient data; data relating to an imaging or other procedure."
However, Weissbrod teaches
wherein the data set comprises data that includes or represents at least one of: drug dosages given to a patient or other subject; physiological or other measurements performed on the patient; one or more of blood pressure, temperature, heart rate, blood oxygenation, electrocardiograph or other electrical measurements; vision or hearing-related measurements; measurements of any of a patient's or other subject's senses or reactions or any other measurements; or at least one of age, height, weight or other patient data; data relating to an imaging or other procedure. (see Weissbrod paragraph [0008] "the analysis of modern datum, such as electronic health records, requires an understanding of the causal relations between relevant variables (in the case of electronic health records, e.g., age, sex, lab measurements, smoking status, etc.)." and paragraph [0013] "the effect of various diabetes second line drugs (i.e., a treatment) on levels of HbA1c (i.e., an outcome) using electronic health records.")
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify the causal discovery apparatus of Rolland as modified by Kreis by applying it to a dataset comprising patient data such as age, lab measurements, and drug treatments from electronic health records as taught by Weissbrod, because Weissbrod teaches that the analysis of electronic health records requires an understanding of the causal relations between relevant variables and that such data sets may contain thousands of variables making it virtually impossible for human experts to build a causal graph manually (see Weissbrod paragraph [0008]), and one of ordinary skill would have been motivated to make this combination to enable causal analysis of patient health data, with a reasonable expectation of success and predictable results.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDULLAH K ABOUD whose telephone number is (571)272-0025. The examiner can normally be reached Mon-Fri 8am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Li B Zhen, can be reached at (571) 272-3768. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDULLAH KHALED ABOUD/ Examiner, Art Unit 2121
/Li B. Zhen/ Supervisory Patent Examiner, Art Unit 2121
Prosecution Insights