Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1, 6-7, and 11-12 are objected to because of the following informalities:
In claim 1, line 4, “given distribution” should recite “a given distribution”.
In claim 1, line 9, “original instance” should recite “an original instance”.
In claim 1, lines 22 and 26 should recite “and” after the semicolon.
In claim 6, line 5, “associated to decisions” should recite “associated with decisions”.
In claim 6, line 6 should recite “and” after the comma.
In claim 7, line 3, “ASV” should recite “asymmetric Shapley values (ASV)”.
In claim 11, line 4, “given distribution” should recite “a given distribution”.
In claim 11, line 7, “original instance” should recite “an original instance”.
In claim 11 on page 4, line 1 should recite “and” after the semicolon.
In claim 12, line 2, the limitation starting with a dash after “comprising” should be on a separate line.
In claim 12, line 20 should recite “and” after the semicolon. On page 5, line 1 should recite “and” after the semicolon.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 10, and 12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
In claim 2, the opening parenthesis in line 4 renders the claim indefinite because there is no corresponding closing parenthesis in the claim. It is unclear if the opening parenthesis is a typographical error. Examiner treats claim 2 as if there were no opening parenthesis.
Claim 10 is rendered indefinite because the list of items starting with “medical predictions” is missing a conjunction such as “and” or “or”. It is unclear whether the end of the claim should recite “survey feedback analysis, or email spam classification.” Examiner treats claim 10 as if it had recited “survey feedback analysis, or email spam classification.”
Claim 12 is rendered indefinite for the following reasons. In claim 12, lines 2-3 recite “receive a training dataset of given distribution”. It is unclear if this training dataset is the same as the training dataset recited in claim 11, line 4. Examiner treats “receive a training dataset of given distribution” as “receive said training dataset of the given distribution”.
In claim 12, the limitations in line 5 to page 5, line 1 recites the same steps as those recited in claim 11, from line 5 to the end of the claim. It is unclear if these limitations in claim 12 means performing the steps from claim 11 for a first time or for a second time. Examiner treats claim 12, from line 5 to page 5, line 1 as performing the steps from claim 11 for a first time.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-13 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claims 1-10 recite a device comprising a processor (a system), claims 11-12 recite a method, and claim 13 recites a non-transitory program storage device (a product). A system, a method, and a product each falls within at least one statutory category of patent-eligible subject matter.
Claim 1
Step 2A Prong 1: Providing a counterfactual explanation of an original decision the human mind with the aid of pencil and paper. Specification paragraphs [0050]-[0052] disclose a counterfactual explanation E may be a set of input variables such as a synthetic data point or E may be a set of features of the model which most influence the decision outputted by the model. A person can reasonably determine a synthetic datapoint or the most influential set of features.
Determine at least one sub-dataset sampled from a so-called prior dataset, said prior dataset being sampled from said given distribution is a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper and a mathematical calculation. Determining a sub-dataset amounts to selecting data from the prior dataset. In specification paragraph [0119], the feature x1 is drawn from a continuous uniform distribution, and the feature x2 is based on values drawn from the same continuous uniform distribution.
Compute, for each among said at least one sub-dataset, at least one corresponding set of Shapley values relating to said model by using at least one Shapley values computing method and based on said at least one sub-dataset and said causal knowledge, so as to obtain at least one computation environment, said at least one computation environment comprising one among said at least one sub-dataset and one among said at least one Shapley values computing method is a mathematical calculation. Specification paragraphs [0078]-[0081] disclose calculating Shapley values for an ith feature of a ML model receiving as input N features. An ith feature is a sub-dataset and a computation environment includes the ith feature and the corresponding Shapley value.
Determine one optimal computation environment among said at least one computation environment by means of at least one metric representative of an ability of the corresponding computation environment to provide counterfactual explanations is a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper, and a metric is a mathematical calculation. Specification paragraph [0095] discloses an equation for a counterfactual ability metric, and [0098] discloses determining the optimal computation environment corresponds to selecting the computation environment having the best success rate.
Determine said counterfactual explanation in the shape of a counterfactual instance or of a set of contributing features by means of the Shapley values corresponding to said original instance within said optimal computation environment and referred to as optimal Shapley values is a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper. The claim recites an abstract idea.
Step 2A Prong 2: An automated decision-making system amounts to a generic computer component for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
At least one input interface configured to receive a training dataset of given distribution amounts to an insignificant extra-solution activity under MPEP 2106.05(g).
At least one processor amounts to a generic computer component for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Train said model based on said training dataset so as to obtain a trained model amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
Obtain said original decision from an implementation of said trained model receiving as input an instance referred to as original instance amounts to invoking computers merely as a tool to perform an existing process under MPEP 2106.05(f).
Obtain causal knowledge relating to said trained model amounts to mere data-gathering, an insignificant extra-solution activity under MPEP 2106.05(g).
At least one output interface configured to output said counterfactual explanation amounts to an insignificant extra-solution activity under MPEP 2106.05(g).
The additional elements as disclosed above, alone or in combination, do not integrate the abstract ideas into a practical application as they are mere insignificant extra solution activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is directed to an abstract idea.
Step 2B: An automated decision-making system amounts to a generic computer component for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
At least one input interface configured to receive a training dataset of given distribution is analogous to presenting offers and gathering statistics, which is a well-understood, routine, and conventional activity recognized by the courts under MPEP 2106.05(d)(II).
At least one processor amounts to a generic computer component for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Train said model based on said training dataset so as to obtain a trained model amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
Obtain said original decision from an implementation of said trained model receiving as input an instance referred to as original instance amounts to invoking computers merely as a tool to perform an existing process under MPEP 2106.05(f).
Obtain causal knowledge relating to said trained model is analogous to receiving data over a network, which is a well-understood, routine, and conventional activity recognized by the courts under MPEP 2106.05(d)(II).
At least one output interface configured to output said counterfactual explanation is analogous to presenting offers and gathering statistics, which is a well-understood, routine, and conventional activity recognized by the courts under MPEP 2106.05(d)(II).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the abstract ideas as they are well-understood, routine and conventional activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is not patent eligible.
Claim 2 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated. Determine said counterfactual explanation by means of a minimum value search guided by a vector comprising said optimal Shapley values under a constraint, said constraint imposing that a decision outputted by said trained model for an instance obtained from a displacement from said original instance along a direction defined by said vector is different from said original decision is a mathematical calculation. Specification paragraphs [0095]-[0096], [0098] disclose a minimum value search.
Step 2A Prong 2 and Step 2B: At least one processor amounts to a generic computer component for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent-eligible.
Claim 3 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated. Said causal knowledge obtained
Step 2A Prong 2 and Step 2B: At least one processor amounts to a generic computer component for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim 4 incorporates the rejection of claim 3.
Step 2A Prong 1: The abstract ideas of claim 3 are incorporated.
Step 2A Prong 2 and Step 2B: Said causal knowledge is a causal graph amounts to a mere field of use and technological environment under MPEP 2106.05(h). The claim is not patent-eligible.
Claim 5 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated.
Step 2A Prong 2: Said causal knowledge obtained by said at least one processor comprises knowledge from at least one expert and is received via said at least one input interface amounts to insignificant extra-solution activity under MPEP 2106.05(g).
Step 2B: Said causal knowledge obtained by said at least one processor comprises knowledge from at least one expert and is received via said at least one input interface is analogous to presenting offers and gathering statistics, which is a well-understood, routine, and conventional activity recognized by the courts under MPEP 2106.05(d)(II). The claim is not patent-eligible.
Claim 6 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated. The at least one sub-dataset being randomly sampled from said prior dataset and belonging to one among: a plurality of instances comprising a label corresponding to a decision different from said original decision, a plurality of instances associated to decisions inferred by said model, said inferred decisions being different from said original decision, and a plurality of instances among the k closest neighbors of said original instance according to a predetermined distance metric, said plurality of instances being associated to decisions inferred by said model, said inferred decisions being different from said original decision is a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper. A person can reasonably select items from said prior dataset at random.
Step 2A Prong 2 and Step 2B: The claim does not recite any additional elements which, alone or in combination, would integrate the abstract ideas into a practical application or which, in combination with the abstract ideas, would be sufficient to amount to significantly more than the abstract ideas. The claim is not patent eligible.
Claim 7 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated. For said at least one sub-dataset, said set of Shapley values is computed by means of at least one heuristic chosen among Shapley Flow, ASV, Powerset Shapley values and sampling Shapley values are mathematical calculations. Specification paragraphs [0075]-[0076], [0079]-[0083] disclose equations for these heuristics.
Step 2A Prong 2 and Step 2B: The claim does not recite any additional elements which, alone or in combination, would integrate the abstract ideas into a practical application or which, in combination with the abstract ideas, would be sufficient to amount to significantly more than the abstract ideas. The claim is not patent eligible.
Claim 8 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated. Assign input data points to one of multiple classes, said multiple classes being at least three mutually exclusive classes amounts to a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper.
Step 2A Prong 2 and Step 2B: Said model is a multi-class classifier amounts to mere instructions to apply the abstract ideas under MPEP 2106.05(f). The claim is not patent eligible.
Claim 9 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated.
Step 2A Prong 2 and Step 2B: Said model is a gradient boosting Tree model amounts to a mere field of use and technological environment under MPEP 2106.05(h). The claim is not patent eligible.
Claim 10 incorporates the rejection of claim 1.
Step 2A Prong 1: The abstract ideas of claim 1 are incorporated.
Step 2A Prong 2 and Step 2B: Provide decisions relating to medical predictions, industrial quality inspection, banking, fraud detection, statistics, survey feedback analysis, email spam classification amounts to a mere field of use and technological environment under MPEP 2106.05(h). The claim is not patent eligible.
Claim 11
Step 2A Prong 1: Providing a counterfactual explanation of an original decision
Determining at least one sub-dataset sampled from a so-called prior dataset sampled from said given distribution is a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper and a mathematical calculation. Determining a sub-dataset amounts to selecting data from the prior dataset. In specification paragraph [0119], the feature x1 is drawn from a continuous uniform distribution, and the feature x2 is based on values drawn from the same continuous uniform distribution.
Computing, for each among said at least one sub-dataset, at least one corresponding set of Shapley values relating to said model by using at least one Shapley values computing method and based on said at least one sub-dataset and said causal knowledge, so as to obtain at least one computation environment, said at least one computation environment comprising one among said at least one sub-dataset and one among said at least one Shapley values computing method is a mathematical calculation. Specification paragraphs [0078]-[0081] disclose calculating Shapley values for an ith feature of a ML model receiving as input N features. An ith feature is a sub-dataset and a computation environment includes the ith feature and the corresponding Shapley value.
Determining one optimal computation environment among said at least one computation environment by means of at least one metric representative of an ability of the corresponding computation environment to provide counterfactual explanations is a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper, and a metric is a mathematical calculation. Specification paragraph [0095] discloses an equation for a counterfactual ability metric, and [0098] discloses determining the optimal computation environment corresponds to selecting the computation environment having the best success rate.
Determining said counterfactual explanation in the shape of a counterfactual instance or of a set of contributing features by means of the Shapley values corresponding to said original instance within said optimal computation environment and referred to as optimal Shapley values is a judgement mental process which can reasonably be performed in the human mind with the aid of pencil and paper. The claim recites an abstract idea.
Step 2A Prong 2: An automated decision-making system amounts to a generic computer component for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Receiving a training dataset of given distribution amounts to an insignificant extra-solution activity under MPEP 2106.05(g).
Training said model based on said training dataset so as to obtain a trained model amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
Obtaining said original decision from an implementation of said trained model receiving as input an instance referred to as original instance amounts to invoking computers merely as a tool to perform an existing process under MPEP 2106.05(f).
Obtaining causal knowledge relating to said model amounts to mere data-gathering, an insignificant extra-solution activity under MPEP 2106.05(g).
The additional elements as disclosed above, alone or in combination, do not integrate the abstract ideas into a practical application as they are mere insignificant extra solution activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is directed to an abstract idea.
Step 2B: An automated decision-making system amounts to a generic computer component for applying the abstract ideas on a generic computer under MPEP 2106.05(f).
Receiving a training dataset of given distribution is analogous to receiving data over a network, which is a well-understood, routine, and conventional activity recognized by the courts under MPEP 2106.05(d)(II).
Training said model based on said training dataset so as to obtain a trained model amounts to mere instructions to apply the abstract ideas on a generic computer under MPEP 2106.05(f).
Obtaining said original decision from an implementation of said trained model receiving as input an instance referred to as original instance amounts to invoking computers merely as a tool to perform an existing process under MPEP 2106.05(f).
Obtaining causal knowledge relating to said model is analogous to receiving data over a network, which is a well-understood, routine, and conventional activity recognized by the courts under MPEP 2106.05(d)(II).
The additional elements as disclosed above, in combination with the abstract ideas, are not sufficient to amount to significantly more than the abstract ideas as they are well-understood, routine and conventional activities as disclosed in combination with generic computer functions that are implemented to perform the abstract ideas disclosed above. The claim is not patent eligible.
Claim 12 recites a method which implements the same features as the system of claim 1 and is therefore rejected for at least the same reasons.
Claim 13 recites a product which implements the same features as the method of claim 11 and is therefore rejected for at least the same reasons.
In Step 2A Prong 2 and Step 2B, a non-transitory program storage device, readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method amounts to generic computer components for applying the abstract ideas on a generic computer under MPEP 2106.05(f). The claim is not patent eligible.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-4 and 6-13 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Albini et al. (“Counterfactual Shapley Additive Explanations”).
Regarding claim 1, Albini teaches: A device for providing a counterfactual explanation of an original decision from an automated decision-making system based on a model of classifier type for decision optimization, comprising: (Page 1054, Abstract, and page 1066, col. 1, § A.2, lines 1-3 discloses a device for providing a counterfactual explanation)
at least one input interface configured to receive a training dataset of given distribution; (Page 1055, col. 1, § 2 from line 1 to col. 2, line 8; page 1061, col. 1, lines 10-12; and page 1066, col. 1, § A.1, lines 1-7 discloses running experiments using publicly available datasets, and splitting each dataset into train and test data. Obtaining the publicly available datasets implies an input interface for receiving them. A “given distribution” is a distribution of the training dataset.)
at least one processor configured to: (Page 1066, col. 1, § A.2, lines 1-3)
train said model based on said training dataset so as to obtain a trained model; (Page 1055, col. 1, the paragraph above § 2 states, “We note that in this paper we concentrate on tree-based models”; § 2 from line 1 to col. 2, line 8; and page 1066, col. 1, § A.1, lines 1-7 discloses training a model.)
obtain said original decision from an implementation of said trained model receiving as input an instance referred to as original instance; (On page 1055, § 2 from line 1 to col. 2, line 2 discloses obtaining a model output f(x) and a model prediction F(x) which will generate an original decision after training the model. Page 1066, col. 1, § A.1, lines 1-7 discloses splitting each dataset into train and test data. The test data includes an “original instance”, and the trained model’s prediction is “said original decision”.)
obtain causal knowledge relating to said trained model; (Causal knowledge includes trained parameters of the tree-based models “f” because parameters have been learned and they cause the model to generate a model output. See page 1055, col. 1, the paragraph above § 2 which states, “We note that in this paper we concentrate on tree-based models”; and page 1066, col. 1, § A.1, lines 1-7. Obtaining causal knowledge could also mean discovering most important features as disclosed in § 2.1 on page 1055.)
determine at least one sub-dataset sampled from a so-called prior dataset, said prior dataset being sampled from said given distribution; (Page 1056, § 3.1, from line 1 to “Differently-labelled samples” discloses generating samples in the training set labelled differently than the input. At least one sub-dataset is an individual sample. The differently-labeled samples are sampled from the training set and thus sampled from “said given distribution”.)
compute, for each among said at least one sub-dataset, at least one corresponding set of Shapley values relating to said model by using at least one Shapley values computing method and based on said at least one sub-dataset and said causal knowledge, so as to obtain at least one computation environment, said at least one computation environment comprising one among said at least one sub-dataset and one among said at least one Shapley values computing method; (Page 1057, col. 1, from line 4 through “Definition 3.1”, where a set of Shapley values are values v(S), a sub-dataset is the background dataset, a Shapley values computing method is the characteristic function for v(S) including C (the name of the counterfactual technique), and a computing environment includes one calculation of v(S). The Shapley values are further based on trained model parameters of the model “f”.)
determine one optimal computation environment among said at least one computation environment by means of at least one metric representative of an ability of the corresponding computation environment to provide counterfactual explanations; (Page 1057, col. 2, start of § 3.3 to page 1058, col. 1, line 5; page 1058, col. 1, § 4.1, lines 1-11; and page 1058, col. 2, subsection “Induced Counterfactual and Counterfactual-Ability” from line 1 through Definition 4.1. A “metric” corresponds to the counterfactual-ability CF, and an “optimal computation environment” is the induced counterfactual for an explanation.)
determine said counterfactual explanation in the shape of a counterfactual instance or of a set of contributing features by means of the Shapley values corresponding to said original instance within said optimal computation environment and referred to as optimal Shapley values; (Page 1057, col. 2, start of § 3.3 to page 1058, col. 1, line 5; page 1058, col. 1, § 4.1, lines 1-11; and page 1058, col. 2, subsection “Induced Counterfactual and Counterfactual-Ability” from line 1 through Definition 4.1. A “counterfactual instance” is an induced counterfactual x’.)
at least one output interface configured to output said counterfactual explanation. (Page 1056, all of Fig. 1 and its caption discloses that using a set of counterfactuals (iv) a more actionable explanation is obtained. Box (iv) outputs an explanation – moving from point x towards a combination of Features A and B in the direction of the arrow. In Page 1058, col. 2, subsection “Induced Counterfactual and Counterfactual-Ability” from line 1 through Definition 4.1, optimizing the counterfactual-ability outputs the induced counterfactual.)
Regarding claim 2, Albini teaches: The device according to claim 1, wherein said at least one processor is configured to determine said counterfactual explanation by means of a minimum value search guided by a vector comprising said optimal Shapley values under a constraint, said constraint imposing that a decision outputted by said trained model for an instance obtained from a displacement from said original instance along a direction defined by said vector is different from said original decision. (Page 1058, col. 1, § 4.1, lines 1-11 discloses finding a counterfactual point towards which the user will tend to move with minimum cost for the user. A “minimum value search” is a cost to a user. A “vector” is a change from an original input x (resulting an original model decision) to an induced counterfactual x’ (resulting in a different model decision). “Optimal Shapley values” are the Shapley values ϕ which correspond to x’.)
Regarding claim 3, Albini teaches: The device according to claim 1, wherein said causal knowledge obtained by said at least one processor is computed by said at least one processor by means of a causal discovery method. (Page 1055, col. 1, the paragraph above § 2 which states, “We note that in this paper we concentrate on tree-based models”; and page 1066, col. 1, § A.1, lines 1-7 teaches training a model for each dataset. A “causal discovery method” is the training process to discover trained model parameters. On page 1067, in Table 2, the right 3 columns disclose performance of a model after training on different datasets.)
Regarding claim 4, Albini teaches: The device according to claim 3, wherein said causal knowledge is a causal graph. (Page 1055, col. 1, the paragraph above § 2 which states, “We note that in this paper we concentrate on tree-based models”; and page 1066, col. 1, § A.1, lines 1-7 teaches training a tree-based model. Since a tree is a type of graph, the trained parameters form a graph which causes the model to output a prediction for an input.)
Regarding claim 6, Albini teaches: The device according to claim 1, wherein the at least one sub-dataset is randomly sampled from said prior dataset and belongs to one among: a plurality of instances comprising a label corresponding to a decision different from said original decision, (P. 1056, § 3.1, lines 5-12 and the distribution called “Differently-labelled samples”. Since the training data is randomly split (according to p. 1066, § A.1, lines 3-4), the sub-dataset is also randomly sampled.)
a plurality of instances associated to decisions inferred by said model, said inferred decisions being different from said original decision,
a plurality of instances among the k closest neighbors of said original instance according to a predetermined distance metric, said plurality of instances being associated to decisions inferred by said model, said inferred decisions being different from said original decision.
Regarding claim 7, Albini teaches: The device according to claim 1, wherein, for said at least one sub-dataset, said set of Shapley values is computed by means of at least one heuristic chosen among Shapley Flow, ASV, Powerset Shapley values and sampling Shapley values. (Sampling Shapley values are the Shapley values computed for randomly-sampled counterfactual points. Page 1055, third bullet point in the middle of the col. 1 discloses this.)
Regarding claim 8, Albini teaches: The device according to claim 1, wherein said model is a multi-class classifier designed to assign input data points to one of multiple classes, said multiple classes being at least three mutually exclusive classes. (Page 1055, col. 1, § 2 to col. 2, line 8 discloses the binary classification model could be trivialized to a multi-class model such as a three-class model. Classes separated by decision thresholds are mutually exclusive.)
Regarding claim 9, Albini teaches: The device according to claim 1, wherein said model is a gradient boosting Tree model. (Page 1055, col. 1, the paragraph above § 2 discloses that tree-based ensemble models such as XGBoost (eXtreme Gradient Boosting) are used for classification; and page 1061, col. 1, line 10 to col. 2, line 1 discloses applying XGBoost.)
Regarding claim 10, Albini teaches: The device according to claim 1, wherein the automated decision-making system is configured to provide decisions relating to medical predictions, industrial quality inspection, banking, (Page 1055, col. 1, line 1 to col. 2, line 2; page 1061, col. 1, line 10 to col. 2, line 1; on page 1067, Table 2, “Dataset” column and the caption in lines 3-4 teaches providing decisions about whether to accept or reject a customer for a bank loan. Decision function F(x) provides a binary decision.)
fraud detection, statistics, survey feedback analysis, email spam classification.
Regarding claim 11, Albini teaches: A computer-implemented method for providing a counterfactual explanation of an original decision from an automated decision-making system based on a model of classifier type for decision optimization, said method comprising: (Page 1054, Abstract)
receiving a training dataset of given distribution; (Page 1061, col. 1, lines 10-12; and page 1066, col. 1, § A.1, lines 1-7 discloses splitting each dataset into train and test data.. A “given distribution” is a distribution of the training dataset.)
training said model based on said training dataset so as to obtain a trained model; (Page 1055, col. 1, the paragraph above § 2 states, “We note that in this paper we concentrate on tree-based models”; § 2 from line 1 to col. 2, line 8; and page 1066, col. 1, § A.1, lines 1-7 discloses training a model.)
obtaining said original decision from an implementation of said trained model receiving as input an instance referred to as original instance; (On page 1055, § 2 from line 1 to col. 2, line 2 discloses obtaining a model output f(x) and a model prediction F(x) which will generate an original decision after training the model. Page 1066, col. 1, § A.1, lines 1-7 discloses splitting each dataset into train and test data. The test data includes an “original instance”, and the trained model’s prediction is “said original decision”.)
obtaining causal knowledge relating to said model; (Causal knowledge includes trained parameters of the tree-based models “f” because parameters have been learned and they cause the model to generate a model output. See page 1055, col. 1, the paragraph above § 2 which states, “We note that in this paper we concentrate on tree-based models”; and page 1066, col. 1, § A.1, lines 1-7. Obtaining causal knowledge could also mean discovering most important features as disclosed in § 2.1 on page 1055.)
determining at least one sub-dataset sampled from a so-called prior dataset sampled from said given distribution; (Page 1056, § 3.1, from line 1 to “Differently-labelled samples” discloses generating samples in the training set labelled differently than the input. At least one sub-dataset is an individual sample. The differently-labeled samples are sampled from the training set and thus sampled from “said given distribution”.)
computing, for each among said at least one sub-dataset, at least one corresponding set of Shapley values relating to said model by using at least one Shapley values computing method and based on said at least one sub-dataset and said causal knowledge, so as to obtain at least one computation environment, said at least one computation environment comprising one among said at least one sub-dataset and one among said at least one Shapley values computing method; (Page 1057, col. 1, from line 4 through “Definition 3.1”, where a set of Shapley values are values v(S), a sub-dataset is the background dataset, a Shapley values computing method is the characteristic function for v(S) including C (the name of the counterfactual technique), and a computing environment includes one calculation of v(S). The Shapley values are further based on trained model parameters of the model “f”.)
determining one optimal computation environment among said at least one computation environment by means of at least one metric representative of an ability of the corresponding computation environment to provide counterfactual explanations; (Page 1057, col. 2, start of § 3.3 to page 1058, col. 1, line 5; page 1058, col. 1, § 4.1, lines 1-11; and page 1058, col. 2, subsection “Induced Counterfactual and Counterfactual-Ability” from line 1 through Definition 4.1. A “metric” corresponds to the counterfactual-ability CF, and an “optimal computation environment” is the induced counterfactual for an explanation.)
determining said counterfactual explanation in the shape of a counterfactual instance or of a set of contributing features by means of the Shapley values corresponding to said original instance within said optimal computation environment and referred to as optimal Shapley values. (Page 1057, col. 2, start of § 3.3 to page 1058, col. 1, line 5; page 1058, col. 1, § 4.1, lines 1-11; and page 1058, col. 2, subsection “Induced Counterfactual and Counterfactual-Ability” from line 1 through Definition 4.1. A “counterfactual instance” is an induced counterfactual x’.)
Regarding claim 12, Albini teaches: The method according to claim 11, wherein said method is implemented by a device comprising: at least one input interface configured to receive a training dataset of given distribution; (Page 1055, col. 1, § 2 from line 1 to col. 2, line 8; page 1061, col. 1, lines 10-12; and page 1066, col. 1, § A.1, lines 1-7 discloses running experiments using publicly available datasets, and splitting each dataset into train and test data. Obtaining the publicly available datasets implies an input interface for receiving them. A “given distribution” is a distribution of the training dataset.)
a processor configured to: … (Page 1066, col. 1, § A.2, lines 1-3)
at least one output interface configured to output said counterfactual explanation. (Page 1056, all of Fig. 1 and its caption discloses that using a set of counterfactuals (iv) a more actionable explanation is obtained. Box (iv) outputs an explanation – moving from point x towards a combination of Features A and B in the direction of the arrow. In Page 1058, col. 2, subsection “Induced Counterfactual and Counterfactual-Ability” from line 1 through Definition 4.1, optimizing the counterfactual-ability outputs the induced counterfactual.)
Regarding the limitations of claim 12 from line 5 to page 5, line 1, the rejection of claim 11 above explains how Albini teaches the remaining limitations of claim 12.
Regarding claim 13, Albini teaches: A non-transitory program storage device, readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method for providing a counterfactual explanation of an original decision from an automated decision-making system based on a model of classifier type for decision optimization according to claim 11. (Albini at page 1054, Abstract, and page 1066, col. 1, § A.2, lines 1-3 discloses a device for providing a counterfactual explanation. Running experiments on Albini’s computer system inherently means there is a non-transitory program storage device as claimed.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Albini et al. (“Counterfactual Shapley Additive Explanations”) in view of Verma et al. (US 11403538 B1).
Regarding claim 5, Albini teaches: The device according to claim 1, wherein said causal knowledge obtained by said at least one processor comprises knowledge (Causal knowledge includes trained parameters of the tree-based models “f” because parameters have been learned and they cause the model to generate a model output. See page 1055, col. 1, the paragraph above § 2 which states, “We note that in this paper we concentrate on tree-based models”; and page 1066, col. 1, § A.1, lines 1-7. Obtaining causal knowledge could also mean discovering most important features as disclosed in § 2.1 on page 1055.)
However, Albini does not explicitly teach as a whole: wherein said causal knowledge obtained by said at least one processor comprises knowledge from at least one expert and is received via said at least one input interface.
But Verma teaches: wherein said causal knowledge obtained
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have obtained causal constraints and relationships via a GUI from a user of Albini’s system. A motivation for the combination is to train the system to generate better counterfactual explanations by encoding which feature can realistically be changed for a user (e.g., income but not race). (C. 10, L. 19-30)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Asher H. Jablon whose telephone number is (571)270-7648. The examiner can normally be reached Monday - Friday, 9:00 am - 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/A.H.J./Examiner, Art Unit 2127
/ABDULLAH AL KAWSAR/Supervisory Patent Examiner, Art Unit 2127
Prosecution Insights