DETAILED ACTION
This action is in response to the RCE filed 28 January 2026.
Claims 1–4, 6–19, 21, and 22 are pending. Claims 1, 14, and 19 are independent.
Claims 1–4, 6–19, 21, and 22 are rejected.
Notice of Pre-AIA or AIA Status
The present application, filed on or after 16 March 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. §§ 102 and 103 (or as subject to pre-AIA 35 U.S.C. §§ 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Continued Examination
A request for continued examination under 37 C.F.R. § 1.114, including the fee set forth in 37 C.F.R. § 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 C.F.R. § 1.114, and the fee set forth in 37 C.F.R. § 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 C.F.R. § 1.114. Applicant's submission filed on 22 December 2025 has been entered.
Response to Arguments
The objection to claim 2 is withdrawn in light of the amendment and accompanying arguments (remarks, p. 7).
Applicant's arguments, see remarks filed 22 December 2025, with respect to the rejection(s) of claim(s) 1–4, 6–19, 21, and 22 under § 103 have been fully considered but are not persuasive.
Applicant argues in substance that Soleimani and Li do not teach determining whether a new decision of a machine learning model, after masking a training example, is the same or different from a decision made before the masking (remarks, pp. 7–8). In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). In this case, Soleimani teaches perturbing input to a machine learning model by removing a training data point, checking whether the decision of the model changes, followed by replacing the removed data back into the training data (Soleimani, ¶ 75). Although Soleimani does not expressly teach perturbing the data by masking it, Li (and Wu) teach perturbing data by replacing text with masking tokens. Therefore, a combination of Soleimani and Li, wherein the perturbation by removal is replaced by perturbation by masking, teaches checking the decision of a model before and after masking the input data.
Claim Interpretation
Claim 19 is interpreted as excluding transitory media, based on paragraph 84 of the specification.
Claim Rejections—35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 C.F.R. § 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. § 102(b)(2)(C) for any potential 35 U.S.C. § 102(a)(2) prior art against the later invention.
Claims 1–4, 6, 7, 14, 15, 19, 21, and 22 are rejected under 35 U.S.C. § 103 as being unpatentable over Soleimani et al. (US 2021/0117863 A1) [hereinafter Soleimani] in view of Li et al.1 (“BERT-ATTACK: Adversarial Attack Against BERT Using BERT”) [hereinafter Li], Wu et al. (“Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT”) [hereinafter Wu], and Joshi et al. (“SpanBERT: Improving Pre-training by Representing and Predicting Spans”) [hereinafter Joshi].
Regarding independent claim 1, Soleimani teaches [a] method for explaining a machine learning model
θ
^
, the method comprising: training the machine learning model
θ
^
with training data
D
including a plurality of datapoints; Training data is used by a machine learning algorithm to provide a trained model (Soleimani, ¶ 75). […] [masking] the training [data]; A training data point is removed from the training data set to form a modified training data set (Soleimani, ¶ 75). determining whether a new decision,
y
'
, of the machine learning model
θ
^
obtained after the masking is the same as the first decision
y
of the machine learning model
θ
^
obtained prior to the [masking] of training example
z
=
x
,
y
; The model is trained using the new data set, a new prediction is made, and the prediction is compared to a prediction made with the full training data (Soleimani, ¶ 75). The comparison determines whether the removal of a particular training data point results in a change in the prediction (Soleimani, ¶ 78). using the [masking] to explain which data spans of the plurality of datapoints in the training data
D
are significant. A measure of the influence of the removed data point is generated and, e.g., displayed to the user visually in a graph or table (Soleimani, ¶ 79). The display provides an explanation of the model (Soleimani, ¶¶ 40, 62).
Soleimani teaches determining the influence of a training data point by removing the data point, but does not expressly teach masking the data point. However, Li teaches: masking [the training data] A word in a sentence input to a machine learning model is masked by replacing the word with “[MASK]” (Li, p. 6195, § 3.1). [determining whether a new decision of the machine learning model
θ
^
obtained after the masking is the same as the first decision
y
of the machine learning model
θ
^
obtained prior to the] masking; The output of the machine learning model with the original input sentence is compared to the output using the masked input sentence (Li, p. 6195, § 3.1). [using the] masking [to explain which of the one or more datapoints in the training data
D
are significant according to the importance of each data span, including the training span
x
i
j
.] An importance score for the word is calculated using the two outputs (Li, p. 6195, § 3.1).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani with those of Li. Doing so would have been a matter of simple substitution of one known element [removing a data point] for another [masking a data point] to obtain predictable results [a machine learning model explanation wherein the influence of a data point is determined by masking the data point].
Soleimani/Li teaches determining an importance of a training data point by masking it, but does not expressly teach identifying the importance of a span of training data. However, Wu teaches: automatically identifying a training span
x
i
j
in a training example
z
=
x
,
y
from the training data
D
, wherein
x
is a training sequence and
y
is a first decision of the machine learning model
θ
^
; A document is modeled as having a number of non-overlapping text spans, each span containing a sequence of tokens (Wu, p. 4167, § 2.3). masking the training span
x
i
j
to provide
z
-
i
j
=
x
-
i
j
,
y
'
as the masking, […] wherein
z
-
i
j
is corresponding training data to the training span
x
i
j
, and
x
-
i
j
is a sequence in which training span
x
i
j
is masked; and An array of tokens in a span are masked (Wu, p. 4167, § 2.3). The masking may be performed by replacing tokens with “[MASK]” (Wu, p. 4167, § 2.1). responsive to the new decision
y
'
being different than the first decision
y
, determining an importance of the training span
x
i
j
on the training example
z
=
x
,
y
using the masking; The masking is used to assess the impact [importance] of the words on another prediction (Wu, p. 4167, § 2).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani/Li with those of Wu. Doing so would have been a matter of applying a known technique [extending perturbation from single words to multi-word spans] to a known method ready for improvement [masking single words to determine their influence] to yield predictable results [a method of explaining ML models by masking single words or spans of words].
Soleimani/Li/Wu teaches masking a training span by masking an array of tokens simultaneously, but does not expressly teach masking each token individually. However, Joshi teaches: [masking] the training span
x
i
j
including datapoints from token
i
to token
j
, A span of tokens is masked by replacing each individual token in the span with the [MASK] token (Joshi, p. 66, § 3.1; p. 65, fig. 1).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani/Li/Wu with those of Joshi. Doing so would have been a matter of simple substitution of one known element [masking a span by masking all tokens simultaneously] for another element [masking a span by masking each token individually] to yield predictable results [a method of determining the influence of a span by masking the individual tokens in the span].
Regarding dependent claim 2, the rejection of claim 1 is incorporated and Soleimani/Li/Wu/Joshi further teaches: wherein the training span
x
i
j
, when masked, significantly changes the decision of the machine learning model
θ
^
. The removal of a data point can change the output prediction, such that the greater the change in the prediction, the greater the influence of a particular data point (Soleimani, ¶ 78). The importance score can be used to determine words having a high significance on the output (Li, p. 6195, § 3.1).
Regarding dependent claim 3, the rejection of claim 1 is incorporated and Soleimani/Li/Wu/Joshi further teaches: wherein the machine learning model
θ
^
is used for natural language processing. The model is BERT, a language model (Li, p. 6193, abstract). The model performs NLP [natural language processing] tasks (Li, p. 6197, § 4.1).
Regarding dependent claim 4, the rejection of claim 1 is incorporated and Soleimani/Li/Wu/Joshi further teaches: wherein the machine learning model comprises a trained neural network. The machine learning algorithm/model may be a neural network (Soleimani, ¶ 96). The model is BERT, a neural network model (Li, p. 6193, abstract).
Regarding dependent claim 6, the rejection of claim 1 is incorporated and Soleimani/Li/Wu/Joshi further teaches: wherein the importance of the training span
x
i
j
is determined using a loss gradient. The importance score is calculated using the difference of the logit output based on the unmasked sentence and the logit output based on the masked sentence (Li, p. 6195, § 3.1). [Applicant states in para. 51 of the specification that this logit difference is equivalent to the loss difference recited in claim 7.]
Regarding dependent claim 7, the rejection of claim 6 is incorporated and Soleimani/Li/Wu/Joshi further teaches: determining the loss gradient as:
imp
x
i
j
z
,
θ
^
=
L
z
-
i
j
;
θ
^
-
L
z
;
θ
^
. The importance score is calculated using the difference of the logit output based on the unmasked sentence and the logit output based on the masked sentence (Li, p. 6195, § 3.1). [Applicant states in para. 51 of the specification that this logit difference is equivalent to the loss difference recited in the claim.]
Regarding independent claim 14, Soleimani teaches [a] method for explaining a machine learning model
θ
^
, the method comprising: automatically identifying a training [span]
x
i
j
in a training example
z
=
x
,
y
from training data
D
used to train the machine learning
θ
^
, wherein
x
is a training [sequence] and
y
is a first decision of the machine learning model
θ
^
; Training data is used by a machine learning algorithm to provide a trained model (Soleimani, ¶ 75). The trained model is used to provide a prediction [decision] (Soleimani, ¶ 75). [masking] the training span
x
i
j
to provide
z
-
i
j
=
x
-
i
j
,
y
, as the masking, […] wherein
z
-
i
j
is corresponding training data to the training span
x
i
j
, and
x
-
i
j
is a sequence in which training span
x
i
j
is [masked]; A training data point is removed from the training data set to form a modified training data set (Soleimani, ¶ 75). […] determining an importance of the training span
x
i
j
on the training example
z
=
x
,
y
using the [masking]. A measure of the influence of the removed data point is generated and, e.g., displayed to the user visually in a graph or table (Soleimani, ¶ 79).
Soleimani teaches determining the influence of a training data point by removing the data point, but does not expressly teach masking the data point. However, Li teaches: masking [the training span
x
i
j
to provide
z
-
i
j
=
x
-
i
j
,
y
'
, as the masking, the training span
x
i
j
including datapoints from token
i
to token
j
, wherein
z
-
i
j
is corresponding training data to the training span
x
i
j
, and
x
-
i
j
is a sequence in which training span
x
i
j
is] masked A word in a sentence input to a machine learning model is masked by replacing the word with “[MASK]” (Li, p. 6195, § 3.1). [determining whether a new decision,
y
'
, of the machine learning model
θ
^
obtained after the masking is the same as the first decision
y
of the machine learning model
θ
^
obtained prior to the] masking of training example
z
=
x
,
y
; The output of the machine learning model with the original input sentence is compared to the output using the masked input sentence (Li, p. 6195, § 3.1). responsive to the new decision
y
'
being different than the first decision
y
, [determining an importance of the training span
x
i
j
on the training example
z
=
x
,
y
using the] masking The output of the machine learning model with the original input sentence is compared to the output using the masked input sentence (Li, p. 6195, § 3.1). An importance score for the word is calculated using the two outputs (Li, p. 6195, § 3.1).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani with those of Li. Doing so would have been a matter of simple substitution of one known element [removing a data point] for another [masking a data point] to obtain predictable results [a machine learning model explanation wherein the influence of a data point is determined by masking the data point].
Soleimani/Li teaches determining an importance of a training data point by masking it, but does not expressly teach identifying the importance of a span of training data. However, Wu teaches: [a training] span A document is modeled as having a number of non-overlapping text spans, each span containing a sequence of tokens (Wu, p. 4167, § 2.3). [masking the training] span An array of tokens in a span are masked (Wu, p. 4167, § 2.3). The masking may be performed by replacing tokens with “[MASK]” (Wu, p. 4167, § 2.1).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani/Li with those of Wu. Doing so would have been a matter of applying a known technique [extending perturbation from single words to multi-word spans] to a known method ready for improvement [masking single words to determine their influence] to yield predictable results [a method of explaining ML models by masking single words or spans of words].
Soleimani/Li/Wu teaches masking a training span by masking an array of tokens simultaneously, but does not expressly teach masking each token individually. However, Joshi teaches: [masking] the training span
x
i
j
including datapoints from token
i
to token
j
, A span of tokens is masked by replacing each individual token in the span with the [MASK] token (Joshi, p. 66, § 3.1; p. 65, fig. 1).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani/Li/Wu with those of Joshi. Doing so would have been a matter of simple substitution of one known element [masking a span by masking all tokens simultaneously] for another element [masking a span by masking each token individually] to yield predictable results [a method of determining the influence of a span by masking the individual tokens in the span].
Regarding dependent claim 15, the rejection of claim 14 is incorporated and Soleimani/Li/Wu/Joshi further teaches: determining an influence of the training span
x
i
j
on the machine learning model
θ
^
. A measure of the influence of the removed data point is generated and, e.g., displayed to the user visually in a graph or table (Soleimani, ¶ 79). The display provides an explanation of the model (Soleimani, ¶¶ 40, 62).
Regarding independent claim 19, Soleimani teaches [a] computer program product for explaining a machine learning model
θ
^
, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to: train the machine learning model
θ
^
with training data
D
including a plurality of datapoints; Training data is used by a machine learning algorithm to provide a trained model (Soleimani, ¶ 75). […] [mask] the training [data]; A training data point is removed from the training data set to form a modified training data set (Soleimani, ¶ 75). determine whether a new decision of the machine learning model
θ
^
obtained after the [masking] is the same as the first decision
y
of the machine learning model
θ
^
obtained prior to the [masking]; The model is trained using the new data set, a new prediction is made, and the prediction is compared to a prediction made with the full training data (Soleimani, ¶ 75). The comparison determines whether the removal of a particular training data point results in a change in the prediction (Soleimani, ¶ 78). use the [masking] to explain which of the one or more datapoints in the training data
D
are significant. A measure of the influence of the removed data point is generated and, e.g., displayed to the user visually in a graph or table (Soleimani, ¶ 79). The display provides an explanation of the model (Soleimani, ¶¶ 40, 62).
Soleimani teaches determining the influence of a training data point by removing the data point, but does not expressly teach masking the data point. However, Li teaches: mask [the training data] A word in a sentence input to a machine learning model is masked by replacing the word with “[MASK]” (Li, p. 6195, § 3.1). [determine whether a new decision of the machine learning model
θ
^
obtained after the] masking [is the same as the first decision
y
of the machine learning model
θ
^
obtained prior to the] masking; and The output of the machine learning model with the original input sentence is compared to the output using the masked input sentence (Li, p. 6195, § 3.1). [use the] masking [to explain which of the one or more datapoints in the training data
D
are significant.] An importance score for the word is calculated using the two outputs (Li, p. 6195, § 3.1).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani with those of Li. Doing so would have been a matter of simple substitution of one known element [removing a data point] for another [masking a data point] to obtain predictable results [a machine learning model explanation wherein the influence of a data point is determined by masking the data point].
Soleimani/Li teaches determining an importance of a training data point by masking it, but does not expressly teach identifying the importance of a span of training data. However, Wu teaches: automatically identifying a training span
x
i
j
in a training example
z
=
x
,
y
from the training data
D
, wherein
x
is a training sequence and
y
is a first decision of the machine learning model
θ
^
; A document is modeled as having a number of non-overlapping text spans, each span containing a sequence of tokens (Wu, p. 4167, § 2.3). mask the training span
x
i
j
to provide
z
-
i
j
=
x
-
i
j
,
y
as the masking, […] wherein
z
-
i
j
is corresponding training data to the training span
x
i
j
, and
x
-
i
j
is a sequence in which training span
x
i
j
is masked; and An array of tokens in a span are masked (Wu, p. 4167, § 2.3). The masking may be performed by replacing tokens with “[MASK]” (Wu, p. 4167, § 2.1). determine an importance of the training span
x
i
j
on the training example
z
=
x
,
y
using the masking; The masking is used to assess the impact [importance] of the words on another prediction (Wu, p. 4167, § 2).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani/Li with those of Wu. Doing so would have been a matter of applying a known technique [extending perturbation from single words to multi-word spans] to a known method ready for improvement [masking single words to determine their influence] to yield predictable results [a method of explaining ML models by masking single words or spans of words].
Soleimani/Li/Wu teaches masking a training span by masking an array of tokens simultaneously, but does not expressly teach masking each token individually. However, Joshi teaches: [mask] the training span
x
i
j
including datapoints from token
i
to token
j
, A span of tokens is masked by replacing each individual token in the span with the [MASK] token (Joshi, p. 66, § 3.1; p. 65, fig. 1).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani/Li/Wu with those of Joshi. Doing so would have been a matter of simple substitution of one known element [masking a span by masking all tokens simultaneously] for another element [masking a span by masking each token individually] to yield predictable results [a method of determining the influence of a span by masking the individual tokens in the span].
Regarding dependent claim 21, the rejection of claim 1 is incorporated and Soleimani/Li/Wu/Joshi further teaches: wherein identifying the training span
x
i
j
includes: identifying the training span
x
i
j
as a key span based on predefined rules defining questions and answers. The masking may be applied in question/answer systems, e.g., to mask a span in a passage corresponding to an answer to a question (Joshi, p. 64, § 1; pp. 67–68, § 4.1).
Regarding dependent claim 22, the rejection of claim 1 is incorporated and Soleimani/Li/Wu/Joshi further teaches: determining a set of training spans, including the training span
x
i
j
, from the training data
D
by dividing text sequences into chunks of text as the set of training spans. The text is represented as a sequence of tokens [chunks of text] (Joshi, p. 66, § 3.1). A model is trained by masking multiple spans [a set of training spans] (Joshi, p. 67, § 3.3).
Claim 8 is rejected under 35 U.S.C. § 103 as being unpatentable over Soleimani in view of Li, Wu, and Joshi, further in view of Koh et al. (“Understanding Black-box Predictions via Influence Functions”) [hereinafter Koh].
Regarding dependent claim 8, the rejection of claim 7 is incorporated. Soleimani/Li/Wu/Joshi teaches determining an influence of a training span, but does not expressly teach scaling. However, Koh teaches: determining an influence of the training span
x
i
j
on the machine learning model
θ
^
by scaling
imp
x
i
j
z
,
θ
^
. A training point
z
is upweighted [scaled] by a value
ϵ
=
-
1
n
(where
n
is the number of training points) to approximate the change due to removing
z
from the training set (Koh, § 2.1).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani/Li/Wu/Joshi with those of Koh. One would have been motivated to do so in order to save time by removing the need to retrain the model for each removed data point (Koh, § 2.1).
Claims 9–13 and 16–18 are rejected under 35 U.S.C. § 103 as being unpatentable over Soleimani in view of Li, Wu, and Joshi, further in view of Barshan et al. (US 2021/0103829 A1) [hereinafter Barshan].
Regarding dependent claim 9, the rejection of claim 1 is incorporated. Soleimani/Li/Wu/Joshi teaches determining an influence of a training example, but does not expressly teach determining the influence of a training example on a test example. However, Barshan teaches: determining an influence of a training example
z
on a test example
z
'
. An influence score is determined for each training data point in a set of training data; the influence score may be determined with respect to an input
z
t
e
s
t
(Barshan, ¶¶ 74–76).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani/Li/Wu/Joshi with those of Barshan. Doing so would have been a matter of simple substitution of one known element [determining an influence score as in Soleimani] for another [determining an influence score based on a specific test example] to obtain predictable results [a method of explaining a ML model using masking and influence scores generated with respect to test data].
Regarding dependent claim 10, the rejection of claim 9 is incorporated and Soleimani/Li/Wu/Joshi/Barshan further teaches: determining an importance
∇
imp
x
'
k
l
z
'
;
θ
^
of a test span
x
'
k
l
on the test example
z
'
; A gradient of loss is calculated for an input [test data] (Barshan, ¶ 74). The importance score is based on the logit output obtained by inputting a masked sentence [test data] (Li, p. 6195, § 3.1). A document [training data] may be modeled as non-overlapping text spans (Wu, p. 4167, § 2.3). The document may be perturbed by masking an array of tokens [test span] (Wu, p. 4167, § 2.3) determining an importance
∇
imp
x
i
j
z
;
θ
^
of a training [span]
x
i
j
on the training example
z
; A gradient of loss is calculated for training data (Barshan, ¶ 74). The importance score is based on the logit output obtained by inputting the unmodified sentence [training data] (Li, p. 6195, § 3.1). determining the influence of the training example
z
on the test example
z
'
using
∇
imp
x
'
k
l
z
'
;
θ
^
and
∇
imp
x
i
j
z
;
θ
^
. The influence score is calculated by multiplying the loss gradient of the input [test] and the loss gradient of the training data points (Barshan, ¶ 74).
Regarding dependent claim 11, the rejection of claim 10 is incorporated and Soleimani/Li/Wu/Joshi/Barshan further teaches: wherein the influence of the training example
z
on the test example
z
'
is determined as
∇
imp
x
'
k
l
z
'
;
θ
^
∇
imp
x
i
j
z
;
θ
^
. The influence score may be determined by multiplying the gradient of loss of the input and the gradient of loss of the training data points (Barshan, ¶ 74).
Regarding dependent claim 12, the rejection of claim 10 is incorporated and Soleimani/Li/Wu/Joshi/Barshan further teaches: evaluating whether the training span
x
i
j
of the training example
z
is semantically related to the test span
x
'
k
l
of the test example
z
'
. A similarity measure between a forecast data point and a training data point (Soleimani, ¶ 47). A similarity may be determined between the gradients of the training data points and test data points (Barshan, ¶¶ 65–67).
Regarding dependent claim 13, the rejection of claim 12 is incorporated and Soleimani/Li/Wu/Joshi/Barshan further teaches: defining a semantic representation of the training span
x
i
j
of training example
z
; and A loss gradient for the training data and test data are calculated and used to determine training data points that are most similar to a prediction (Barshan, ¶¶ 65–67). measuring the similarity of the semantic representation of the training span
x
i
j
of training example
z
to a semantic representation of the test span
x
'
k
l
of the test example
z
'
. A similarity may be determined between the gradients of the training data points and test data points (Barshan, ¶¶ 65–67).
Regarding dependent claim 16, the rejection of claim 14 is incorporated. Soleimani/Li/Wu/Joshi teaches determining an influence of a training example, but does not expressly teach determining the influence of a training example on a test example. However, Barshan teaches: determining an influence of the training example
z
=
x
,
y
on a test example
z
'
. An influence score is determined for each training data point in a set of training data; the influence score may be determined with respect to an input
z
t
e
s
t
(Barshan, ¶¶ 74–76).
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to combine the teachings of Soleimani/Li/Wu/Joshi with those of Barshan. Doing so would have been a matter of simple substitution of one known element [determining an influence score as in Soleimani] for another [determining an influence score based on a specific test example] to obtain predictable results [a method of explaining a ML model using masking and influence scores generated with respect to test data].
Regarding dependent claim 17, the rejection of claim 16 is incorporated and Soleimani/Li/Wu/Joshi/Barshan further teaches: determining an importance
∇
imp
x
'
k
l
z
'
;
θ
^
of a test span
x
'
k
l
on the test example
z
'
; A gradient of loss is calculated for an input [test data] (Barshan, ¶ 74). The importance score is based on the logit output obtained by inputting a masked sentence [test data] (Li, p. 6195, § 3.1). determining an importance
∇
imp
x
i
j
z
;
θ
^
of the training span
x
i
j
on the training example
z
=
x
,
y
; and A gradient of loss is calculated for training data (Barshan, ¶ 74). The importance score is based on the logit output obtained by inputting the unmodified sentence [training data] (Li, p. 6195, § 3.1). determining an influence of the training example
z
=
x
,
y
on the test example
z
'
using
∇
imp
x
'
k
l
z
'
;
θ
^
and
∇
imp
x
i
j
z
;
θ
^
. The influence score is calculated by multiplying the loss gradient of the input [test] and the loss gradient of the training data points (Barshan, ¶ 74).
Regarding dependent claim 18, the rejection of claim 17 is incorporated and Soleimani/Li/Wu/Joshi/Barshan further teaches: evaluating whether the training span
x
i
j
of the training example
z
=
x
,
y
is semantically related to the test span
x
'
k
l
of the test example
z
'
. A similarity measure between a forecast data point and a training data point (Soleimani, ¶ 47). A similarity may be determined between the gradients of the training data points and test data points (Barshan, ¶¶ 65–67).
Conclusion
The prior art made of record and not relied upon is considered pertinent to Applicant's disclosure. Liu et al. teaches masking individual explainable features and calculating the changes in predictions of a ML model (see US 2021/0374601, ¶ 24). Ratner et al. teaches masking ML model inputs to improve explanatory quality of the ML model output (see US 2021/0142176 A1, abstract).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tyler Schallhorn whose telephone number is 571-270-3178. The examiner can normally be reached Monday through Friday, 8:30 a.m. to 6 p.m. (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tamara Kyle can be reached on 571-272-4241. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (in the USA or Canada) or 571-272-1000.
/Tyler Schallhorn/Examiner, Art Unit 2144
/TAMARA T KYLE/Supervisory Patent Examiner, Art Unit 2144
1 Cited by Applicant in an IDS filed 31 May 2021.
Prosecution Insights