DETAILED ACTION
This action is responsive to the amendment filed on 03/12/2026. Claims 1-20 are pending in the case. Claims 6, 8, 13, and 19 are currently amended. Claims 1, 8, and 15 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Applicant is advised that should claim 7 be found allowable, claim 20 will be objected to under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application are duplicates or else are so close in content that they both cover the same thing, despite a slight difference in wording, it is proper after allowing one claim to object to the other as being a substantial duplicate of the allowed claim. See MPEP § 608.01(m).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1:
Step 1 Statutory Category: Claim 1 is directed to a process, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial Exception: Claim 1 recites, in part, “determining a topological uncertainty of the artificial neural network by: forming a bipartite graph between input and output nodes in a layer of the artificial neural network; and generating a persistence diagram as a function of the bipartite graph”. This limitation, under the broadest reasonable interpretation, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “determining a latent uncertainty of the artificial neural network”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, the claim recites: “estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty”. This limitation under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C).
Step 2A Prong 2 Integration into a Practical Application: This judicial exception is not integrated into a practical application. In particular the claim recites: “an artificial neural network”. This is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h).
Step 2B Significantly More: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element: “an artificial neural network” generally links the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 2, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the bipartite graph comprises a weight matrix for the layer and data input into the layer”. This limitation is a continuation of the “determining a topological uncertainty of the artificial neural network by: forming a bipartite graph between input and output nodes in a layer of the artificial neural network; and generating a persistence diagram as a function of the bipartite graph” limitation identified as an abstract idea in the rejection of the parent claim, therefore the claim recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 3, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the latent uncertainty comprises a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer”. This limitation is a continuation of the “determining a latent uncertainty of the artificial neural network” limitation identified as an abstract idea in the rejection of the parent claim, thus the claim recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 4, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the uncertainty of the artificial neural network comprises an out of distribution condition when the topological uncertainty is greater than a first threshold and the latent uncertainty is greater than a second threshold”. This limitation recites mathematical concepts in addition to those identified in the rejection of the parent claim, and thus recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 5, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the computing of the persistence diagram comprises:
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
wherein L comprises a number of layers in the artificial neural network; wherein l comprises a particular layer in the artificial neural network; wherein x comprises an input data value; wherein F comprises a description of the artificial neural network; wherein k(x) comprises a class of x derived from processing by the artificial neural network; and wherein Dtrain comprises an average of data used to train the artificial neural network”. This limitation recites the abstract idea of a mathematical formula or equation, as directed to “a claim that recites a numerical formula or equation will be considered as falling within the "mathematical concepts" grouping. In addition, there are instances where a formula or equation is written in text format that should also be considered as falling within this grouping”. See MPEP § 2106.04(a)(2)(I)(B).
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 6, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the estimating the uncertainty of the artificial neural network comprises an out of distribution condition, and verifying the out of distribution condition”. This limitation is a continuation of the “estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty” limitation identified as an abstract idea in the rejection of the parent claim, thus the claim recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 7, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the estimating the uncertainty comprises an in distribution condition and a classification of data input with an acceptable uncertainty”. This limitation is a continuation of the “estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty” limitation identified as an abstract idea in the rejection of the parent claim, thus the claim recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Regarding claim 8:
Step 1 Statutory Category: Claim 8 is directed to a machine, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial Exception: Claim 8 recites, in part, “determining a topological uncertainty of an artificial neural network by: forming a bipartite graph between input and output nodes in a layer of the artificial neural network; and generating a persistence diagram as a function of the bipartite graph”. This limitation, under the broadest reasonable interpretation, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “determining a latent uncertainty of the artificial neural network”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, the claim recites: “estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty”. This limitation under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C).
Step 2A Prong 2 Integration into a Practical Application: This judicial exception is not integrated into a practical application. In particular the claim recites: “a non-transitory machine-readable medium comprising instructions that when executed by a processor executes a process”. This limitation is an additional element that that amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “the artificial neural network”. This is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h).
Step 2B Significantly More: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element: “a non-transitory machine-readable medium comprising instructions that when executed by a processor executes a process” amounts to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the additional element “the artificial neural network” generally links the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 9, the rejection of claim 8 is incorporated, and further, claim 9 is substantially similar to claim 2 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 10, the rejection of claim 8 is incorporated, and further, claim 10 is substantially similar to claim 3 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 11, the rejection of claim 8 is incorporated, and further, claim 11 is substantially similar to claim 4 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 12, the rejection of claim 8 is incorporated, and further, claim 12 is substantially similar to claim 5 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 13, the rejection of claim 8 is incorporated, and further, claim 13 is substantially similar to claim 6 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 14, the rejection of claim 8 is incorporated, and further, claim 14 is substantially similar to claim 7 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 15:
Step 1 Statutory Category: Claim 15 is directed to a machine, which falls under one of the four statutory categories.
Step 2A Prong 1 Judicial Exception: Claim 15 recites, in part, “determining a topological uncertainty of an artificial neural network by: forming a bipartite graph between input and output nodes in a layer of the artificial neural network; and generating a persistence diagram as a function of the bipartite graph”. This limitation, under the broadest reasonable interpretation, covers the recitation of mathematical concepts, see MPEP §2106.04(a)(2)(I). Further, the claim recites: “determining a latent uncertainty of the artificial neural network”. This limitation, under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C). Further, the claim recites: “estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty”. This limitation under the broadest reasonable interpretation, covers the recitation of a mathematical calculation, as directed to “a claim that recites a mathematical calculation, when the claim is given its broadest reasonable interpretation in light of the specification, will be considered as falling within the "mathematical concepts" grouping. A mathematical calculation is a mathematical operation (such as multiplication) or an act of calculating using mathematical methods to determine a variable or number”. See MPEP §2106.04(a)(2)(I)(C).
Step 2A Prong 2 Integration into a Practical Application: This judicial exception is not integrated into a practical application. In particular the claim recites: “a system”, “a computer processor”, and “a memory coupled to the computer processor”. These limitations are additional elements that that amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. See MPEP §2106.05(f). Further, the claim recites: “an artificial neural network”. This is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. See MPEP §2106.05(h).
Step 2B Significantly More: The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements: “a system”, “a computer processor”, and “a memory coupled to the computer processor” amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process. Elements that merely amount to adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer in its ordinary capacity as a tool to perform an existing process cannot provide an inventive concept. Further, the additional element “an artificial neural network” generally links the use of the judicial exception to a particular technological environment or field of use. Elements that merely generally link the use of the judicial exception to a particular technological environment or field of use cannot provide an inventive concept. The claim is not patent eligible.
Regarding claim 16, the rejection of claim 15 is incorporated, and further, claim 16 is substantially similar to claim 3 and claim 10 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 17, the rejection of claim 15 is incorporated, and further, claim 17 is substantially similar to claim 4 and claim 11 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 18, the rejection of claim 15 is incorporated, and further, claim 18 is substantially similar to claim 5 and claim 12 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 19, the rejection of claim 15 is incorporated, and further, claim 19 is substantially similar to claim 6 and claim 13 respectively, and is rejected in the same manner and reasoning applying.
Regarding claim 20, the rejection of claim 1 is incorporated, and further, the claim recites: “wherein the estimating the uncertainty comprises an in distribution condition and a classification of data input with an acceptable uncertainty”. This limitation is a continuation of the “estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty” limitation identified as an abstract idea in the rejection of the parent claim, thus the claim recites a judicial exception.
The claim does not include any additional elements that amount to an integration of the judicial exception into a practical application, nor to significantly more than the judicial exception. The claim is not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lacombe et al., Topological Uncertainty: Monitoring trained neural networks through persistence of activation graphs, 05/07/2021, https://arxiv.org/pdf/2105.04404 hereinafter referred to as “Lacombe” in view of Gomes et al., IGEOOD: AN INFORMATION GEOMETRY APPROACH TO OUT-OF-DISTRIBUTION DETECTION, 03/15/2022, https://arxiv.org/pdf/2203.07798 hereinafter referred to as “Gomes” in further view of Kadayam Viswanathan et al., U.S. Patent Application Publication No. 20210073631, “Kadayam Viswanathan”.
Lacombe teaches A process for estimating an uncertainty of an artificial neural network comprising:
determining a topological uncertainty of the artificial neural network (Lacombe, Page 3, Col 2, Paragraph 2, Lines 7-8, and Equation 1, “The Topological Uncertainty of x is defined to be
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
”) by:
forming a bipartite graph between input and output nodes in a layer of the artificial neural network (Lacombe, Page 3, Section 3, Lines 3-4, “Given a trained network F and an observation x, we build a sequence of activation graphs
G
1
x
,
F
,
…
,
G
L
-
1
(
x
,
F
)
”; Lacombe, Page 2, Section 2.2, Paragraph 1, Lines 12-16, “In this way, we obtain a sequence of bipartite graphs
(
G
l
x
,
F
)
l
called the activation graphs of the pair (x, F), whose vertices are
V
l
⊔
V
l
+
1
and edges weights are given by the aforementioned formula”; The “activation graphs” are considered to be the “bipartite graph”); and
generating a persistence diagram as a function of the bipartite graph (Lacombe, Page 3, Section 3, Lines 5-6, “We then compute a MST of each
G
l
x
,
F
, which in turn induces a persistence diagram
D
l
x
,
F
:=
μ
(
G
l
x
,
F
)”).
Lacombe does not explicitly teach determining a latent uncertainty of the artificial neural network nor estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty.
Gomes teaches determining a latent uncertainty of the artificial neural network (Gomes, Page 5, OOD and confidence score, Lines 8-11, and Equation 5, “we propose the Fisher-Rao distance-based OOD detection score FR0(x) for the logits to be the sum of the distances between f(x) and each individual class conditional centroid µy given by Eq. (3) … We denote it by
F
R
0
(
x
)
≜
∑
y
∈
Y
d
F
R
-
L
o
g
i
t
s
q
θ
∙
f
x
,
q
θ
∙
μ
y
”; Gomes, Page 6, Section 3.3, Lines 1-3, “For each layer, we define a set of class-conditional Gaussian distributions with diagonal standard deviation matrix σ (
l
) and class-conditional mean
μ
y
(
l
)
, where y ∈ {1, . . . , C} and
l
is the index of the latent feature”; Gomes, Page 6, Fisher-Rao distance-based feature-wise confidence score, Lines 1-3, and Equations 11 and 12, “We derive a confidence score by applying the Fisher-Rao distance between the test sample x and the closest class-conditional diagonal Gaussian distribution”; “
F
R
l
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
l
,
σ
l
”; “
F
R
l
'
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
'
l
,
σ
'
l
”; Gomes, Page 6, Feature Ensemble, Lines 1-2, “To further improve performance, we combine the confidence scores of the logits and the ones from the low-level features through a linear combination”; Gomes, Page 6, Equation 13, “
F
R
x
≜
α
0
F
R
0
x
+
∑
l
α
l
∙
F
R
l
x
+
α
l
'
∙
F
R
l
'
x
”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention to have modified the uncertainty estimation method of Lacombe to include determining a latent uncertainty as taught by Gomes. The motivation for doing so would have been that combining the confidence scores of the logits and the ones from the low-level features through a linear combination further improve performance of OOD detection (Gomes, Page 6, Feature Ensemble, Lines 1-2).
Lacombe in view of Gomes teaches a topological uncertainty and a latent uncertainty they do not teach estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty.
Kadayam Viswanathan teaches estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty (Kadayam Viswanathan, Paragraph 0064, Lines 6-9, “The aleatoric uncertainty
σ
2
of the unknown output
y
^
can be determined from the combination of the epistemic uncertainty
σ
1
and the total uncertainty
σ
t
o
t
, for example, according to the relation of Eqns. (3a) and (3b)”; Kadayam Viswanathan, Paragraph 0064, Equation 3a, “
σ
t
o
t
2
=
σ
1
2
+
σ
2
2
”; ).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention to have modified the uncertainty estimation method of the proposed combination to include estimating a total uncertainty of the neural network as taught by Kadayam Viswanathan. The motivation to do so would have been that a total uncertainty can be calibrated to get better results and a combination approach allows for a better understanding of the components of the uncertainty (Kadayam Viswanathan, Paragraph 0065, Lines 1-6, “The total uncertainty
σ
t
o
t
can be calibrated to get better results for the training datasets … One of the key benefits of this approach is a better understanding of the components of the uncertainty”).
Regarding claim 2, the rejection of claim 1 is incorporated, and further, the proposed combination teaches The process of claim 1, wherein the bipartite graph comprises a weight matrix for the layer and data input into the layer (Lacombe, Page 2, Section 2.2, Paragraph 1, “Activation graphs. Let us consider a neural network F and two layers of size
d
l
and
d
l
+
1
respectively, connected by a matrix
W
l
∈
R
d
l
×
d
l
+
1
. One can build a bipartite graph
G
l
whose vertex set is
V
l
⊔
V
l
+
1
with |
V
l
| =
d
l
and |
V
l
+
1
| =
d
l
+
1
, and edge set is
E
l
=
V
l
×
V
l
+
1
. Following [Gebhart et al., 2019], given an instance x
∈
R
d
, one can associate to each edge (i,j)
∈
E
l
the weight
|
W
l
i
,
j
∙
x
l
i
|
, where
x
l
(i) (resp.
W
l
(i, j)) denotes the i-th coordinate of x
∈
R
d
(resp. entry (i, j) of
W
l
∈
R
d
l
×
d
l
+
1
). Intuitively, the quantity
|
W
l
i
,
j
∙
x
l
i
|
encodes how much the observation x “activates” the connection between the i-th unit of the
l
-th layer and the j-th unit of the (
l
+1)-th layer of F. In this way, we obtain a sequence of bipartite graphs
(
G
l
x
,
F
)
l
called the activation graphs of the pair (x, F), whose vertices are
V
l
⊔
V
l
+
1
and edges weights are given by the aforementioned formula”; Lacombe, Page 10, Section B, Paragraph 1, Lines 4-6, “Note that instantiating the SimplexTree (representation of the activation of the graph in Gudhi) from the matrix
(
W
l
i
,
j
∙
x
l
i
)
i
j
”).
Regarding claim 3, the rejection of claim 1 is incorporated.
The proposed combination thus far does not explicitly teach wherein the latent uncertainty comprises a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer.
Gomes teaches wherein the latent uncertainty comprises a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer (Gomes, Page 5, OOD and confidence score, Lines 8-11, and Equation 5, “we propose the Fisher-Rao distance-based OOD detection score FR0(x) for the logits to be the sum of the distances between f(x) and each individual class conditional centroid µy given by Eq. (3) … We denote it by
F
R
0
(
x
)
≜
∑
y
∈
Y
d
F
R
-
L
o
g
i
t
s
q
θ
∙
f
x
,
q
θ
∙
μ
y
”; Gomes, Page 6, Section 3.3, Lines 1-3, “For each layer, we define a set of class-conditional Gaussian distributions with diagonal standard deviation matrix σ (
l
) and class-conditional mean
μ
y
(
l
)
, where y ∈ {1, . . . , C} and
l
is the index of the latent feature”; Gomes, Page 6, Fisher-Rao distance-based feature-wise confidence score, Lines 1-3, and Equations 11 and 12, “We derive a confidence score by applying the Fisher-Rao distance between the test sample x and the closest class-conditional diagonal Gaussian distribution”; “
F
R
l
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
l
,
σ
l
"
;
"
F
R
l
'
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
'
l
,
σ
'
l
”; Gomes, Page 6, Feature Ensemble, Lines 1-2, “To further improve performance, we combine the confidence scores of the logits and the ones from the low-level features through a linear combination”; Gomes, Page 6, Equation 13, “
F
R
x
≜
α
0
F
R
0
x
+
∑
l
α
l
∙
F
R
l
x
+
α
l
'
∙
F
R
l
'
x
”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the uncertainty estimation method of the proposed combination to include a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer as the latent uncertainty as taught by Gomes. The motivation to do so would have been that the Fisher-Rao distance disclosed by Gomes that includes a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer is more effective at distinguishing between out of distribution data and in distribution data (Gomes, Page 4, Section 3.1, Paragraph 2, Lines 9-12, “we expect the Fisher-Rao distance between Gaussian distributions to be more effective in distinguishing between distributions. Figure 1c shows that the Fisher-Rao distance distinguishes better between “In-dist.” and “OOD II” samples, while the other distances fail”).
Regarding claim 4, the rejection of claim 1 is incorporated, and further, the proposed combination teaches wherein the uncertainty of the artificial neural network comprises an out of distribution condition when the topological uncertainty is greater than a first threshold (Lacombe, Page 5, Col 2, Paragraph 2, Lines 3-5, “A new observation x is classified as an OOD sample if TU(x, F) is larger than the q-th quantile of Ttrain”; The “q-th quantile of Ttrain” is considered to be the first threshold)
The proposed combination thus far does not explicitly teach the latent uncertainty is greater than a second threshold.
Gomes teaches the latent uncertainty is greater than a second threshold (Gomes, Page 5, The OOD detector, Lines 1-3, “The detector consists of a threshold-based function for discriminating between in-distribution and OOD data. This threshold δ and parameters are set so that the true positive rate, i.e., the in-distribution samples correctly classified as in-distribution, becomes 95%”).
It would have been obvious, to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the uncertainty estimation method of the proposed combination, to include comparing the latent uncertainty to a threshold as taught by Gomes. The motivation for doing so would have been to ensure that the true positive rate is 95% (Gomes, Page 5, The OOD detector).
Regarding claim 5, the rejection of claim 1 is incorporated, and further, the proposed combination teaches wherein the computing of the persistence diagram comprises:
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
wherein L comprises a number of layers in the artificial neural network; wherein l comprises a particular layer in the artificial neural network; wherein x comprises an input data value; wherein F comprises a description of the artificial neural network; wherein k(x) comprises a class of x derived from processing by the artificial neural network; and wherein Dtrain comprises an average of data used to train the artificial neural network (Lacombe, Page 3, Col 2, Paragraph 2, Lines 1-11, and Equation 1, “Since storing the whole set of training diagrams for each class and each layer
D
l
x
t
r
a
i
n
,
F
:
a
r
g
m
a
x
F
x
t
r
a
i
n
=
k
might be inefficient in practice, we propose to summarize these sets through their respective Frechet means
D
l
,
k
x
t
r
a
i
n
-
. For a new observation
x
∈
R
d
, let k(x)=argmax(F(x)) be its predicted class, and
D
l
x
,
F
l
the corresponding persistence diagrams. The Topological Uncertainty of x is defined to be
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
which is the average distance over layers between the persistence diagrams of the activation graphs of (x; F) and the average diagrams stored from the training set”).
Regarding claim 6, the rejection of claim 1 is incorporated, and further, the proposed combination teaches wherein the estimating the uncertainty of the artificial neural network comprises an out of distribution condition, and comprising verifying the out of distribution condition (Lacombe, Page 5, Col 2, Paragraph 2, Lines 3-5, “A new observation x is classified as an OOD sample if TU(x, F) is larger than the q-th quantile of Ttrain”; Comparing the topological uncertainty to the “q-th quantile of Ttrain” is considered to be “verify[ing] the out of distribution condition”).
Regarding claim 7, the rejection of claim 1 is incorporated, and further, the proposed combination teaches wherein the estimating the uncertainty comprises an in distribution condition and a classification of data input with an acceptable uncertainty (Lacombe, Page 5, Col 2, Paragraph 2, Lines 3-5, “A new observation x is classified as an OOD sample if TU(x, F) is larger than the q-th quantile of Ttrain”; A person of ordinary skill in the art would recognize that an observation would be considered “in distribution” if the topological uncertainty is smaller or equal to the “q-th quantile of Ttrain”).
Regarding claim 8, Lacombe teaches A non-transitory machine-readable medium comprising instructions that when executed by a processor executes a process (Lacombe, Page 10, Section B, Lines 1-4, “Our implementation relies on tensorflow 2 [Abadi et al., 2016] for neural network training and on Gudhi [Maria et al., 2014] for persistence diagrams (MST) computation”; see also Lacombe, Page 10, Section B.1, Datasets description; A person of ordinary skill in the art would recognize that this requires a generic computer, providing evidence for a non-transitory machine-readable medium comprising instructions and a processor) comprising:
determining a topological uncertainty of an artificial neural network (Lacombe, Page 3, Col 2, Paragraph 2, Lines 7-8, and Equation 1, “The Topological Uncertainty of x is defined to be
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
”) by:
forming a bipartite graph between input and output nodes in a layer of the artificial neural network (Lacombe, Page 3, Section 3, Lines 3-4, “Given a trained network F and an observation x, we build a sequence of activation graphs
G
1
x
,
F
,
…
,
G
L
-
1
(
x
,
F
)
”; Lacombe, Page 2, Section 2.2, Paragraph 1, Lines 12-16, “In this way, we obtain a sequence of bipartite graphs
(
G
l
x
,
F
)
l
called the activation graphs of the pair (x, F), whose vertices are
V
l
⊔
V
l
+
1
and edges weights are given by the aforementioned formula”; The “activation graphs” are considered to be the “bipartite graph”); and
generating a persistence diagram as a function of the bipartite graph (Lacombe, Page 3, Section 3, Lines 5-6, “We then compute a MST of each
G
l
x
,
F
, which in turn induces a persistence diagram
D
l
x
,
F
:=
μ
(
G
l
x
,
F
)”).
Lacombe does not explicitly teach determining a latent uncertainty of the artificial neural network nor estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty.
Gomes teaches determining a latent uncertainty of the artificial neural network (Gomes, Page 5, OOD and confidence score, Lines 8-11, and Equation 5, “we propose the Fisher-Rao distance-based OOD detection score FR0(x) for the logits to be the sum of the distances between f(x) and each individual class conditional centroid µy given by Eq. (3) … We denote it by
F
R
0
(
x
)
≜
∑
y
∈
Y
d
F
R
-
L
o
g
i
t
s
q
θ
∙
f
x
,
q
θ
∙
μ
y
”; Gomes, Page 6, Section 3.3, Lines 1-3, “For each layer, we define a set of class-conditional Gaussian distributions with diagonal standard deviation matrix σ (
l
) and class-conditional mean
μ
y
(
l
)
, where y ∈ {1, . . . , C} and
l
is the index of the latent feature”; Gomes, Page 6, Fisher-Rao distance-based feature-wise confidence score, Lines 1-3, and Equations 11 and 12, “We derive a confidence score by applying the Fisher-Rao distance between the test sample x and the closest class-conditional diagonal Gaussian distribution”; “
F
R
l
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
l
,
σ
l
”; “
F
R
l
'
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
'
l
,
σ
'
l
”; Gomes, Page 6, Feature Ensemble, Lines 1-2, “To further improve performance, we combine the confidence scores of the logits and the ones from the low-level features through a linear combination”; Gomes, Page 6, Equation 13, “
F
R
x
≜
α
0
F
R
0
x
+
∑
l
α
l
∙
F
R
l
x
+
α
l
'
∙
F
R
l
'
x
”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention to have modified the uncertainty estimation method of Lacombe to include determining a latent uncertainty as taught by Gomes. The motivation for doing so would have been that combining the confidence scores of the logits and the ones from the low-level features through a linear combination further improve performance of OOD detection (Gomes, Page 6, Feature Ensemble, Lines 1-2).
Lacombe in view of Gomes teaches a topological uncertainty and a latent uncertainty they do not teach estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty.
Kadayam Viswanathan teaches estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty (Kadayam Viswanathan, Paragraph 0064, Lines 6-9, “The aleatoric uncertainty
σ
2
of the unknown output
y
^
can be determined from the combination of the epistemic uncertainty
σ
1
and the total uncertainty
σ
t
o
t
, for example, according to the relation of Eqns. (3a) and (3b)”; Kadayam Viswanathan, Paragraph 0064, Equation 3a, “
σ
t
o
t
2
=
σ
1
2
+
σ
2
2
”; ).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention to have modified the uncertainty estimation method of the proposed combination to include estimating a total uncertainty of the neural network as taught by Kadayam Viswanathan. The motivation to do so would have been that a total uncertainty can be calibrated to get better results and a combination approach allows for a better understanding of the components of the uncertainty (Kadayam Viswanathan, Paragraph 0065, Lines 1-6, “The total uncertainty
σ
t
o
t
can be calibrated to get better results for the training datasets … One of the key benefits of this approach is a better understanding of the components of the uncertainty”).
Regarding claim 9, the rejection of claim 8 is incorporated, and further, the proposed combination teaches wherein the bipartite graph comprises a weight matrix for the layer and data input into the layer (Lacombe, Page 2, Section 2.2, Paragraph 1, “Activation graphs. Let us consider a neural network F and two layers of size
d
l
and
d
l
+
1
respectively, connected by a matrix
W
l
∈
R
d
l
×
d
l
+
1
. One can build a bipartite graph
G
l
whose vertex set is
V
l
⊔
V
l
+
1
with |
V
l
| =
d
l
and |
V
l
+
1
| =
d
l
+
1
, and edge set is
E
l
=
V
l
×
V
l
+
1
. Following [Gebhart et al., 2019], given an instance x
∈
R
d
, one can associate to each edge (i,j)
∈
E
l
the weight
|
W
l
i
,
j
∙
x
l
i
|
, where
x
l
(i) (resp.
W
l
(i, j)) denotes the i-th coordinate of x
∈
R
d
(resp. entry (i, j) of
W
l
∈
R
d
l
×
d
l
+
1
). Intuitively, the quantity
|
W
l
i
,
j
∙
x
l
i
|
encodes how much the observation x “activates” the connection between the i-th unit of the
l
-th layer and the j-th unit of the (
l
+1)-th layer of F. In this way, we obtain a sequence of bipartite graphs
(
G
l
x
,
F
)
l
called the activation graphs of the pair (x, F), whose vertices are
V
l
⊔
V
l
+
1
and edges weights are given by the aforementioned formula”; Lacombe, Page 10, Section B, Paragraph 1, Lines 4-6, “Note that instantiating the SimplexTree (representation of the activation of the graph in Gudhi) from the matrix
(
W
l
i
,
j
∙
x
l
i
)
i
j
”).
Regarding claim 10, the rejection of claim 8 is incorporated.
The proposed combination does not explicitly teach wherein the latent uncertainty comprises a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer.
Gomes teaches wherein the latent uncertainty comprises a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer (Gomes, Page 5, OOD and confidence score, Lines 8-11, and Equation 5, “we propose the Fisher-Rao distance-based OOD detection score FR0(x) for the logits to be the sum of the distances between f(x) and each individual class conditional centroid µy given by Eq. (3) … We denote it by
F
R
0
(
x
)
≜
∑
y
∈
Y
d
F
R
-
L
o
g
i
t
s
q
θ
∙
f
x
,
q
θ
∙
μ
y
”; Gomes, Page 6, Section 3.3, Lines 1-3, “For each layer, we define a set of class-conditional Gaussian distributions with diagonal standard deviation matrix σ (
l
) and class-conditional mean
μ
y
(
l
)
, where y ∈ {1, . . . , C} and
l
is the index of the latent feature”; Gomes, Page 6, Fisher-Rao distance-based feature-wise confidence score, Lines 1-3, and Equations 11 and 12, “We derive a confidence score by applying the Fisher-Rao distance between the test sample x and the closest class-conditional diagonal Gaussian distribution”; “
F
R
l
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
l
,
σ
l
"
;
"
F
R
l
'
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
'
l
,
σ
'
l
”; Gomes, Page 6, Feature Ensemble, Lines 1-2, “To further improve performance, we combine the confidence scores of the logits and the ones from the low-level features through a linear combination”; Gomes, Page 6, Equation 13, “
F
R
x
≜
α
0
F
R
0
x
+
∑
l
α
l
∙
F
R
l
x
+
α
l
'
∙
F
R
l
'
x
”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the uncertainty estimation method of the proposed combination to include a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer as the latent uncertainty as taught by Gomes. The motivation to do so would have been that the Fisher-Rao distance disclosed by Gomes that includes a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer is more effective at distinguishing between out of distribution data and in distribution data (Gomes, Page 4, Section 3.1, Paragraph 2, Lines 9-12, “we expect the Fisher-Rao distance between Gaussian distributions to be more effective in distinguishing between distributions. Figure 1c shows that the Fisher-Rao distance distinguishes better between “In-dist.” and “OOD II” samples, while the other distances fail”).
Regarding claim 11, the rejection of claim 8 is incorporated, and further, the proposed combination teaches wherein the uncertainty of the artificial neural network comprises an out of distribution condition when the topological uncertainty is greater than a first threshold (Lacombe, Page 5, Col 2, Paragraph 2, Lines 3-5, “A new observation x is classified as an OOD sample if TU(x, F) is larger than the q-th quantile of Ttrain”; The “q-th quantile of Ttrain” is considered to be the first threshold).
The proposed combination does not explicitly teach the latent uncertainty is greater than a second threshold.
Gomes teaches the latent uncertainty is greater than a second threshold (Gomes, Page 5, The OOD detector, Lines 1-3, “The detector consists of a threshold-based function for discriminating between in-distribution and OOD data. This threshold δ and parameters are set so that the true positive rate, i.e., the in-distribution samples correctly classified as in-distribution, becomes 95%”).
It would have been obvious, to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the uncertainty estimation method of the proposed combination, to include comparing the latent uncertainty to a threshold as taught by Gomes. The motivation for doing so would have been to ensure that the true positive rate is 95% (Gomes, Page 5, The OOD detector).
Regarding claim 12, the rejection of claim 8 is incorporated, and further, the proposed combination teaches wherein the computing of the persistence diagram comprises:
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
wherein L comprises a number of layers in the artificial neural network; wherein l comprises a particular layer in the artificial neural network; wherein x comprises an input data value; wherein F comprises a description of the artificial neural network; wherein k(x) comprises a class of x derived from processing by the artificial neural network; and wherein Dtrain comprises an average of data used to train the artificial neural network (Lacombe, Page 3, Col 2, Paragraph 2, Lines 1-11, and Equation 1, “Since storing the whole set of training diagrams for each class and each layer
D
l
x
t
r
a
i
n
,
F
:
a
r
g
m
a
x
F
x
t
r
a
i
n
=
k
might be inefficient in practice, we propose to summarize these sets through their respective Frechet means
D
l
,
k
x
t
r
a
i
n
-
. For a new observation
x
∈
R
d
, let k(x)=argmax(F(x)) be its predicted class, and
D
l
x
,
F
l
the corresponding persistence diagrams. The Topological Uncertainty of x is defined to be
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
which is the average distance over layers between the persistence diagrams of the activation graphs of (x; F) and the average diagrams stored from the training set”).
Regarding claim 13, the rejection of claim 8 is incorporated, and further, the proposed combination teaches wherein the estimating the uncertainty of the artificial neural network comprises an out of distribution condition, and comprising verifying the out of distribution condition (Lacombe, Page 5, Col 2, Paragraph 2, Lines 3-5, “A new observation x is classified as an OOD sample if TU(x, F) is larger than the q-th quantile of Ttrain”; Comparing the topological uncertainty to the “q-th quantile of Ttrain” is considered to be “verify[ing] the out of distribution condition”).
Regarding claim 14, the rejection of claim 8 is incorporated, and further, the proposed combination teaches wherein the estimating the uncertainty comprises an in distribution condition and a classification of data input with an acceptable uncertainty (Lacombe, Page 5, Col 2, Paragraph 2, Lines 3-5, “A new observation x is classified as an OOD sample if TU(x, F) is larger than the q-th quantile of Ttrain”; A person of ordinary skill in the art would recognize that an observation would be considered “in distribution” if the topological uncertainty is smaller or equal to the “q-th quantile of Ttrain”).
Regarding claim 15, Lacombe teaches A system comprising: a computer processor; and a memory coupled to the computer processor; (Lacombe, Page 10, Section B, Lines 1-4, “Our implementation relies on tensorflow 2 [Abadi et al., 2016] for neural network training and on Gudhi [Maria et al., 2014] for persistence diagrams (MST) computation”; see also Lacombe, Page 10, Section B.1, Datasets description; A person of ordinary skill in the art would recognize that this requires a generic computer, providing evidence for a non-transitory machine-readable medium comprising instructions and a processor) wherein the computer processor and the memory are operable for
determining a topological uncertainty of the artificial neural network (Lacombe, Page 3, Col 2, Paragraph 2, Lines 7-8, and Equation 1, “The Topological Uncertainty of x is defined to be
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
”) by:
forming a bipartite graph between input and output nodes in a layer of an artificial neural network (Lacombe, Page 3, Section 3, Lines 3-4, “Given a trained network F and an observation x, we build a sequence of activation graphs
G
1
x
,
F
,
…
,
G
L
-
1
(
x
,
F
)
”; Lacombe, Page 2, Section 2.2, Paragraph 1, Lines 12-16, “In this way, we obtain a sequence of bipartite graphs
(
G
l
x
,
F
)
l
called the activation graphs of the pair (x, F), whose vertices are
V
l
⊔
V
l
+
1
and edges weights are given by the aforementioned formula”; The “activation graphs” are considered to be the “bipartite graph”); and
generating a persistence diagram as a function of the bipartite graph (Lacombe, Page 3, Section 3, Lines 5-6, “We then compute a MST of each
G
l
x
,
F
, which in turn induces a persistence diagram
D
l
x
,
F
:=
μ
(
G
l
x
,
F
)”).
Lacombe does not explicitly teach determining a latent uncertainty of the artificial neural network nor estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty.
Gomes teaches determining a latent uncertainty of the artificial neural network (Gomes, Page 5, OOD and confidence score, Lines 8-11, and Equation 5, “we propose the Fisher-Rao distance-based OOD detection score FR0(x) for the logits to be the sum of the distances between f(x) and each individual class conditional centroid µy given by Eq. (3) … We denote it by
F
R
0
(
x
)
≜
∑
y
∈
Y
d
F
R
-
L
o
g
i
t
s
q
θ
∙
f
x
,
q
θ
∙
μ
y
”; Gomes, Page 6, Section 3.3, Lines 1-3, “For each layer, we define a set of class-conditional Gaussian distributions with diagonal standard deviation matrix σ (
l
) and class-conditional mean
μ
y
(
l
)
, where y ∈ {1, . . . , C} and
l
is the index of the latent feature”; Gomes, Page 6, Fisher-Rao distance-based feature-wise confidence score, Lines 1-3, and Equations 11 and 12, “We derive a confidence score by applying the Fisher-Rao distance between the test sample x and the closest class-conditional diagonal Gaussian distribution”; “
F
R
l
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
l
,
σ
l
”; “
F
R
l
'
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
'
l
,
σ
'
l
”; Gomes, Page 6, Feature Ensemble, Lines 1-2, “To further improve performance, we combine the confidence scores of the logits and the ones from the low-level features through a linear combination”; Gomes, Page 6, Equation 13, “
F
R
x
≜
α
0
F
R
0
x
+
∑
l
α
l
∙
F
R
l
x
+
α
l
'
∙
F
R
l
'
x
”).
It would be obvious to a person of ordinary skill in the art, before the effective filing date of the invention to have modified the uncertainty estimation method of Lacombe to include determining a latent uncertainty as taught by Gomes. The motivation for doing so would have been that combining the confidence scores of the logits and the ones from the low-level features through a linear combination further improve performance of OOD detection (Gomes, Page 6, Feature Ensemble, Lines 1-2).
Lacombe in view of Gomes teaches a topological uncertainty and a latent uncertainty they do not teach estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty.
Kadayam Viswanathan teaches estimating the uncertainty of the artificial neural network as a function of the topological uncertainty and the latent uncertainty (Kadayam Viswanathan, Paragraph 0064, Lines 6-9, “The aleatoric uncertainty
σ
2
of the unknown output
y
^
can be determined from the combination of the epistemic uncertainty
σ
1
and the total uncertainty
σ
t
o
t
, for example, according to the relation of Eqns. (3a) and (3b)”; Kadayam Viswanathan, Paragraph 0064, Equation 3a, “
σ
t
o
t
2
=
σ
1
2
+
σ
2
2
”; ).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention to have modified the uncertainty estimation method of the proposed combination to include estimating a total uncertainty of the neural network as taught by Kadayam Viswanathan. The motivation to do so would have been that a total uncertainty can be calibrated to get better results and a combination approach allows for a better understanding of the components of the uncertainty (Kadayam Viswanathan, Paragraph 0065, Lines 1-6, “The total uncertainty
σ
t
o
t
can be calibrated to get better results for the training datasets … One of the key benefits of this approach is a better understanding of the components of the uncertainty”).
Regarding claim 16, the rejection of claim 15 is incorporated.
The proposed combination does not explicitly teach wherein the latent uncertainty comprises a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer.
Gomes teaches wherein the latent uncertainty comprises a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer (Gomes, Page 5, OOD and confidence score, Lines 8-11, and Equation 5, “we propose the Fisher-Rao distance-based OOD detection score FR0(x) for the logits to be the sum of the distances between f(x) and each individual class conditional centroid µy given by Eq. (3) … We denote it by
F
R
0
(
x
)
≜
∑
y
∈
Y
d
F
R
-
L
o
g
i
t
s
q
θ
∙
f
x
,
q
θ
∙
μ
y
”; Gomes, Page 6, Section 3.3, Lines 1-3, “For each layer, we define a set of class-conditional Gaussian distributions with diagonal standard deviation matrix σ (
l
) and class-conditional mean
μ
y
(
l
)
, where y ∈ {1, . . . , C} and
l
is the index of the latent feature”; Gomes, Page 6, Fisher-Rao distance-based feature-wise confidence score, Lines 1-3, and Equations 11 and 12, “We derive a confidence score by applying the Fisher-Rao distance between the test sample x and the closest class-conditional diagonal Gaussian distribution”; “
F
R
l
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
l
,
σ
l
"
;
"
F
R
l
'
x
=
m
i
n
y
∈
Y
d
F
R
-
G
a
u
s
s
x
,
σ
l
,
μ
y
'
l
,
σ
'
l
”; Gomes, Page 6, Feature Ensemble, Lines 1-2, “To further improve performance, we combine the confidence scores of the logits and the ones from the low-level features through a linear combination”; Gomes, Page 6, Equation 13, “
F
R
x
≜
α
0
F
R
0
x
+
∑
l
α
l
∙
F
R
l
x
+
α
l
'
∙
F
R
l
'
x
”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the uncertainty estimation method of the proposed combination to include a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer as the latent uncertainty as taught by Gomes. The motivation to do so would have been that the Fisher-Rao distance disclosed by Gomes that includes a function of a centroid for a latent representation of a class, a standard deviation of the latent representation of the class, and a latent representation of input data into the layer is more effective at distinguishing between out of distribution data and in distribution data (Gomes, Page 4, Section 3.1, Paragraph 2, Lines 9-12, “we expect the Fisher-Rao distance between Gaussian distributions to be more effective in distinguishing between distributions. Figure 1c shows that the Fisher-Rao distance distinguishes better between “In-dist.” and “OOD II” samples, while the other distances fail”).
Regarding claim 17, the rejection of claim 15 is incorporated, and further, the proposed combination teaches, wherein the uncertainty of the artificial neural network comprises an out of distribution condition when the topological uncertainty is greater than a first threshold (Lacombe, Page 5, Col 2, Paragraph 2, Lines 3-5, “A new observation x is classified as an OOD sample if TU(x, F) is larger than the q-th quantile of Ttrain”; The “q-th quantile of Ttrain” is considered to be the first threshold).
The proposed combination does not explicitly teach the latent uncertainty is greater than a second threshold.
Gomes teaches the latent uncertainty is greater than a second threshold (Gomes, Page 5, The OOD detector, Lines 1-3, “The detector consists of a threshold-based function for discriminating between in-distribution and OOD data. This threshold δ and parameters are set so that the true positive rate, i.e., the in-distribution samples correctly classified as in-distribution, becomes 95%”).
It would have been obvious, to a person of ordinary skill in the art, before the effective filing date of the invention, to have modified the uncertainty estimation method of the proposed combination, to include comparing the latent uncertainty to a threshold as taught by Gomes. The motivation for doing so would have been to ensure that the true positive rate is 95% (Gomes, Page 5, The OOD detector).
Regarding claim 18, the rejection of claim 15 is incorporated, and further, the proposed combination teaches wherein the computing of the persistence diagram comprises:
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
wherein L comprises a number of layers in the artificial neural network; wherein l comprises a particular layer in the artificial neural network; wherein x comprises an input data value; wherein F comprises a description of the artificial neural network; wherein k(x) comprises a class of x derived from processing by the artificial neural network; and wherein Dtrain comprises an average of data used to train the artificial neural network (Lacombe, Page 3, Col 2, Paragraph 2, Lines 1-11, and Equation 1, “Since storing the whole set of training diagrams for each class and each layer
D
l
x
t
r
a
i
n
,
F
:
a
r
g
m
a
x
F
x
t
r
a
i
n
=
k
might be inefficient in practice, we propose to summarize these sets through their respective Frechet means
D
l
,
k
x
t
r
a
i
n
-
. For a new observation
x
∈
R
d
, let k(x)=argmax(F(x)) be its predicted class, and
D
l
x
,
F
l
the corresponding persistence diagrams. The Topological Uncertainty of x is defined to be
T
U
x
,
F
≔
1
L
∑
l
=
1
L
D
i
s
t
(
D
l
x
,
F
,
D
l
,
k
x
t
r
a
i
n
-
)
which is the average distance over layers between the persistence diagrams of the activation graphs of (x; F) and the average diagrams stored from the training set”).
Regarding claim 19, the rejection of claim 15 is incorporated, and further, the proposed combination teaches wherein the estimating the uncertainty of the artificial neural network comprises an out of distribution condition, and comprising verifying the out of distribution condition (Lacombe, Page 5, Col 2, Paragraph 2, Lines 3-5, “A new observation x is classified as an OOD sample if TU(x, F) is larger than the q-th quantile of Ttrain”; Comparing the topological uncertainty to the “q-th quantile of Ttrain” is considered to be “verify[ing] the out of distribution condition”).
Regarding claim 20, the rejection of claim 15 is incorporated, and further, the proposed combination teaches wherein the estimating the uncertainty comprises an in distribution condition and a classification of data input with an acceptable uncertainty (Lacombe, Page 5, Col 2, Paragraph 2, Lines 3-5, “A new observation x is classified as an OOD sample if TU(x, F) is larger than the q-th quantile of Ttrain”; A person of ordinary skill in the art would recognize that an observation would be considered “in distribution” if the topological uncertainty is smaller or equal to the “q-th quantile of Ttrain”).
Response to Arguments
Applicant’s amendments to claims 6, 8, 13, and 19 with respect to 35 U.S.C. 112(b) indefiniteness rejections to the claims have been fully considered, and overcome the rejections set forth in the nonfinal office action dated 03/12/2026. Consequently, the 35 U.S.C. 112(b) indefiniteness rejections to the claims have been withdrawn.
Applicant’s arguments regarding the 35 U.S.C. 101 rejections of the claims have been fully considered but are unpersuasive.
Applicant first argues, in pages 8-9, Section A of the response, that the claimed invention is “not merely a mathematical concept” and the steps are “not performed in the abstract, but rather on concrete data structures within a neural network implemented on a computer processor and memory”. Examiner respectfully disagrees. Mere physical or tangible implementation of an exception is not in itself an inventive concept and does not guarantee eligibility, see MPEP 2106.05(I)(A). Further, the data structures being within a neural network is an additional element that generally links the use of the judicial exception to a particular technological environment or field of use. Applicant next points to Enfish, LLC v. Microsoft Corp on page 9 of the response, arguing the invention is not abstract because it improves the ability of neural networks to detect uncertainty and out-of-distribution data. Examiner respectfully disagrees. An improvement to uncertainty and out-of-distribution data detection may be an improvement in an abstract idea, but not an improvement in the functioning of a computer, as a computer. Further, an improvement in the abstract idea itself (e.g. a recited mathematical concept) is not an improvement in technology, see MPEP 2106.05(a)(II). Applicant next points to McRO, Inc. v. Bandai Namco Games America Inc. on page 9 of the response, and argues that because the claims automate the detection and quantification of uncertainty in neural networks, the claims are patent eligible. Examiner respectfully disagrees. In McRO, the court relied on the specification’s explanation of how the particular rules recited in the claim enabled the automation of specific animation tasks that previously could only be performed subjectively by humans, see MPEP 2106.05(a); thus, the “particular rules recited in the claim” demonstrated the improvement. Additionally, the fact pattern of this case is not identical to that of the instant case, and thus the same rationale cannot be applied. In the instant case, an improvement to uncertainty quantification and out-of-distribution data detection may be an improvement in an abstract idea, but not an improvement in the functioning of a computer, as a computer. Further, an improvement in the abstract idea itself (e.g. a recited mathematical concept) is not an improvement in technology, see MPEP 2106.05(a)(II).
Applicant next argues, on pages 12-13, Section B of the response, that the claims do not “preempt all uses of mathematical concepts for uncertainty estimation” but rather recite a specific combination of steps applied to neural network architecture. Examiner respectfully disagrees. Estimating uncertainty, according to applicant’s specification paragraph 0011, “the edge weights of the bipartite graph are a function of the weight matrix for the particular layer, and the data that are input into that particular layer” and the paragraph goes on to disclose the persistence diagram equation, which a person of ordinary skill in the art would recognize as math. Applicant next argues in page 10, bullet 4 of the response, that the “present invention addresses the technical problem of uncertainty estimation in neural networks”. Examiner respectfully disagrees. An improvement to uncertainty estimation may be an improvement in an abstract idea, but not an improvement in the functioning of a computer, as a computer. Further, an improvement in the abstract idea itself (e.g. a recited mathematical concept) is not an improvement in technology, see MPEP 2106.05(a)(II).
Applicant next argues, on pages 10-11, Section III of the response, that the claims recite significantly more than the judicial exception. Examiner respectfully disagrees. On page 10, Section 3, Bullet 1, Applicant asserts the claims recite specific structural elements, and while this is true, the mere physical or tangible implementation of an exception is not in itself an inventive concept and does not guarantee eligibility, see MPEP 2106.05(I)(A). On page 10, Section 3, Bullet 2, Applicant asserts the specification describes how the claimed process improves the accuracy and reliability of neural network predictions. An improvement to predicting may be an improvement in an abstract idea, but not an improvement in the functioning of a computer, as a computer; the use of a neural network generally links an abstract idea to a technological environment or field of use. With regard to BASCOM Global Internet Services, Inc. v. AT&T Mobility LLC, and Thales Visionix Inc. v. United States, the fact patterns of these cases are not identical to that of the instant case, and therefore the same rationale cannot be applied. Applicant argues the present claims recite a specific arrangement of topological and latent uncertainty measures; Examiner respectfully disagrees. While the claim determines a topological uncertainty and a latent uncertainty and then estimates an uncertainty of the neural network as a function of the topological and latent uncertainty, this does not constitute a specific arrangement. The broadest reasonable interpretation of the independent claims includes any method of determining the latent uncertainty of the neural network and any function of the topological and latent uncertainties. Applicant makes no direct arguments as to how the claims recite a “specific arrangement”. Applicant argues “claims that use mathematical formulas in a specific application to solve a technical problem are patent eligible”. However, an improvement in the abstract idea itself (e.g. a recited mathematical concept) is not an improvement in technology, see MPEP 2106.05(a)(II).
Applicant's arguments regarding the remainder of the claims rely upon the arguments asserted with respect to the independent claims, and are thus unpersuasive.
Applicant’s arguments regarding the 35 U.S.C. 103 rejections of the claims have been fully considered but are unpersuasive.
Applicant first argues, on page 12 of the response, that the proposed combination fails to teach or suggest the specific, integrated framework recited in the claims. Examiner respectfully disagrees. Applicant specifically argues what Lacombe and Gomes teach and do not teach in sections II and III on pages 13 and 14 of the response. In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Applicant next argues, on page 14, section IV that “neither Lacombe nor Gomes suggests fusing topological TU with an independent latent metric to produce a single uncertainty estimate. In the non-final office action dated 12/16/2025, Kadayam Viswanathan is relied upon to teach the referenced limitation. Applicant argues on page 15, Section B, Bullet 1 that the claims require estimating uncertainty “as a function of” both components, which is not taught by either reference. In the non-final office action dated 12/16/2025, Kadayam Viswanathan is relied upon to teach the referenced limitation. Applicant argues on page 15, Section B, Bullet 2, that the claims require that topological uncertainty be greater than a first threshold and that latent uncertainty be greater than a second threshold, and that neither reference teaches a “two-dimensional thresholding”. Lacombe teaches a first threshold, Gomes teaches a second threshold, and proper rationale has been provided for combining the references as required by the MPEP; further, a person of ordinary skill in the art is also a person of ordinary creativity, not an automation; in many cases a person of ordinary skill will be able to fit the teachings of multiple patents together like pieces of a puzzle, see MPEP 2141.03(I). Applicant argues on page 15, Section B, Bullet 3, that Lacombe does not teach that the bipartite graph comprises both the weight matrix and input data because Lacombe teaches activation-derived weights. Examiner respectfully disagrees. The broadest reasonable interpretation of claims 2/9/15 includes activation-derived weights, as it does not place any limitation on the weight matrix or data input used in the bipartite graph.
Applicant next argues claim by claim distinctions in pages 15-16, section V of the response. Applicant argues on page 15, section V, bullet 1, that neither reference nor their combination teaches “as a function of the topological uncertainty and the latent uncertainty”, however, in the non-final office action dated 12/16/2025, Kadayam Viswanathan is relied upon to teach the referenced limitation. Applicant argues on page 15, section V, bullet 2, that the bipartite graph comprises a weight matrix and data input which is not taught by the references. Examiner respectfully disagrees. The bipartite graphs formed in Lacombe comprise “instance
x
∈
R
d
” which is considered to be the “input data” as well as matrix vertices (Lacombe, Page 2, Section 2.2, Activation Graphs). Applicant argues on pages 15-16, section V, bullet 3, that Gomes does not teach determining latent uncertainty in the claimed functional form nor combine it with topological uncertainty for overall estimation, however, in the non-final office action dated 12/16/2025, Kadayam Viswanathan is relied upon to teach the referenced combination limitation. Further, Gomes does teach latent uncertainty in a functional form, see Gomes, Page 6, Equation 13. Applicant argues on page 16, section V, bullet 1, that neither reference discloses conjunctive thresholding across two heterogeneous uncertainty sources. Lacombe teaches a first threshold, Gomes teaches a second threshold, and proper rationale has been provided for combining the references as required by the MPEP; further, a person of ordinary skill in the art is also a person of ordinary creativity, not an automation; in many cases a person of ordinary skill will be able to fit the teachings of multiple patents together like pieces of a puzzle, see MPEP 2141.03(I). Applicant argues on page 16, section V, bullet 2, that Lacombe only partially teaches the persistence-diagram computation claimed in claims 5/12/18. Examiner respectfully disagrees. Each parameter claimed has been properly mapped to Lacombe and each parameter in the reference represents the same thing as recited in the instant claim.
Applicant next argues on page 16, section VI of the response, that the office action’s mapping relies on hindsight. In response to applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MOLLY CLARKE SIPPEL whose telephone number is (571)272-3270. The examiner can normally be reached Monday - Friday, 7:30 a.m. - 4:30 p.m. ET..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kakali Chaki can be reached at (571)272-3719. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.C.S./Examiner, Art Unit 2122
/KAKALI CHAKI/Supervisory Patent Examiner, Art Unit 2122
Prosecution Insights