DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgement is made of the applicant’s claim for Foreign priority to German Patent Application No. DE10/20222030346 filed on March 28, 2022.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-5 and 8-9 are rejected under 35 U.S.C. §112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor regards as the invention. The claims recite mathematical symbols and expressions (
σ
z
2
,
μ
z
,
μ
y
,
D
c
,
z
,
y
,
p
z
D
c
,
a
n
d
N
(
z
|
μ
z
,
σ
z
2
) that lack antecedent basis or clear definition in the claims. The meaning of these variables and functions is not apparent from the claimed language, and one of ordinary skill in the art would not understand the metes and bounds of the invention with reasonable clarity. See MPEP 2173.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-11 are rejected under 35 U.S.C. 101 because the claimed invention is directed
to a judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more.
101 Subject Matter Eligibility Analysis
Step 1: Claims 1-11 are within the four statutory categories (a process, machine, manufacture or composition of matter).
Claims 1-6, and 11 are directed to a method consisting of a series of steps, meaning that it is directed to the statutory category of process.
Claims 7-10, are written in the form of a “machine” claim (e.g., an architecture or device including a neural network. However, the claims do not recite any structural or hardware (e.g., processor, memory, or tangible medium) and instead describe algorithmic functionality and relationships among abstract parameters. Accordingly, although presented in machine form, claims 7-10 fail to define a statutory machine and instead directed to software per se, which is non-statutory subject matter under 35 U.S.C. 101. See MPEP 2106 (Step 1).
The Step 2A/2B analysis below is provided for all claims for completeness.
Step 2A Prong One, Step 2A Prong Two, and Step 2B Analysis:
Step 2A Prong One asks if the claim recites a judicial exception (abstract idea, law of nature, or natural phenomenon). If the claim recites a judicial exception, analysis proceeds to Step 2A Prong Two, which asks if the claim recites additional elements that integrate the abstract idea into a practical application. If the claim does not integrate the judicial exception, analysis proceeds to Step 2B, which asks if the claim amounts to significantly more than the judicial exception. If the claim does not amount to significantly more than the judicial exception, the claim is not eligible subject matter under 35 U.S.C. 101.
None of the claims represent an improvement to technology.
Regarding claim 1, the following claim elements are abstract ideas:
assessing uncertainties in a model using a neural network including a neural process, the model modeling a technical system and/or a system behavior of the technical system (This is an abstract idea of a “mental process.” The limitation recites observation and evaluation of uncertainty associated with a model of a system. Such assessing” constitutes an act of judgement – reviewing model behavior, recognizing its level of confidence or error, and deciding how uncertain it is. A personally could mentally examine model predictions, compare them to expected outcomes, and reason about how reliable the model is. These activities – observation, comparison, and evaluation – can be performed in the human mind or with pen and paper and therefore falls within the mental-process grouping of abstract ideas. See MPEP 2106.04(a)(2)(III).),
determining a model uncertainty
σ
z
2
(This is an abstract idea of a mathematical concept and alternatively, a mental process. It recites computing a statistical variance (
σ
2
) for a model parameter – i.e., applying known formulas/relationships among numerical values. Such calculation can be carried out mentally with basic computing tools like a paper worksheet, a hand held calculator, or a simple spreadsheet (e.g., summing squared deviations and dividing by N or N-1). Because it amounts to performing math and evaluation that can be done in the human mind or with pen-and-paper/basic tools, it falls within the abstract idea groupings of mathematical relationships/formulas and mental processes. See MPEP 2106.04(a)(a)(2)(I) and MPEP 2106.04(a)(2(III).);
determining a variance of an output of the model
σ
y
2
based on the model uncertainty (This is an abstract idea of a mathematical concept and mental process. It recites computing a statistical variance for the model’s output as a function of previously obtained uncertainty values – a mathematical relationship among numerical parameters. Such computation can be performed mentally using basic computing tools such as a calculator, spreadsheet or pen and paper, to apply the standard variance formula (e.g., summing squared deviations or propagating error terms). Because it involves mathematical analysis and evaluation that can be performed in the human mind with rudimentary tools, it falls within the mathematical concept and mental process groupings of abstract ideas.
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
a neural network including a neural process (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
a technical system (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
at least one decoder section trained
Regarding claim 2, the rejection of claim 1 is incorporated herein. Further, claim 2 recites the following abstract ideas:
wherein the model uncertainty is quantified by a variance of a latent spatial distribution
p
(
z
|
D
c
)
(This is abstract idea of a mathematical concept and mental process. It recites expressing model uncertainty as the variance of a probability distribution, which constitutes a mathematical relationship among statistical variables. Such calculation – determining the spread of a latent distribution
p
(
z
|
D
c
)
can be performed mentally with basic computing tools, such as a calculator, spreadsheet, or pen and paper, by applying standard variance and probability formulas. Because it involves mathematical evaluation and reasoning that can be executed conceptually or with rudimentary aid, it falls within the mathematical concept and mental process groupings of abstract ideas.).
Regarding claim 3, the rejection of claim 1 is incorporated herein. Further, claim 3 recites the following abstract ideas:
wherein the model uncertainty is calculated as a variance
σ
z
2
of a Gaussian distribution, where
σ
z
2
=
σ
z
2
(
D
c
)
, via a latent variable z from a set of contexts
D
c
of observations, wherein
p
z
D
c
=
N
(
z
|
μ
z
D
c
,
σ
z
2
D
c
)
(This is an abstract idea of a mathematical concept. The limitation recites a Gaussian probability distribution and specifies the computation of a variance and mean value
μ
z
(
D
c
)
based on contextual observations. Such formulation and parameterization of a Gaussian distribution represent the use of mathematical relationships, formulas, and statistical functions. Because it is directed to expressing data using mathematical symbols and equations that define a probability distribution, it falls within the mathematical concept grouping of abstract ideas.
Regarding claim 4, the rejection of claim 3 is incorporated herein. Further, claim 4 recites the following abstract ideas:
wherein a mean value
μ
z
of the Gaussian distribution, where
μ
z
=
μ
z
(
D
c
)
,
is calculated via the latent variable z from the set of contexts
D
C
of observations, wherein
p
z
D
c
=
N
(
z
|
μ
z
D
c
,
σ
z
2
D
c
)
(This is an abstract idea of a mathematical concept. The limitation recites calculating a mean value of a Gaussian probability distribution based on contextual observations, which constitutes the use of a mathematical formula and statistical relationships to define a distribution. Expressing or computing such mean and variance parameters involves applying mathematical equations to numerical data and therefore falls within the mathematical relationships/formula grouping of abstract ideas.).
Regarding claim 5, the rejection of claim 1 is incorporated herein. Further, claim 5 recites the following abstract ideas:
wherein a mean value
μ
y
of the output is calculated (This is an abstract idea of a mathematical concept and mental process. It recites calculating the mean (average) value for the model output, which is a mathematical operation involving summing a set of numerical values and dividing by their count. Such an operation can be performed mentally or with basic computing tools such as a calculator, spreadsheet, or pen and paper. Because it involves applying a mathematical formula and reasoning steps that can be carried out in the human mind or with rudimentary tools, it falls within the mathematical relationship/formula and mental process groupings of abstract ideas.).
Regarding claim 6, the rejection of claim 1 is incorporated herein. Further, claim 6 recites the following abstract ideas:
wherein an uncertainty of the neural network is predicted based on the model uncertainty and on the variance of the output (This is an abstract idea of a mathematical concept and mental process. It recites computing or predicting an overall uncertainty value by combing previously determined statistical quantities – namely, model uncertainty and output variance – through mathematical relationships. Such prediction can be carried out mentally or with basic computing tools such as a calculator, spreadsheet, or pen and paper by applying standard mathematical operations (e.g., addition, weighting, propagation of variance). Because it involves applying mathematical formulas and evaluative reasoning that can be performed in the human mind or with rudimentary aids, it falls within the mathematical relationship/formula and mental process groupings of abstract ideas.).
Regarding claim 7, the following claim elements are abstract ideas:
assessing uncertainties in a model using a neural network including a neural process, the model modeling a technical system and/or a system behavior of the technical system (This is an abstract idea of a “mental process.” The limitation recites observation and evaluation of uncertainty associated with a model of a system. Such assessing” constitutes an act of judgement – reviewing model behavior, recognizing its level of confidence or error, and deciding how uncertain it is. A personally could mentally examine model predictions, compare them to expected outcomes, and reason about how reliable the model is. These activities – observation, comparison, and evaluation – can be performed in the human mind or with pen and paper and therefore falls within the mental-process grouping of abstract ideas. See MPEP 2106.04(a)(2)(III).),
at least one decoder section trained to determine a variance of an output of the model based on a model uncertainty (This is an abstract idea of a mental process and mathematical concept. It recites determining a variance of a model output as a function of model uncertainty – a mathematical relationship between statistical parameters. Such calculation can be performed in the human mind with basic computing tools such a calculator, spreadsheet, or pen and paper by applying standard variance or error-propagation formulas. The recitation of a “decoder section trained” is mere instructions to apply the abstract idea using generic computer functionality and therefore represents insignificant extra-solution activity and a high-level recitation of generic computer components.).
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
a neural network including a neural process (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
a technical system (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
Regarding claim 8, the rejection of claim 7 is incorporated herein. Further, claim 8 recites the following abstract ideas:
wherein the neural network includes at least one encoder section, the encoder section being trained to determine the model uncertainty as a variance
σ
z
2
and/or a mean value
μ
2
of a Gaussian distribution via a latent variable z from a set of contexts
D
c
of observations, where
p
z
D
c
=
N
(
z
|
μ
z
,
σ
z
2
)
(This is an abstract idea of a mathematical concept. The limitation recites the formulation of a Gaussian probability distribution and the computation of its parameters – mean and variance – based on contextual data. These operations represent the use of mathematical relationships, equations, and statistical functions to characterize data and therefore amount to the manipulation of mathematical symbols and statistical formulas.)
Regarding claim 9, the rejection of claim 7 is incorporated herein. Further, claim 9 recites the following abstract ideas:
wherein the neural network includes at least one further decoder section, the further decoder section being trained to determine a mean value
μ
y
of the output based on an input point x and on a latent sample z (This is an abstract idea of a mathematical concept. The limitation recites determining a mean value of an output as a mathematical function of an input x and a latent variable z, which represents a mathematical relationship among numerical variables. Such formulation and evaluation of a mean value constitute mathematical computation and expression of a function, which fall within the mathematical relationship/formula grouping of abstract ideas. The recitation of a “decoder section being trained” is mere instructions to apply the abstract idea using generic computer functionality and therefore represents insignificant extra solution activity and a high-level recitation of generic computer components.)
Regarding claim 10, the following claim elements are abstract ideas:
assessing uncertainties in a model using a neural network including a neural process, the model modeling a technical system and/or a system behavior of the technical system (This is an abstract idea of a “mental process.” The limitation recites observation and evaluation of uncertainty associated with a model of a system. Such assessing” constitutes an act of judgement – reviewing model behavior, recognizing its level of confidence or error, and deciding how uncertain it is. A personally could mentally examine model predictions, compare them to expected outcomes, and reason about how reliable the model is. These activities – observation, comparison, and evaluation – can be performed in the human mind or with pen and paper and therefore falls within the mental-process grouping of abstract ideas. See MPEP 2106.04(a)(2)(III).),
at least one decoder section trained to determine a variance of an output of the model based on a model uncertainty (This is an abstract idea of a mental process and mathematical concept. It recites determining a variance of a model output as a function of model uncertainty – a mathematical relationship between statistical parameters. Such calculation can be performed in the human mind with basic computing tools such a calculator, spreadsheet, or pen and paper by applying standard variance or error-propagation formulas. The recitation of a “decoder section trained” is mere instructions to apply the abstract idea using generic computer functionality and therefore represents insignificant extra-solution activity and a high-level recitation of generic computer components.).
The following claim elements are additional elements which, taken alone or in combination with the other elements, do not integrate the judicial exception into a practical application nor amount to significantly more than the judicial exception:
a neural network including a neural process (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
a technical system (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
A device (This a high-level recitation of generic computer components for performing the abstract idea. See MPEP 2106.05(f).)
Regarding claim 11, the rejection of claim 1 is incorporated herein. Further, claim 11 recites the following abstract ideas:
ascertaining an inadmissible deviation of the system behavior of the technical system from a standard value range based on the variance of the output of the model (This is an abstract idea of a mathematical concept and mental process. It recites evaluating where the model’s output deviates from an expected or standard range by comparing a calculated variance to predefined thresholds – an act of mathematical comparison and judgement. Such analysis can be performed in the human mind with basic computing tools such as a calculator, spreadsheet, or pen and paper by applying standard deviation and range-comparison formulas. Because it involves mathematical evaluation and decision-making that can be performed in the human mind or with rudimentary aids, it falls within the mathematical relationships/formulas and mental process groupings of abstract ideas.).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 7-10 are rejected under § 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 7 recites an “architecture of a neural network” configured to determine a variance of an output of a model based on a model uncertainty. The claim does not recite any tangible components such a process, memory, or computer-readable medium, and instead merely describes an arrangement of functional elements (i.e., a neural network and its architecture) at an abstract, conceptual level. As such, the claim amounts to a description of software per se or an algorithmic design without any physical embodiment or transformation. Accordingly, claim 7 is non-statutory under 35 U.S.C. §101 for being directed to software per se.
Claims 8 and 9 depend from claim 7 and therefore also fail to define any tangible structure or statutory category. The additional limitations merely elaborate on functional relationships of the same disembodied architecture (e.g., encoder or decoder sections trained to determine statistical parameters) without introducing any physical hardware or transformation. Accordingly, claims 8 and 9 are likewise directed to software per se and are non-statutory under 35 U.S.C. §101.
Claim 10 recites a “device” that includes a neural network configured for accessing uncertainties in a model, wherein the neural network includes at least one decoder section trained to determine a variance of an output of the model based on a model uncertainty. Although the claim is nominally directed to a “device,” it does not recite any tangible hardware elements such as a processor, memory, or storage medium. Instead, it merely describes the functional behavior of a neural network in abstract terms without specifying any structural implementation. As such, the claim encompasses only a disembodied algorithmic construct rather than a statutory machine or manufacture. Accordingly, claim 10 is non-statutory under 35 U.S.C. §101 for being directed to software per se.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-9, and 11 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Kim et. al. (NPL: “Attentive Neural Processes” (Published: 2019)).
Regarding claim 1, Kim discloses:
An architecture of a neural network including a neural process, the neural network being configured for assessing uncertainties in a model, the model modeling a technical system and/or a system behavior of the technical system, the neural network comprising (Kim, Introduction, page 2, “In theory, increasing the dimensionality of the representation could address this issue, but we show in Section 4 that in practice, this is not sufficient. To address this issue, we draw inspiration from GPs, which also define a family of conditional distributions for regression…We implement a similar mechanism in NPs using differentiable attention that learns to attend to the contexts relevant to the given target, while preserving the permutation invariance in the contexts. We evaluate the resulting Attentive Neural Processes (ANPs) on 1D function regression and on 2D image regression. Our results show that ANPs greatly improve upon NPs in terms of reconstruction of contexts as well as speed of training, both against iterations and wall clock time. We also demonstrate that ANPs show enhanced expressiveness relative to the NP and is able to model a wider range of functions.” – describes how the Attentive Neural Process uses differentiable attention within a neural network to improve predictions and capture broader range of functions, thereby enhancing how the model represents and evaluates model uncertainty in its learned relationships between inputs and outputs.”):
determining a model uncertainty
σ
z
2
; and determining a variance of an output of the model
σ
y
2
based on the model uncertainty. (Kim, section 2.1, equation 2, “The likelihood
p
y
T
x
T
,
r
C
is modelled by a Gaussian factorized across the targets…with mean and variance given by passing
x
i
and
r
C
through an MLP.” and “The latent variable version of the NP model includes a global latent z to account for uncertainty in the predictions of
y
T
for a given observed (
(
x
C
,
y
C
)
. It is incorporated into the model via a latent path that complements the deterministic path described above. Here
z
is modelled by a factorized Gaussian parametrised by
s
C
≔
s
x
C
,
y
C
,
with s being a function of the same properties as r with
q
z
s
ϕ
≔
p
z
,
the prior on
z
The likelihood is referred to as the
d
e
c
o
d
e
r
, and
q
,
r
,
s
form the
e
n
c
o
d
e
r
…The motivation for having a global latent is to model different realisations of the data generating stochastic process” – the model’s uncertainty is determined by the encoder through a latent path introducing a global latent variable z, modeled by the factorized Gaussian distribution, where the variance quantifies the model uncertainty. The output variance is then determined by the decoder, which defines a Gaussian likelihood whose variance depends on the latent variable z. The final predictive distribution integrates over this latent Gaussian, showing the output variance is computed directly based on the model uncertainty.).
Regarding claim 2, Kim discloses:
The method as recited in claim 1, wherein the model uncertainty is quantified by a variance of a latent spatial distribution
p
(
z
|
D
C
)
(Kim, section 2.1, equation 2 “The likelihood
p
y
T