7DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This action is made non-final.
The preliminary amendment of claims 1-9 filed on 09/12/2024 has been reviewed and considered by this office action.
Information Disclosure Statement
The information disclosure statement filed on 09/12/2024 has been reviewed and considered by this office action.
Drawings
The drawings filed on 09/12/2024 have been reviewed and are considered acceptable.
Specification
The specification and amendments to the specification filed on 09/12/2024 have been reviewed and are considered acceptable.
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: “Dynamic Sample-Size Optimization for Bayesian Updating.”
Applicant is reminded of the proper content of an abstract of the disclosure.
A patent abstract is a concise statement of the technical disclosure of the patent and should include that which is new in the art to which the invention pertains. The abstract should not refer to purported merits or speculative applications of the invention and should not compare the invention with the prior art.
If the patent is of a basic nature, the entire technical disclosure may be new in the art, and the abstract should be directed to the entire disclosure. If the patent is in the nature of an improvement in an old apparatus, process, product, or composition, the abstract should include the technical disclosure of the improvement. The abstract should also mention by way of example any preferred modifications or alternatives.
Where applicable, the abstract should include the following: (1) if a machine or apparatus, its organization and operation; (2) if an article, its method of making; (3) if a chemical compound, its identity and use; (4) if a mixture, its ingredients; (5) if a process, the steps.
Extensive mechanical and design details of an apparatus should not be included in the abstract. The abstract should be in narrative form and generally limited to a single paragraph within the range of 50 to 150 words in length.
See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
Claim Objections
Claim 5 is objected to because of the following informalities:
In claim 5, the language “processing the belief distribution which has been updated… in a given step, for a process to be performed in a next step” is difficult to parse. Examiner interprets claim 5 to mean “wherein the at least one processor processes the updated belief distribution from a current iteration to generate an initial belief distribution for a next iteration of the optimal variable candidate generation process, the objective function evaluation process, the inverse temperature optimization process, the weight evaluation process, and the belief distribution updating process,” as supported by [0117] of Applicant’s specification.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Applicant may wish to consider amending the independent claims to more clearly reflect the physical control implementation disclosed, for example, in paragraphs [0082]-[0087] of Applicant’s specification.
Merely transmitting a calculated control input to a control target, as recited in claim 4, does not appear to integrate the optimization into a practical application. An amendment could instead require the calculated control input to cause the control target to operate, rather than merely be transmitted to it, and could further tie that operation to the optimization by using an observed state of the control target in evaluating the objective function or in a subsequent optimization process.
Step 1: Claims 1-5 and 9 are directed to a machine or an article of manufacture. Claims 6-8 are directed to a process.
With respect to claim 1:
2A Prong 1: The claim recites an abstract idea. Specifically:
an optimal variable candidate generation process of generating a plurality of optimal variable candidates, based on a belief distribution; (Mental process – generating a plurality of optimal variable candidates is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
an objective function evaluation process of evaluating an objective function for each of the plurality of optimal variable candidates; (Mental process – evaluating an objective function is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
an inverse temperature optimization process of calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other; (Mental process – calculating an inverse temperature using an optimization technique is an evaluation using an optimization technique that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
a weight evaluation process of calculating the weight for the objective function, based on the inverse temperature (Mental process – calculating the weight for an objective function is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
at least one processor, the at least one processor being configured to carry out: (Mere recitation of a generic computer component – see MPEP § 2106.05(b)(I))
a belief distribution updating process of updating the belief distribution, based on the weight, the belief distribution, and each of the optimal variable candidates (Insignificant extra-solution activity – updating a belief distribution represents post-solution activity – see MPEP § 2106.05(g))
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
at least one processor, the at least one processor being configured to carry out: (Mere recitation of a generic computer component – see MPEP § 2106.05(b)(I))
a belief distribution updating process of updating the belief distribution, based on the weight, the belief distribution, and each of the optimal variable candidates (Performing repetitive calculations has been deemed a well‐understood, routine, and conventional function – see MPEP § 2106.05(d)(ll))
Therefore, the claim is ineligible.
With respect to claim 2:
2A Prong 1: The claim recites an abstract idea. Specifically:
wherein in the optimal variable candidate generation process, the at least one processor generates the plurality of optimal variable candidates, based on an initial belief distribution which has been inputted or on the belief distribution which has been updated in the belief distribution updating process (Mental process – generating a plurality of optimal variable candidates is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
Therefore, the claim is ineligible.
With respect to claim 3:
2A Prong 1: The claim recites an abstract idea. Specifically:
wherein in the objective function evaluation process, the at least one processor evaluates, for each of the plurality of optimal variable candidates, an objective function which depends on a state of a control target that is observed by a state observation apparatus (Mental process – evaluating an objective function is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
Therefore, the claim is ineligible.
With respect to claim 4:
2A Prong 1: The claim recites an abstract idea. Specifically:
wherein the at least one processor further carries out a control input conversion process of calculating control input in accordance with a predetermined conversion rule based on the belief distribution which has been updated in the belief distribution updating process and (Mental process – calculating a control input is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
transmitting the calculated control input to a control target (Insignificant extra-solution activity – transmitting a control input to a target represents post-solution activity – see MPEP § 2106.05(g))
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
transmitting the calculated control input to a control target (Receiving or transmitting data over a network has been deemed a well‐understood, routine, and conventional function – see MPEP § 2106.05(d)(ll))
Therefore, the claim is ineligible.
With respect to claim 5:
2A Prong 1: The claim recites an abstract idea. Specifically:
wherein the at least one processor further carries out a belief distribution processing process of processing the belief distribution which has been updated in the belief distribution updating process in a given step, for a process to be performed in a next step in the optimal variable candidate generation process, the objective function evaluation process, the inverse temperature optimization process, the weight evaluation process, and the belief distribution updating process (Mental process – processing the belief distribution is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
Therefore, the claim is ineligible.
With respect to claim 6:
2A Prong 1: The claim recites an abstract idea. Specifically:
a method for optimization, said method comprising: at least one processor generating a plurality of optimal variable candidates, based on a belief distribution; (Mental process – generating a plurality of optimal variable candidates is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
the at least one processor evaluating an objective function for each of the plurality of optimal variable candidates; (Mental process – evaluating an objective function is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
the at least one processor calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other; (Mental process – calculating an inverse temperature using an optimization technique is an evaluation using an optimization technique that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
the at least one processor calculating the weight for the objective function, based on the inverse temperature ((Mental process – calculating the weight for an objective function is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
the at least one processor updating the belief distribution, based on the weight, the belief distribution, and each of the optimal variable candidates (Insignificant extra-solution activity – updating a belief distribution represents post-solution activity – see MPEP § 2106.05(g))
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
the at least one processor updating the belief distribution, based on the weight, the belief distribution, and each of the optimal variable candidates (Performing repetitive calculations has been deemed a well‐understood, routine, and conventional function – see MPEP § 2106.05(d)(ll))
Therefore, the claim is ineligible.
With respect to claim 7:
2A Prong 2: The additional elements recited in the claim do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
further comprising, before the generating the plurality of optimal variable candidates, the at least one processor receiving input of the target effective sample size and an initial belief distribution (Insignificant extra-solution activity (mere data gathering) – see MPEP § 2106.05(g))
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
further comprising, before the generating the plurality of optimal variable candidates, the at least one processor receiving input of the target effective sample size and an initial belief distribution (Receiving or transmitting data over a network has been deemed a well‐understood, routine, and conventional function – see MPEP § 2106.05(d)(ll))
Therefore, the claim is ineligible.
With respect to claim 8:
2A Prong 2: The additional elements recited in the claim do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
further comprising, after the updating, the at least one processor outputting the belief distribution which has been updated if a predetermined termination condition is satisfied and (Insignificant extra-solution activity – outputting an updated belief distribution represents post-solution activity – see MPEP § 2106.05(g))
the at least one processor repeating a process from the generating of the plurality of optimal variable candidates if the predetermined termination condition is not satisfied (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea – see MPEP § 2106.05(f))
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
further comprising, after the updating, the at least one processor outputting the belief distribution which has been updated if a predetermined termination condition is satisfied and (Insignificant extra-solution activity – outputting an updated belief distribution represents post-solution activity – see MPEP § 2106.05(g))
the at least one processor repeating a process from the generating of the plurality of optimal variable candidates if the predetermined termination condition is not satisfied (Performing repetitive calculations has been deemed a well‐understood, routine, and conventional function – see MPEP § 2106.05(d)(ll))
Therefore, the claim is ineligible.
With respect to claim 9:
2A Prong 1: The claim recites an abstract idea. Specifically:
an optimal variable candidate generation process of generating a plurality of optimal variable candidates, based on a belief distribution; (Mental process – generating a plurality of optimal variable candidates is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
an objective function evaluation process of evaluating an objective function for each of the plurality of optimal variable candidates; (Mental process – evaluating an objective function is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
an inverse temperature optimization process of calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other; (Mental process – calculating an inverse temperature using an optimization technique is an evaluation using an optimization technique that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
a weight evaluation process of calculating the weight for the objective function, based on the inverse temperature (Mental process – calculating the weight for an objective function is an evaluation that can be practically performed in the human mind, or by a human using a pen and paper as a physical aid – see MPEP § 2106.04(a)(2)(III))
2A Prong 2: The additional elements recited in the claim do not integrate the abstract idea into a practical application, individually or in combination.
Additional elements:
a non-transitory storage medium storing a program for causing a computer to function as an optimization apparatus, the program causing the computer to carry out: (Mere recitation of a generic computer component – see MPEP § 2106.05(b)(I))
a belief distribution updating process of updating the belief distribution, based on the weight, the belief distribution, and each of the optimal variable candidates (Insignificant extra-solution activity – updating a belief distribution represents post-solution activity – see MPEP § 2106.05(g))
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
a non-transitory storage medium storing a program for causing a computer to function as an optimization apparatus, the program causing the computer to carry out: (Mere recitation of a generic computer component – see MPEP § 2106.05(b)(I))
a belief distribution updating process of updating the belief distribution, based on the weight, the belief distribution, and each of the optimal variable candidates (Performing repetitive calculations has been deemed a well‐understood, routine, and conventional function – see MPEP § 2106.05(d)(ll))
Therefore, the claim is ineligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 are rejected under 35 U.S.C. 103 as being unpatentable over Rudenko et al. (US 2022/0050469 A1), in view of Duffield et al. (Duffield, Samuel, and Sumeetpal S. Singh. “Ensemble Kalman Inversion for General Likelihoods.” arXiv e-prints (2021): arXiv-2110).
Regarding claim 1, Rudenko teaches an optimization apparatus comprising
at least one processor ([0038]: “The control unit 11 may have a microprocessor or a microcontroller as well as a memory for storing data and an algorithm code”), the at least one processor being configured to carry out:
an optimal variable candidate generation process of generating a plurality of optimal variable candidates, based on a belief distribution ([0013]: “the informed variant of theoretic model predictive control may iteratively evaluate a number of control trajectory samples derived from control trajectory sample prior based on a given distribution at each time step to obtain a further control trajectory sample”; [0015]: “Sampling from this control trajectory prior distribution”);
an objective function evaluation process of evaluating an objective function for each of the plurality of optimal variable candidates ([0053]: “In step S5, starting with the control trajectory prior U* as obtained in step S3, but in further iterations modified control trajectory prior U*, movement costs are predicted. This is performed by predicting future states of the system (line 10) with the time horizon tf, and the corresponding movement costs Sk are accumulated (line 11)”);
a weight evaluation process of calculating the weight for the objective function, based on the inverse temperature ([0054]: “In step S6 (line 14), importance sampling weights wk are obtained by an algorithm 'ImportanceSamplingWeights' as shown in the flowchart of FIG. 3”; [0055]: “The importance sampling weights wk are determined based on the accumulated trajectory costs Sk of each control trajectory sample and the given λ”; [0056]: “ importance sampling weights wk for each of the control trajectory samples (determined so far) are obtained depending on a difference between the respective control trajectory costs Sk and the minimum control trajectory costs over all samples”); and
a belief distribution updating process of updating the belief distribution, based on the weight, the belief distribution, and each of the optimal variable candidates ([0045]: “It is preferable to use above formula as an update rule in an iterative manner”; [0058]: “In step S7 (line15), based on the importance sampling weights wk, a further control trajectory sample U* is obtained for the loop between lines 5 and 16”; [0072]: “the control trajectory sample distribution is guided towards low-cost areas of the state space”).
While Rudenko teaches solving an approximative minimization problem by an importance sampling technique and calculating weights for objective values using a given λ, where 1/λ functions as an inverse temperature coefficient ([0054]: “In step S6 (line 14), importance sampling weights wk are obtained by an algorithm 'ImportanceSamplingWeights' as shown in the flowchart of FIG. 3”; [0055]: “The importance sampling weights wk are determined based on the accumulated trajectory costs Sk of each control trajectory sample and the given λ”), Rudenko does not explicitly teach “an inverse temperature optimization process of calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other.”
Duffield further teaches an inverse temperature optimization process of calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other (Page 4, Section 3.2: “The stepsizes
h
l
l
-
1
L
equivalently define an inverse temperature schedule
0
=
λ
0
<
λ
1
<
…
<
λ
L
where
h
l
=
λ
l
-
λ
l
-
1
>
0
for
l
=
1
,
…
,
L
. For SMC samplers (Del Moral et al., 2006), it is common for the next inverse temperature to be selected adaptively [15] such that the effective sample size (of the sequential importance weights
w
l
i
ⅈ
=
1
N
) decreases by a fixed amount at each iteration. That is, select
λ
l
such that ESS
w
l
i
ⅈ
=
1
N
=
∑
i
=
1
N
w
l
i
2
-
1
≈
ρ
N
, where the normalised weights are a function of
λ
l
and
ρ
∈
(
0,1
)
is a tuning parameter that controls the size of the steps. The root for
λ
l
can be found using a numerical bisection algorithm in
(
λ
l
-
1
,
λ
L
]
and requires no additional likelihood evaluations”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to adapt the apparatus of Rudenko to incorporate the teachings of Duffield so as to include an inverse temperature optimization process of calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other. Doing so would allow inverse temperature to be optimized with the aim of moving through manageable intermediate distributions instead of a single difficult update (Duffield, Page 4, Section 3.2: “for difficult non-Gaussian problems, moving directly from prior to posterior in one step is an extremely difficult task. By taking many smaller steps the particles can explore the state-space and have a better chance of settling in regions of high posterior probability”).
Regarding claim 2, Rudenko in view of Duffield teaches the optimization apparatus according to claim 1.
Rudenko further teaches wherein in the optimal variable candidate generation process, the at least one processor generates the plurality of optimal variable candidates, based on an initial belief distribution which has been inputted or on the belief distribution which has been updated in the belief distribution updating process ([0047]: “1. Input: u*: Initial control sequence”; [0063]: “The control trajectory sample prior u* is generated based on the actual state of the system x, the trajectory control set of the last time step u*, initial conditions needed by the network, such as some sensor inputs σ, the robotic device 1 current state φ0, ν0, a covariance metric Σ, terminal costs ϕ, state costs c and the transition model F”).
Regarding claim 3, Rudenko in view of Duffield teaches the optimization apparatus according to claim 1.
Rudenko further teaches wherein in the objective function evaluation process, the at least one processor evaluates, for each of the plurality of optimal variable candidates, an objective function which depends on a state of a control target that is observed by a state observation apparatus ([0022]: “the control trajectory sample prior may be obtained by the sum of the modelled control trajectory sample obtained by the trajectory prediction model and the control trajectory sample obtained in the last time step, particularly each weighted according to their trajectory costs according to a given cost function”; [0049]: “The state of the system may be determined by processing sensor signals to obtain a pose of the robotic device 1 in/relative to the environment E and the poses and velocities of each of dynamic obstacles 2, 3, 4 in the environment E as well known in the art”; [0044]: “The goal of IT-MPC is to solve the stochastic optimal control problem of the form
U
*
≈
a
r
g
m
i
n
U
∈
U
E
Q
U
,
Σ
(
∑
t
=
0
t
f
-
1
c
(
x
t
)
+
ϕ
(
x
t
f
)
+
λ
u
t
-
1
T
∑
-
1
u
t
where U is the set of possible input sequences for the system,
c
(
x
t
)
is a state-based cost,
ϕ
(
x
t
f
)
is a terminal cost and a rate λ>0”).
Regarding claim 4, Rudenko in view of Duffield teaches the optimization apparatus according to claim 1.
Rudenko further teaches wherein the at least one processor further carries out a control input conversion process of calculating control input in accordance with a predetermined conversion rule based on the belief distribution which has been updated in the belief distribution updating process ([0047]: “17. ApplyControl (u0*)”) and transmitting the calculated control input to a control target ([0060]: “In step S9 (line 17) of the informed IT-MPC algorithm, the first control data (u*0) is applied to the robotic device 1”).
Regarding claim 5, Rudenko in view of Duffield teaches the optimization apparatus according to claim 1.
Rudenko further teaches wherein the at least one processor further carries out a belief distribution processing process of processing the belief distribution which has been updated in the belief distribution updating process in a given step, for a process to be performed in a next step in the optimal variable candidate generation process, the objective function evaluation process, the inverse temperature optimization process, the weight evaluation process, and the belief distribution updating process ([0044]: “ The optimal control distribution
Q
*
can be derived as
q
*
(
V
)
=
1
η
e
x
p
(
-
1
λ
S
(
V
)
p
(
V
∣
0
,
Σ
)
where S(V) denotes the state cost of an entire trajectory, p may be a Gaussian with covariance Σ and η is a normalization factor”; [0060]: “The predicted trajectory is updated to the following time step in step S10 (lines 18 to 20) so that (u*0) is set to the former U*1 and so on”).
Regarding claim 6, Rudenko teaches a method for optimization, said method comprising: at least one processor ([0038]: “The control unit 11 may have a microprocessor or a microcontroller as well as a memory for storing data and an algorithm code”) generating a plurality of optimal variable candidates, based on a belief distribution ([0013]: “the informed variant of theoretic model predictive control may iteratively evaluate a number of control trajectory samples derived from control trajectory sample prior based on a given distribution at each time step to obtain a further control trajectory sample”; [0015]: “Sampling from this control trajectory prior distribution”);
the at least one processor evaluating an objective function for each of the plurality of optimal variable candidates ([0053]: “In step S5, starting with the control trajectory prior U* as obtained in step S3, but in further iterations modified control trajectory prior U*, movement costs are predicted. This is performed by predicting future states of the system (line 10) with the time horizon tf, and the corresponding movement costs Sk are accumulated (line 11)”);
the at least one processor calculating the weight for the objective function, based on the inverse temperature ([0054]: “In step S6 (line 14), importance sampling weights wk are obtained by an algorithm 'ImportanceSamplingWeights' as shown in the flowchart of FIG. 3”; [0055]: “The importance sampling weights wk are determined based on the accumulated trajectory costs Sk of each control trajectory sample and the given λ”; [0056]: “ importance sampling weights wk for each of the control trajectory samples (determined so far) are obtained depending on a difference between the respective control trajectory costs Sk and the minimum control trajectory costs over all samples”); and
the at least one processor updating the belief distribution, based on the weight, the belief distribution, and each of the optimal variable candidates ([0045]: “It is preferable to use above formula as an update rule in an iterative manner”; [0058]: “In step S7 (line15), based on the importance sampling weights wk, a further control trajectory sample U* is obtained for the loop between lines 5 and 16”; [0072]: “the control trajectory sample distribution is guided towards low-cost areas of the state space”).
While Rudenko teaches solving an approximative minimization problem by an importance sampling technique and calculating weights for objective values using a given λ, where 1/λ functions as an inverse temperature coefficient ([0054]: “In step S6 (line 14), importance sampling weights wk are obtained by an algorithm 'ImportanceSamplingWeights' as shown in the flowchart of FIG. 3”; [0055]: “The importance sampling weights wk are determined based on the accumulated trajectory costs Sk of each control trajectory sample and the given λ”), Rudenko does not explicitly teach “the at least one processor calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other.”
Duffield further teaches the at least one processor calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other (Page 4, Section 3.2: “The stepsizes
h
l
l
-
1
L
equivalently define an inverse temperature schedule
0
=
λ
0
<
λ
1
<
…
<
λ
L
where
h
l
=
λ
l
-
λ
l
-
1
>
0
for
l
=
1
,
…
,
L
. For SMC samplers (Del Moral et al., 2006), it is common for the next inverse temperature to be selected adaptively [15] such that the effective sample size (of the sequential importance weights
w
l
i
ⅈ
=
1
N
) decreases by a fixed amount at each iteration. That is, select
λ
l
such that ESS
w
l
i
ⅈ
=
1
N
=
∑
i
=
1
N
w
l
i
2
-
1
≈
ρ
N
, where the normalised weights are a function of
λ
l
and
ρ
∈
(
0,1
)
is a tuning parameter that controls the size of the steps. The root for
λ
l
can be found using a numerical bisection algorithm in
(
λ
l
-
1
,
λ
L
]
and requires no additional likelihood evaluations”).
The reasons to combine Duffield into Rudenko are the same as articulated in the rejection of claim 1 above.
Regarding claim 7, Rudenko in view of Duffield teaches the method according to claim 6.
While Rudenko teaches receiving initial process inputs before sampling ([0047]: “1. Input: u*: Initial control sequence, K: Number of samples, tf: Time horizon, F: Transition model, ϕ, c: State cost, Σ, λ: Hyper-parameter, C decoder conditions… an initial configuration of parameters of the process is given”), Rudenko does not explicitly teach “before the generating the plurality of optimal variable candidates, the at least one processor receiving input of the target effective sample size and an initial belief distribution.”
Duffield further teaches further comprising, before the generating the plurality of optimal variable candidates, the at least one processor receiving input of the target effective sample size and an initial belief distribution (Page 1: “
p
(
x
)
is a prior distribution”; Page 4, Algorithm 1: “1: Given a sequence of inverse temperatures
λ
l
l
=
0
L
and stopping criterion - which may both be adaptive, see Sections 3.2 and 3.3. 2: Sample from prior
x
0
i
~
p
x
”; Page 4, Section 3.2: “select
λ
l
such that ESS
w
l
i
ⅈ
=
1
N
=
∑
i
=
1
N
w
l
i
2
-
1
≈
ρ
N
, where the normalised weights are a function of
λ
l
and
ρ
∈
(
0,1
)
is a tuning parameter that controls the size of the steps”; Page 5, Section 4: “we determine the stepsize
h
l
(equivalently the next inverse temperature
λ
l
) adaptively such that the ESS of the pseudo-weights (12) is approximately 0.5N”).
Regarding claim 8, Rudenko in view of Duffield teaches the method according to claim 6.
Rudenko further teaches further comprising, after the updating, the at least one processor outputting the belief distribution which has been updated if a predetermined termination condition is satisfied and the at least one processor repeating a process from the generating of the plurality of optimal variable candidates if the predetermined termination condition is not satisfied ([0061]: “In step S10 it is queried if the task is completed. If positive (alternative: yes), the process is ended, otherwise (alternative: no) the process is continued with step S2”).
Regarding claim 9, Rudenko teaches a non-transitory storage medium storing a program for causing a computer to function as an optimization apparatus ([0038]: “The control unit 11 may have a microprocessor or a microcontroller as well as a memory for storing data and an algorithm code”), the program causing the computer to carry out:
an optimal variable candidate generation process of generating a plurality of optimal variable candidates, based on a belief distribution ([0013]: “the informed variant of theoretic model predictive control may iteratively evaluate a number of control trajectory samples derived from control trajectory sample prior based on a given distribution at each time step to obtain a further control trajectory sample”; [0015]: “Sampling from this control trajectory prior distribution”);
an objective function evaluation process of evaluating an objective function for each of the plurality of optimal variable candidates ([0053]: “In step S5, starting with the control trajectory prior U* as obtained in step S3, but in further iterations modified control trajectory prior U*, movement costs are predicted. This is performed by predicting future states of the system (line 10) with the time horizon tf, and the corresponding movement costs Sk are accumulated (line 11)”);
a weight evaluation process of calculating the weight for the objective function, based on the inverse temperature ([0054]: “In step S6 (line 14), importance sampling weights wk are obtained by an algorithm 'ImportanceSamplingWeights' as shown in the flowchart of FIG. 3”; [0055]: “The importance sampling weights wk are determined based on the accumulated trajectory costs Sk of each control trajectory sample and the given λ”; [0056]: “ importance sampling weights wk for each of the control trajectory samples (determined so far) are obtained depending on a difference between the respective control trajectory costs Sk and the minimum control trajectory costs over all samples”); and
a belief distribution updating process of updating the belief distribution, based on the weight, the belief distribution, and each of the optimal variable candidates ([0045]: “It is preferable to use above formula as an update rule in an iterative manner”; [0058]: “In step S7 (line15), based on the importance sampling weights wk, a further control trajectory sample U* is obtained for the loop between lines 5 and 16”; [0072]: “the control trajectory sample distribution is guided towards low-cost areas of the state space”).
While Rudenko teaches solving an approximative minimization problem by an importance sampling technique and calculating weights for objective values using a given λ, where 1/λ functions as an inverse temperature coefficient ([0054]: “In step S6 (line 14), importance sampling weights wk are obtained by an algorithm 'ImportanceSamplingWeights' as shown in the flowchart of FIG. 3”; [0055]: “The importance sampling weights wk are determined based on the accumulated trajectory costs Sk of each control trajectory sample and the given λ”), Rudenko does not explicitly teach “an inverse temperature optimization process of calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other.”
Duffield further teaches an inverse temperature optimization process of calculating, by using an optimization technique, an inverse temperature such that a target effective sample size which has been inputted and an effective sample size of a weight for the objective function are substantially equal to each other (Page 4, Section 3.2: “The stepsizes
h
l
l
-
1
L
equivalently define an inverse temperature schedule
0
=
λ
0
<
λ
1
<
…
<
λ
L
where
h
l
=
λ
l
-
λ
l
-
1
>
0
for
l
=
1
,
…
,
L
. For SMC samplers (Del Moral et al., 2006), it is common for the next inverse temperature to be selected adaptively [15] such that the effective sample size (of the sequential importance weights
w
l
i
ⅈ
=
1
N
) decreases by a fixed amount at each iteration. That is, select
λ
l
such that ESS
w
l
i
ⅈ
=
1
N
=
∑
i
=
1
N
w
l
i
2
-
1
≈
ρ
N
, where the normalised weights are a function of
λ
l
and
ρ
∈
(
0,1
)
is a tuning parameter that controls the size of the steps. The root for
λ
l
can be found using a numerical bisection algorithm in
(
λ
l
-
1
,
λ
L
]
and requires no additional likelihood evaluations”);
The reasons to combine Duffield into Rudenko are the same as articulated in the rejection of claim 1 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Schäfer, Christian, and Nicolas Chopin. “Sequential Monte Carlo on large binary sampling spaces.” arXiv preprint arXiv:1101.6037 (2011): SMC particles, importance weights, ESS-controlled step-length selection by bisection, resampling/move steps, and updating a belief model from weighted particles
US 2021/0205984 A1: Latent robot dynamics with belief states, sampling state trajectories from current beliefs, reward/objective evaluation, and cross entropy search for action sequences
US 2017/0252924 A1: Generating multiple candidate robot end-effector motions and using cross-entropy optimization with a Gaussian distribution to select a candidate
US 2021/0192555 A1: Updating pricing candidates via cross-entropy minimization, candidate distributions, reinforcement-learning reward evaluation, and iterative planning
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Magdalena Kossek whose telephone number is (571)272-5603. The examiner can normally be reached Mon-Fri 8:00-5:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Robert Fennema can be reached at (571)272-2748. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.I.K./Examiner, Art Unit 2117
/ROBERT E FENNEMA/Supervisory Patent Examiner, Art Unit 2117
Prosecution Insights