Detailed Action
Claims 1-15 are presently pending.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “a storage device that stores” and “an arithmetic device that calculates” in claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Regarding claim 1,
Step 1: Claim 1 recites an apparatus comprising: a storage device and an arithmetic device. Therefore, it is directed to the statutory category of a machine.
2A Prong 1:
2A Prong 2:
A future state estimation apparatus comprising: (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
a storage device that stores a state transition model in which first state transition probability representing probability in which a prediction target shifts from a first state to a second state after elapse of a first time is expressed by a linear combination of weighted basis functions; and (insignificant extra-solution activity MPEP 2106.05(g)(iii) of mere data gathering)
an arithmetic device that calculates … (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
2B:
A future state estimation apparatus comprising: (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
a storage device that stores a state transition model in which first state transition probability representing probability in which a prediction target shifts from a first state to a second state after elapse of a first time is expressed by a linear combination of weighted basis functions; and (indicated as an insignificant extra-solution activity MPEP 2106.05(g)(iii) in Step 2A Prong 2. Therefore, it is re-evaluated as well understood, routine and conventional activity MPEP 2106.05(d)(II)(iv) of storing and retrieving information in memory)
an arithmetic device that calculates … (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
Regarding claim 2,
Step 1: A machine, as above.
2A Prong 1: The future state estimation apparatus according to claim 1, wherein the product-sum calculation is calculation of a series of the weighting matrix, and wherein the wording "after elapse of the second time" denotes "after elapse of infinite time" or "after elapse of an infinite step". (mathematical calculation – instant specification [0032]-[0033])
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Regarding claim 3,
Step 1: A machine, as above.
2A Prong 1: The future state estimation apparatus according to claim 1, wherein the arithmetic device calculates an optimal operation amount of a device controlling the prediction target based on the second state transition probability. (mathematical calculation – instant specification paragraph [0075])
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Regarding claim 4,
Step 1: A machine, as above.
2A Prong 1: The future state estimation apparatus according to claim 1, wherein the arithmetic device calculates an element value of the weighting matrix from time series of a measurement value of the prediction target. (mathematical calculation – instant specification paragraph [0102])
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Regarding claim 5,
Step 1: A machine, as above.
2A Prong 1: The future state estimation apparatus according to claim 4, wherein the arithmetic device updates the state transition model by using the element value of the weighting matrix. (mathematical calculation – instant specification paragraph [0102])
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Regarding claim 6,
Step 1: A machine, as above.
2A Prong 1: The future state estimation apparatus according to claim 4, wherein the arithmetic device learns the element value of the weighting matrix, and updates the state transition model by using the learned element value of the weighting matrix. (mathematical calculation – instant specification paragraph [0102]-[0103])
2A Prong 2: This judicial exception is not integrated into a practical application.
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Regarding claim 7,
Step 1: A machine, as above.
2A Prong 1: Incorporates the rejection of claim 5.
2A Prong 2: further comprising an output device, wherein the arithmetic device causes the output device to output information representing any two or more of the state transition model before update, the state transition model after update, and a difference between the state transition models before and after update. (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
2B: further comprising an output device, wherein the arithmetic device causes the output device to output information representing any two or more of the state transition model before update, the state transition model after update, and a difference between the state transition models before and after update. (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
Regarding claim 8,
Step 1: A machine, as above.
2A Prong 1: Incorporates the rejection of claim 1.
2A Prong 2: further comprising an output device, wherein the arithmetic device causes the output device to output probability of transition from a state of a transition source to a state of a transition destination in any one or more of an elapsed time, an elapsed step, a time range, and a step range. (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
2B: further comprising an output device, wherein the arithmetic device causes the output device to output probability of transition from a state of a transition source to a state of a transition destination in any one or more of an elapsed time, an elapsed step, a time range, and a step range. (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
Regarding claim 9,
Step 1: A machine, as above.
2A Prong 1: wherein the basis function is a radial basis function. (mathematical calculation – instant specification paragraph [0108]-[0109])
2A Prong 2: The future state estimation apparatus according to claim 1 (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
2B: The future state estimation apparatus according to claim 1 (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
Regarding claim 10,
Step 1: A machine, as above.
2A Prong 1: wherein the radial basis function is a normal distribution function. (mathematical calculation – instant specification paragraph [0108]-[0109])
2A Prong 2: The future state estimation apparatus according to claim 9 (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
2B: The future state estimation apparatus according to claim 9 (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
Regarding claim 11,
Step 1: A machine, as above.
2A Prong 1: calculates the second state transition probability from the sum of values obtained by multiplying respective probability stored in the storage device by an exponentiation of an attenuation rate according to elapse of time. (mathematical calculation – instant specification paragraph [0110])
2A Prong 2: The future state estimation apparatus according to claim 10, wherein the arithmetic device stores, in the storage device, probability in which the prediction target shifts from the first state to the second state after elapse of time of an integral multiple of the first time (insignificant extra-solution activity MPEP 2106.05(g)(iii) of mere data gathering)
2B: The future state estimation apparatus according to claim 10, wherein the arithmetic device stores, in the storage device, probability in which the prediction target shifts from the first state to the second state after elapse of time of an integral multiple of the first time (indicated as an insignificant extra-solution activity MPEP 2106.05(g)(iii) in Step 2A Prong 2. Therefore, it is re-evaluated as well understood, routine and conventional activity MPEP 2106.05(d)(II)(iv) of storing and retrieving information in memory)
Regarding claim 12,
Step 1: A machine, as above.
2A Prong 1: calculates the second state transition probability based on an inverse matrix of a difference between a unit matrix and the matrix stored in the storage device. (mathematical calculation – instant specification [0112])
2A Prong 2: The future state estimation apparatus according to claim 10, wherein the arithmetic device stores, in the storage device, a matrix in which the product of a transposition matrix of a conversion matrix in which an integration value of the normal distribution function is as an element and the weighting matrix is multiplied by an attenuation rate (insignificant extra-solution activity MPEP 2106.05(g)(iii) of mere data gathering)
2B: The future state estimation apparatus according to claim 10, wherein the arithmetic device stores, in the storage device, a matrix in which the product of a transposition matrix of a conversion matrix in which an integration value of the normal distribution function is as an element and the weighting matrix is multiplied by an attenuation rate (indicated as an insignificant extra-solution activity MPEP 2106.05(g)(iii) in Step 2A Prong 2. Therefore, it is re-evaluated as well understood, routine and conventional activity MPEP 2106.05(d)(II)(iv) of storing and retrieving information in memory)
Regarding claim 13,
Step 1: A machine, as above.
2A Prong 1: wherein the arithmetic device calculates the second state transition probability from a Frobenius inner product of the product of the weighting matrix and the inverse matrix and a Gaussian function matrix. (mathematical calculation – instant specification [0114])
2A Prong 2: The future state estimation apparatus according to claim 12 (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
2B: The future state estimation apparatus according to claim 12 (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
Regarding claim 14,
Step 1: A machine, as above.
2A Prong 1: calculates an operation amount of a device controlling the prediction target. (mathematical calculation – instant specification paragraph [0115])
2A Prong 2: The future state estimation apparatus according to claim 1, (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
wherein the arithmetic device is installed in a plant (a field of use and technological environment MPEP 2106.05(h) – limiting the usage of the arithmetic device)
2B: The future state estimation apparatus according to claim 1, (mere instructions to apply an exception using a generic computer component MPEP 2106.05(f))
wherein the arithmetic device is installed in a plant (a field of use and technological environment MPEP 2106.05(h) – limiting the usage of the arithmetic device)
Regarding claim 15,
Step 1: A machine, as above.
2A Prong 1: Incorporates the rejection of claim 14.
2A Prong 2: The future state estimation apparatus according to claim 14, wherein the weighting matrix is a matrix or a vector, and wherein the prediction target is a physical amount of a target controlled by the device or a surrounding environment of the target. (a field of use and technological environment MPEP 2106.05(h) – limiting the usage of the arithmetic device)
2B: The future state estimation apparatus according to claim 14, wherein the weighting matrix is a matrix or a vector, and wherein the prediction target is a physical amount of a target controlled by the device or a surrounding environment of the target. (a field of use and technological environment MPEP 2106.05(h) – limiting the usage of the arithmetic device)
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 and 3-6 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Hasuo et al. (US 20120084237 A1, hereinafter ‘Hasuo’).
Regarding claim 1, Hasuo teaches:
A future state estimation apparatus comprising: ([Hasuo, Fig. 7] and [0112] The apparatus comprises a storage unit 13 and a target state setting unit 34)
a storage device that stores a state transition model in which first state transition probability representing probability in which a prediction target shifts from a first state to a second state after elapse of a first time is expressed by a linear combination of weighted basis functions; and ([Hasuo, 0146] The state transition model is stored in the model storage unit 13 by repeatedly calculating the recurrence formula of
T
(
S
)
←
m
a
x
U
∑
S
'
P
S
S
’
U
T
(
S
'
)
a predetermined number of times)
an arithmetic device that calculates second state transition probability representing probability in which the prediction target shifts from the first state to the second state by the time after elapse of a second time, by product- sum calculation of a weighting matrix representing a matrix in which weights of the respective weighed basis functions are as elements. ([Hasuo, Fig. 13, block S26], [0146]-[0147] and [0184] The target state setting unit 34 obtains an existence probability
T
(
S
)
(i.e., second state transition probability) of being the current state by the state transitions within a predetermined number of times (i.e., time after elapse of a second time) for each state S based on the initial value of the existence probability
T
(
S
’
)
and the state transition model
P
S
S
’
U
. The
P
S
S
’
U
is a matrix and it is supported by [Fig. 29A], [Fig. 29B], [0412]-[0423] and [0429] as it is represented as a Hidden Markov Model with actions
U
m
, states
S
i
and
S
j
. The columns of the transition matrix
P
S
S
’
U
are basis functions, and the weights are from the function
T
(
S
'
)
. The equation 6
T
(
S
)
←
m
a
x
U
∑
S
'
P
S
S
’
U
T
(
S
'
)
discloses calculating a summation of a product of
P
S
S
’
U
and
T
(
S
’
)
)
Regarding claim 3, Hasuo teaches:
The future state estimation apparatus according to claim 1,
wherein the arithmetic device calculates an optimal operation amount of a device controlling the prediction target based on the second state transition probability. ([0064], [0111] and [0112] The action control unit selects an action to be performed next among actions that the agent can perform based on the observation value from the sensor and the state transition model stored in the model storage unit, and supplies an action signal corresponding to the action to the learning unit and the actuator)
Regarding claim 4, Hasuo teaches:
The future state estimation apparatus according to claim 1,
wherein the arithmetic device calculates an element value of the weighting matrix from time series of a measurement value of the prediction target. ([Hasuo, 0146]-[0148] shows that the existence probability T(S) (weighting matrix) of the current state is repeatedly calculated based on the state transition model
P
S
S
’
U
and obtained for each state. [Fig. 6, block S14] and [0101]-[0102] The state transition model
P
S
S
’
U
is updated by increasing the frequency indicated by the state transition model
P
S
S
’
U
by one and based on the observation value (i.e., measurement value) from the sensor 11.
P
S
S
’
U
is a matrix and it is supported by [Fig. 29A], [Fig. 29B], [0412]-[0423] and [0429] as it is represented as a Hidden Markov Model with actions
U
m
, states
S
i
and
S
j
)
Regarding claim 5, Hasuo teaches:
The future state estimation apparatus according to claim 4,
wherein the arithmetic device updates the state transition model by using the element value of the weighting matrix. ([Hasuo, 0146]-[0148] shows that the existence probability T(S) (weighting matrix) of the current state is repeatedly calculated based on the state transition model
P
S
S
’
U
and obtained for each state. [Fig. 6, block S14] and [0101]-[0102] The state transition model
P
S
S
’
U
is updated by increasing the frequency indicated by the state transition model
P
S
S
’
U
by one and based on the observation value (i.e., measurement value) from the sensor 11.
P
S
S
’
U
is a matrix and it is supported by [Fig. 29A], [Fig. 29B], [0412]-[0423] and [0429] as it is represented as a Hidden Markov Model with actions
U
m
, states
S
i
and
S
j
)
Regarding claim 6, Hasuo teaches:
The future state estimation apparatus according to claim 4,
wherein the arithmetic device learns the element value of the weighting matrix, and updates the state transition model by using the learned element value of the weighting matrix. ([Hasuo, 0146]-[0148] shows that the existence probability T(S) (weighting matrix) of the current state is repeatedly calculated based on the state transition model
P
S
S
’
U
and obtained for each state. [Hasuo, Fig. 6, block S14] and [0101]-[0102] The state transition model
P
S
S
’
U
is updated by increasing the frequency indicated by the state transition model
P
S
S
’
U
by one and based on the observation value from the sensor 11.
P
S
S
’
U
is a weighting matrix and it is supported by [Fig. 29A], [Fig. 29B], [0412]-[0423] and [0429] as it is represented as a Hidden Markov Model with actions
U
m
, states
S
i
and
S
j
)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Hasuo in view of Li et al. (US 20170358090 A1, hereinafter ‘Li’).
Regarding claim 2, Hasuo teaches:
The future state estimation apparatus according to claim 1,
wherein the product-sum calculation is calculation of a series of the weighting matrix[Hasuo, Fig. 13, block S26], [0146]-[0147] and [0184] The target state setting unit 34 obtains an existence probability
T
(
S
)
(i.e., second state transition probability) of being the current state by the state transitions within a predetermined number of times (i.e., time after elapse of a second time) for each state S based on the initial value of the existence probability
T
(
S
’
)
and the state transition model
P
S
S
’
U
. The equation 6
T
(
S
)
←
m
a
x
U
∑
S
'
P
S
S
’
U
T
(
S
'
)
discloses calculating a summation of a product of
P
S
S
’
U
and
T
(
S
’
)
)
However, Hasuo does not specifically disclose:
wherein the wording "after elapse of the second time" denotes "after elapse of infinite time" or "after elapse of an infinite step".
Li teaches:
wherein the wording "after elapse of the second time" denotes "after elapse of infinite time" or "after elapse of an infinite step". ([Li, 0038] For a Markov chain, the conditional distribution P(X(t)|X(t’) t’<=t0)=P(X(t)|X(t0)) denotes a probability from a state space X(t0) at time t0 to a state space X(t) at time t. If the Markov chain is irreducible and ergodic, every state will occur infinitely many times but their respective frequencies of occurring will converge to the stationary distribution)
Before the effective filing date of the invention to a person of ordinary skill in the art, it would
have been obvious, having the teachings of Hasuo and Li to implement the future state estimation apparatus of the present invention. The suggestion and/or motivation for doing so is to improve the usability of the system by allowing the system to calculate the state to which the system converges over an infinitely extending timeframe.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Hasuo in view of Takada et al. (US 20210325837 A1, hereinafter ‘Takada’).
Regarding claim 7, Hasuo teaches:
The future state estimation apparatus according to claim 5.
However, Hasuo does not specifically disclose:
further comprising an output device, wherein the arithmetic device causes the output device to output information representing any two or more of the state transition model before update, the state transition model after update, a difference between the state transition models before and after update.
Takada teaches:
further comprising an output device, wherein the arithmetic device causes the output device to output information representing any two or more of the [Takada, 0044] - [0046] The output control unit 105 outputs (displays) at least one of the parameters of the model after being updated and the differences between the parameters of the model before the update and the parameters after the update. The parameters of the state transition model and updating the state transition model are taught by the primary reference Hasuo as shown in Claim 1 mapping)
Before the effective filing date of the invention to a person of ordinary skill in the art, it would
have been obvious, having the teachings of Hasuo and Takada to use the method of outputting information representing model before update and a difference between the model before update and the model after update of Takada to implement the future state estimation apparatus of the present invention. The suggestion and/or motivation for doing so is to allow the system to execute the model with greater accuracy using an updated model that follows changing data [Takada, 0046].
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Hasuo in view of Braviner et al. (US 20220277213 A1, hereinafter ‘Braviner’)
Regarding claim 8, Hasuo teaches:
The future state estimation apparatus according to claim 1.
However, Hasuo does not specifically disclose:
further comprising an output device, wherein the arithmetic device causes the output device to output probability of transition from a state of a transition source to a state of a transition destination in any one or more of an elapsed time, an elapsed step, a time range, and a step range.
Braviner teaches:
further comprising an output device, wherein the arithmetic device causes the output device to output probability of transition from a state of a transition source to a state of a transition destination in any one or more of an elapsed time, an elapsed step, a time range, and a step range. ([Hasuo, 0021] The function C* outputs the probability that the agent 100 will reach goal state s (i.e., transition destination) within time horizon h (i.e., in an elapsed time) by taking action a. C is the CAE model’s approximation of C)
Before the effective filing date of the invention to a person of ordinary skill in the art, it would
have been obvious, having the teachings of Hasuo and Braviner to use the method of output probability of transition from a state of a transition source to a state of a transition destination within an elapsed time of Braviner to implement the future state estimation apparatus of the present invention. The suggestion and/or motivation for doing so is to improve a conventional reinforcement learning technique of the present invention by allowing the system to balancing the risks of more speedily reaching the goal state with the possibility of not reaching the goal state in time [Braviner, 0008].
Claims 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Hasuo in view of Zhang et al. (US 10181101 B2, hereinafter ‘Zhang’).
Regarding claim 14, Hasuo teaches:
The future state estimation apparatus according to claim 1, wherein the arithmetic device
P
S
S
’
U
is a matrix and it is supported by [Fig. 29A], [Fig. 29B], [0412]-[0423] and [0429] as it is represented as a Hidden Markov Model with actions
U
m
, states
S
i
and
S
j
. [0069]-[0071] shows that the action
U
1
increases the acceleration (i.e., physical amount of a target) of the agent to the upper direction)
Hasuo does not specifically disclose:
wherein the arithmetic device is installed in a plant.
Zhang teaches:
wherein the arithmetic device is installed in a plant. ([Zhang, Claim 1] shows that the Markov chain forecast model is deployed in a fossil fuel power plant to improve the operation of the fossil fuel power plant)
Before the effective filing date of the invention to a person of ordinary skill in the art, it would
have been obvious, having the teachings of Hasuo and Zhang to use the method of installing the device in a plant of Zhang to implement the future state estimation apparatus of the present invention. The suggestion and/or motivation for doing so is to expand the application of the Markov chain state transition model to industrial sectors, thereby improving the operation of a power plant.
Regarding claim 15, Hasuo teaches:
The future state estimation apparatus according to claim 14, wherein the weighting matrix is a matrix or a vector, and wherein the prediction target is a physical amount of a target controlled by the device or a surrounding environment of the target. (
P
S
S
’
U
is a matrix and it is supported by [Fig. 29A], [Fig. 29B], [0412]-[0423] and [0429] as it is represented as a Hidden Markov Model with actions
U
m
, states
S
i
and
S
j
. [0069]-[0071] shows that the action
U
1
increases the acceleration (i.e., physical amount of a target) of the agent to the upper direction)
Conclusion
Claims 9-13 have been searched, but have not been rejected with respect to any prior art statutes.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 20200285204 A1 (This prior art is pertinent, because it discloses deploying a machine-learned state transition model in a plant)
“Fast Direct Policy Evaluation using Multiscale Analysis of Markov Diffusion Processes” (This prior art is pertinent, because it discloses calculating state transition probabilities based on weight values and current state values)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JUN KWON whose telephone number is (571)272-2072. The examiner can normally be reached Monday – Friday 8:00AM – 5:00PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at (571)270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JUN KWON/Examiner, Art Unit 2127
/BRIAN M SMITH/Primary Examiner, Art Unit 2122
Prosecution Insights