DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-18 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding Claims 1, 7, and 13 the claims recite "the reinforcement learning model".
There is insufficient antecedent basis for this limitation in the claim, as the model is not yet introduced in the claim, making the scope of the claim as a whole unclear.
Claims 2-6, 8-12, 14-18 are rejected for being dependent on rejected claims and failing to cure the deficiencies.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Stamatis et al (A Storage Expansion Planning Framework using Reinforcement Learning and Simulation-Based Optimization, published 01/10/2020, see attached, hereinafter Stamatis) in view of Daniel et al (US 20210221247, hereinafter Daniel).
Regarding Claim 1, Stamatis et al teaches:
a method for a reinforcement learning based battery energy storage system (BESS) project approval decision model, comprising: (see at least "In this paper, Markov Decision Processes (MDP) and Reinforcement Learning (RL) algorithms are used to propose a novel approach for solving large-scale expansion planning problems by considering the stochastic and dynamic nature of the problem. The proposed dynamic algorithm answers all the critical questions, such as (1) whether it is actually necessary to add storage in the energy system, (2) when to install this storage, (3) how much capacity it should be added, and (4) which storage technology should be chosen." in page 3)
gathering data related to the BESS across a plurality of sources, (see at least training datasets discussed on page 21-22, 27 )
computing prior benefit realization probabilities and risk exposure probabilities from the gathered data for the BESS; (see at least equations 9 and 10 and balancing cost of investment with benefit of loss of load cost prevention and “Finding the sweet spot between these two extreme strategies, along with the correct timing and appropriate selection of storage type, is the bulk of this research work.” On page 15 and probabilistic transition matrix for determining rewards on page 19-20 and probabilistic pricing on page 26)
training the reinforcement learning model from the computed prior benefit realization probabilities and the risk exposure probabilities; and (see training discussed on page 27)
executing the reinforcement learning model to generate a decision for a target BESS project approval in response to an input associated with input target BESS. (see “In this section, we present a detailed conceptual formulation of the problem under investigation. The primary objective of this analytical framework is to sequentially determine the optimal battery storage investment strategy to expand capacity for a system of distributed electricity generation plants connected in a microgrid network. It should be made clear at this point that there is only one scope at this problem: the long-term one. However, in order to efficiently estimate some cost components of the problem, we deploy a simulation-based technique, followed by a machine learning algorithm, that utilize operational level details (such as battery scheduling, outage modeling etc.).” on page 9)
Stamatis does not appear to explicitly teach all of the following, but Daniel does teach:
wherein one of the plurality of sources comprises a blockchain system configured to provide a BESS performance data for existing installations across stakeholders of the existing installations; (see at least " Of relevance within embodiments of the present invention, and exchange means, is where such ledger approaches are used to help govern interactions for assets within a close community, building, site, community or low voltage network. Within these approaches, a para-chain model can be used where part of the local energy system, such as a substation or special meter, can be used for confirmation and validation of local transactions, negating the energy and data intensive issues with full distributed block-chains. An approach is also to use what we term ‘mesh-chains’, where ledgers or block-chains are created at stable nodes, representing an assumed level of trust, such as by smart meters, charger points, at particular locations, as well as within assets such as electric vehicles, each time they cross-over or interact with another ledger, thus each creating an audit trail of each transaction that has metered energy flow for, charge event by a charger, charge/discharge by a vehicle, with each transaction creating a shared hash and timestamp noting its interaction within the grid." in par. 0107 and “A further example and embodiment, is where distributed ledger approaches are used to create and manage a smart contract between parties or form a shareable coin to mediate e.g. how KWh's of solar generation, Battery capacity, or local flexibility is shared on either a local ledger basis—where a trusted party is an asset such as a meter/charger/network node, within a location, that is included as a location-stamp within a hash of a time-stamp and transaction between parties. Such approaches are particularly advantageous where asset use has depreciation costs attached to it, such as in a stationary battery resource or electric vehicle, where ‘coins’ could record the asset depreciation and carbon cost, as well as cost and ownership of energy into the asset, in order to correctly account value for any net use, export or sharing of energy from the resource.” In par. 0109)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method taught by Stamatis to incorporate the teachings of Daniel wherein blockchain ledgers are used to gather real-time energy usage and trading data for decision-making models. The motivation to incorporate the teachings of Daniel would be to more accurately account for the cost of use of energy storage and transmission equipment (see par. 0109)
Regarding Claim 2, Stamatis as modified by Daniel (references to Stamatis) teaches:
the method of claim 1,
wherein the decision for the target BESS comprises risk-benefit analysis for the decision. (see at least Figs. 4, 6-8)
Regarding Claim 3, Stamatis as modified by Daniel (references to Stamatis) teaches:
the method of claim 1,
wherein the data related to the BESS comprises battery module, battery manufacturing characteristics, BESS product and system integrator characteristics. (see battery characteristics on Table 7 on page 26 and system integrator characteristics in nomenclature section starting on page 7)
Regarding Claim 4, Stamatis as modified by Daniel (references to Stamatis) teaches:
the method of claim 1,
wherein the training the reinforcement learning model comprises formulating a states model configured to learn benefit and risk for the BESS and return a state to a learning environment of the reinforcement learning model, wherein the reinforcement learning model undergoes policy updates from the learning environment. (see agent updating policy based on state, action, reward from environment relationships explained on page 15-16)
Regarding Claim 5, Stamatis as modified by Daniel (references to Stamatis) teaches:
the method of claim 1,
wherein the decision indicates whether the target BESS project is to proceed or to be deferred. (see examples of when the policy recommends that no battery expansion should be done at one time point while recommending to expand with a certain type and size of storage capacity at other times on page 30-31)
Regarding Claim 6, Stamatis as modified by Daniel (references to Stamatis) teaches:
the method of claim 1
wherein the training the reinforcement learning model comprises assigning decision sequences based on the prior benefit realization probabilities and the risk exposure probabilities. (see case studies discussed starting on page 25 where sequences of four decisions are recommended by the RL model over 20 year periods)
Regarding Claim 7, Stamatis as modified by Daniel also teaches:
a non-transitory computer readable medium, storing instructions for implementing the method of Claim 1 (see Claim 1 analysis for rejection of the method)
Regarding Claim 8, Stamatis as modified by Daniel also teaches:
a non-transitory computer readable medium, storing instructions for implementing the method of Claim 2 (see Claim 2 analysis for rejection of the method)
Regarding Claim 9, Stamatis as modified by Daniel also teaches:
a non-transitory computer readable medium, storing instructions for implementing the method of Claim 3 (see Claim 3 analysis for rejection of the method)
Regarding Claim 10, Stamatis as modified by Daniel also teaches:
a non-transitory computer readable medium, storing instructions for implementing the method of Claim 4 (see Claim 4 analysis for rejection of the method)
Regarding Claim 11, Stamatis as modified by Daniel also teaches:
a non-transitory computer readable medium, storing instructions for implementing the method of Claim 5 (see Claim 5 analysis for rejection of the method)
Regarding Claim 12, Stamatis as modified by Daniel also teaches:
a non-transitory computer readable medium, storing instructions for implementing the method of Claim 6 (see Claim 6 analysis for rejection of the method)
Regarding Claim 13, Stamatis as modified by Daniel also teaches:
An apparatus for implementing the method of Claim 1 (see Claim 1 analysis for rejection of the method)
Regarding Claim 14, Stamatis as modified by Daniel also teaches:
An apparatus for implementing the method of Claim 2 (see Claim 2 analysis for rejection of the method)
Regarding Claim 15, Stamatis as modified by Daniel also teaches:
An apparatus for implementing the method of Claim 3 (see Claim 3 analysis for rejection of the method)
Regarding Claim 16, Stamatis as modified by Daniel also teaches:
An apparatus for implementing the method of Claim 4 (see Claim 4 analysis for rejection of the method)
Regarding Claim 17, Stamatis as modified by Daniel also teaches:
An apparatus for implementing the method of Claim 5 (see Claim 5 analysis for rejection of the method)
Regarding Claim 18, Stamatis as modified by Daniel also teaches:
An apparatus for implementing the method of Claim 6 (see Claim 6 analysis for rejection of the method)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DYLAN M KATZ whose telephone number is (571)272-2776. The examiner can normally be reached Mon-Thurs. 8:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Lin can be reached on (571) 270-3976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DYLAN M KATZ/Primary Examiner, Art Unit 3657
Prosecution Insights