DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The Amendment filed 01/20/2026 has been entered. Claims 1-20 are presented for examination.
Claim Rejections – 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 6 and 13 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Claims 6 and 13 recite "the target reactive power setpoint." There is insufficient antecedent basis for this term in the claims.
Claim Rejections – 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 4-11, 13-17, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Li (CN113541192 translation) over Sohani (US 2021/0210959). Regarding claim 1, Li teaches a device comprising:
at least one processor; and
memory including instructions that when executed by the at least one processor (inherently, Li includes a system with a processor and memory/programs to implement the described operations), cause the at least one processor to:
learn, for a distributed energy resource (DER) electrically coupled to a power distribution network, a function for optimizing a selected parameter for power flow within the power distribution network (page 7, to implement a reactive power-voltage control strategy using a trained machine learning model, wind farm data is collected in real time, which is used to implement the control strategy; pages 6, 2, 1, the model obtains a state set of inputs, which includes among other things voltage at nodes of an electrical network of the wind farm, which are connected also to the grid; based on the inputs, the model determines an action response i.e., the reactive power set value for each wind turbine in the wind farm; pages 2-3, 5-7, the machine learning model is trained to optimize or minimize voltage deviation and system losses),
the learned function mapping local voltage values of the DER to target reactive power setpoints for the DER (page 7, to implement a reactive power-voltage control strategy using a trained machine learning model, wind farm data is collected in real time, which is used to implement the control strategy; pages 6, 2, the model obtains a state set of inputs, which includes among other things voltage at nodes of the wind farm; based on the inputs, the model determines an action response i.e., the reactive power set value for each wind turbine in the wind farm).
However, Li does not expressly disclose control a reactive power output of the DER based on the learned function and based on a step size for steering the reactive power output to one of the target reactive power setpoints, wherein the step size is selected to satisfy a stability parameter.
In the same field of endeavor, Sohani teaches
control a reactive power output of the DER based on the learned function (Fig. 9, [0059, 0060, 0014, 0040-0041, 0043, 0064, 0067], claims 15-16, the system can vary the reactive power setpoints associated with a wind farm and distribute them to obtain a desired voltage response; a parameter e.g., reactive power, can be adjusted at a voltage regulator of each wind turbine to generate a desired output/response)
and based on a step size for steering the reactive power output to one of the target reactive power setpoints, wherein the step size is selected to satisfy a stability parameter (Li pages 7-8 teaches that the system collects real-time data e.g. local voltage, and dynamically determines a reactive power setting in response to the real-time data; this means the reactive power setting is constantly adjusted i.e., there is an amount of change or a step between iterations of the reactive power setting; these can be considered to be steps and inherently involve a step size; as noted in Li pages 2-3, 5-7 the reactive power settings are determined to ensure stability, with respect to voltage deviation and system losses, and also are based on keeping voltages within particular limits; as noted on Li page 2, the system also attempts to constrain reactive power to a particular range; it would be obvious to utilize the above techniques when controlling reactive power output of a wind farm, as taught in Sohani e.g., see Sohani Fig. 9, [0059, 0060, 0014, 0040-0041, 0043, 0064, 0067], claims 15-16).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to have incorporated control a reactive power output of the DER based on the learned function and based on a step size for steering the reactive power output to one of the target reactive power setpoints, wherein the step size is selected to satisfy a stability parameter as suggested in Sohani into Li, because Li and Sohani pertain to analogous fields of technology. Li pertains to utilizing and training a machine learning model to determine suitable reactive power setpoints based on various local parameters, such as voltage. Li further teaches generating a reactive action command based on the above determinations and to ensure stability with respect to system losses, voltage deviation and reactive power output e.g., see Li pages 5, 1, 2. Sohani also pertains to a system for determining reactive power setpoints based on local parameters such as voltage. In Sohani, a reactive power setpoint is determined and distribute to control the operations of the wind farm and generate a desired voltage response. It would be desirable to incorporate this feature into Li so that the techniques of Li can be used to command and operate a wind farm, which appears to be the intention of the Li invention e.g., see Sohani Fig. 9, [0059, 0060, 0014].
Regarding claim 2, the combination of Li and Sohani teaches the invention as claimed in claim 1. The combination of Li and Sohani also teaches wherein the selected parameter comprises power generation cost, line losses, or deviations from a nominal voltage (Li page 7, to implement a reactive power-voltage control strategy using a trained machine learning model, wind farm data is collected in real time, which is used to implement the control strategy; pages 6, 2, 1, the model obtains a state set of inputs, which includes among other things voltage at nodes of an electrical network of the wind farm, which are connected also to the grid; based on the inputs, the model determines an action response i.e., the reactive power set value for each wind turbine in the wind farm; pages 2-3, 5-8, the machine learning model is trained to optimize or minimize voltage deviation e.g., from a rated voltage or 1 p.u., and system losses).
Regarding claim 4, the combination of Li and Sohani teaches the invention as claimed in claim 3. The combination of Li and Sohani also teaches
train the machine learning network based at least in part on a set of reference local voltage values associated with the DER and a set of reference equilibrium points associated with the power distribution network (Li page 1, 5-6, the model is trained to map a current state to a reactive power instruction; Li pages 6, 5, 2, 3, the model is trained to map a state including multiple parameters, such as voltage at nodes of the wind farm, to actions i.e., reactive power set value for each wind turbine; page 7, the training process involves iteratively mapping multiple candidate voltages/parameters to candidate reactive power set values, until rewards are stable; Li pages 5-6, the training process takes into account various reference points e.g., a reference voltage of a node of 1 p.u., constraint values on voltage for the nodes i.e., Vimin and Vimax).
Regarding claim 5, the combination of Li and Sohani teaches the invention as claimed in claim 3. The combination of Li and Sohani also teaches
train the machine learning network based at least in part on:
the target reactive power setpoints (Li page 1, 5-6, the model is trained to map a current state to a reactive power instruction; Li pages 6, 5, 2, 3, the model is trained to map a state including multiple parameters, such as voltage at nodes of the wind farm and reactive power output, to actions i.e., reactive power set value for each wind turbine; page 7, the training process involves iteratively mapping multiple candidate voltages/parameters to candidate reactive power set values, until rewards are stable); and
one or more reactive power injections associated with the power distribution network, wherein the one or more reactive power injections are non-controllable by the device (Li page 7, to implement a reactive power-voltage control strategy using a trained machine learning model, wind farm data is collected in real time, which is used to implement the control strategy; pages 6, 2-3, the model obtains a state set of inputs, which includes among other things voltage at nodes of the wind farm; see also Sohani Fig. 5, [0053], the system obtains an electrical signal e.g., local voltage, at a point of interconnection with the grid, and uses it to determine reactive power setpoints/output; Sohani Fig. 7, [0056] further indicates that there may be other wind farms not controlled by the system described in Sohani which cause reactive power/perturbations detectable at the point of interconnection, thereby affecting locally detected voltage sensitivity; put another way, by utilizing local voltage e.g., at a local point of interconnection, for the training process, as described in Li, the training process can take into account reactive power output/injections from other wind farms, since the local voltage can reflect in part the reactive power injections of those other wind farms connected to the grid).
Regarding claim 6, the combination of Li and Sohani teaches the invention as claimed in claim 1. The combination of Li and Sohani also teaches
iteratively determine the target reactive power setpoint for the DER; and
iteratively set the reactive power output of the DER in response to one or more iterative determinations of the target reactive power setpoint (Li page 1, 5-6, the model is trained to map a current state to a reactive power instruction; pages 6, 5, 2, 3, the model is trained to map a state including multiple parameters, such as voltage at nodes of the wind farm, to actions i.e., reactive power set value for each wind turbine; page 7, the training process involves iteratively mapping multiple candidate voltages/parameters to candidate reactive power set values, until rewards are stable; see also pages 7-8, the system collects real-time data e.g. local voltage, and dynamically determines a reactive power setting in response to the real-time data; this means the reactive power is set repeatedly in response to changes in the detected voltage i.e., incremental increases or decreases in the voltage, and may involve incremental increases/decreases in reactive power setpoints).
Regarding claim 7, the combination of Li and Sohani teaches the invention as claimed in claim 1. The combination of Li and Sohani also teaches the learned function minimizes the selected parameter of the power distribution network (Li page 7, to implement a reactive power-voltage control strategy using a trained machine learning model, wind farm data is collected in real time, which is used to implement the control strategy; pages 6, 2, 1, the model obtains a state set of inputs, which includes among other things voltage at nodes of an electrical network of the wind farm, which are connected also to the grid; based on the inputs, the model determines an action response i.e., the reactive power set value for each wind turbine in the wind farm; pages 2-3, 5-7, the machine learning model is trained to optimize or minimize voltage deviation and system losses).
Regarding claim 8, the combination of Li and Sohani teaches the invention as claimed in claim 1. The combination of Li and Sohani also teaches wherein learning the function, controlling the reactive power output, or both is independent of at least one second DER (Sohani Figs. 3, 9, [0059, 0060, 0014, 0040-0043, 0064, 0067], claims 15-16, the system can vary the reactive power setpoints associated with a particular, local wind farm electrical power system from the system and distribute them to obtain a desired voltage response; as noted in [0054-0056], there may be other neighboring wind farm electrical power systems; however, the above control operations do not apply to those systems).
Regarding claim 9, Sohani teaches the invention as claimed in claim 1. Sohani also a reactive power controller device associated with the DER (Sohani Figs. 3, 5, [0038-0043, 0058] describes a controller that manages the voltage of the wind farm system, including managing particular related parameters e.g., reactive power setpoint/output e.g., a central master controller).
Regarding claim 10, the claim corresponds to claim 1 and is rejected for the same reasons.
Regarding claim 11, the combination of Li and Sohani teaches the invention as claimed in claim 10. Claim 11 also corresponds to claim 2 and is rejected for the same reasons.
Regarding claim 13, the combination of Li and Sohani teaches the invention as claimed in claim 10. Claim 13 also corresponds to claim 6 and is rejected for the same reasons.
Regarding claim 14, the combination of Li and Sohani teaches the invention as claimed in claim 10. Claim 14 also corresponds to claim 7 and is rejected for the same reasons.
Regarding claim 15, the combination of Li and Sohani teaches the invention as claimed in claim 10. Claim 15 also corresponds to claim 8 and is rejected for the same reasons.
Regarding claim 16, the claim corresponds to claim 1 and is rejected for the same reasons. The combination of Li and Sohani also teaches
a device associated with a distributed energy resource (DER) electrically coupled
to a power distribution network (Sohani Fig. 3, Abstract, [0038, 0042-0043] describe a master controller and a wind farm electrical power system), the device comprising:
sensing circuitry (Sohani [0009-0010] describes a sensor for detecting signals e.g., voltage);
processing circuitry; and
control circuitry (Sohani Fig. 4, [0044] teaches a controller with memory with data).
Regarding claim 17, the combination of Li and Sohani teaches the invention as claimed in claim 16. Claim 17 also corresponds to claim 2 and is rejected for the same reasons.
Regarding claim 19, the combination of Li and Sohani teaches the invention as claimed in claim 16. Claim 19 also corresponds to claim 6 and is rejected for the same reasons.
Regarding claim 20, the combination of Li and Sohani teaches the invention as claimed in claim 16. Claim 20 also corresponds to claim 7 and is rejected for the same reasons.
Claims 3, 12 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Li and Sohani, as applied in claims 1, 10 and 16, and further in view of Jaisson (Jaisson T., "Deep differentiable reinforcement learning and optimal trading," arXiv, published April 8, 2022) and further in view of Sinay (US 2020/0097439).
Regarding claim 3, the combination of Li and Sohani teaches the invention as claimed in claim 1. The combination of Li and Sohani also teaches learn the function based on output of a machine learning network designed to ensure that the learned function is bounded (Li pages 2, 6, the output reactive power range is bounded e.g., to Qkmin and Qkmax).
However, the combination of Li and Sohani does not expressly disclose the machine learning network designed to ensure that the learned function differentiable and non-increasing.
In the same field of endeavor, Jaisson teaches the machine learning network designed to ensure that the learned function differentiable (page 1, it is known that in many reinforcement learning applications, the underlying functions are differentiable).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to have incorporated the machine learning network designed to ensure that the learned function differentiable as suggested in Jaisson into Li and Sohani, because Li and Jaisson pertain to analogous fields of technology. Li pertains to a machine learning model based on reinforcement learning e.g., see Li page 1. Jaisson teaches that it is known for reinforcement learning algorithms to be differentiable. It would be desirable to incorporate this feature into Li so that Li can utilize a variety of well known reinforcement learning algorithms e.g., see Jaisson page 1.
However, the combination of Li, Sohani and Jaisson does not expressly disclose the machine learning network designed to ensure that the learned function is non-increasing.
In the same field of endeavor, Sinay ([0024, 0003-0006], it is known to apply a non-increasing directional constraint to a machine learning model representing a negative relationship between inputs and outputs; as noted in [0006], such an approach can enhance interpretability and transparency while preserving predictive performance).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the invention to have incorporated the machine learning network designed to ensure that the learned function is non-increasing as suggested in Sinay into Li, Sohani and Jaisson because Li and Sinay pertain to analogous fields of technology. Li pertains to a system that utilizes a machine learning algorithm to map local voltages to corresponding reactive power output values. Generally, when voltage is low, reactive power may be injected to compensate i.e., a negative relationship between input and output. Sinay teaches that it is known to apply a non-increasing directional constraint to a machine learning model representing a negative relationship between inputs and outputs. It would be desirable to incorporate this feature into Li to improve interpretability and transparency e.g., see Sinay [0024, 0003-0006].
Regarding claim 12, the combination of Li and Sohani teaches the invention as claimed in claim 10. Claim 12 also corresponds to claim 3 and is rejected for the same reasons.
Regarding claim 18, the combination of Li and Sohani teaches the invention as claimed in claim 16. Claim 18 also corresponds to claim 3 and is rejected for the same reasons.
Response to Arguments
The Examiner acknowledges the Applicant's amendments to claims 1, 10 and 16.
Regarding independent claims 1, 9 and 17, the Applicant alleges that Sohani does not teach the amended limitation of "learn … a function for optimizing a selected parameter for power flow within the power distribution network, the learned function mapping local voltage values of the DER to target reactive power setpoints for the DER; and control a reactive power output of the DER based on the learned function and based on a step size for steering the reactive power output to one of the target reactive power setpoints, wherein the step size is selected to satisfy a stability parameter." Examiner has therefore rejected claims 1, 9 and 17 under 35 U.S.C. 103 as being taught by the combination of Li and Sohani. Applicant's remarks are moot in view of the new grounds of rejection. In the view of the Examiner, Li teaches a machine learning model/algorithm that optimizes for various parameters e.g., to minimize voltage deviation and system losses. Additionally, the machine learning model/algorithm maps voltage values at local nodes of a wind farm to reactive power output values. See Li pages 1-3 and 5-7. Sohani teaches controlling reactive power output on a real-time basis i.e., as local voltage values come in. Thus, the reactive power output steps up or down from previous reactive power output values. As noted above, based on the teachings of Li, the model underlying the reactive power output values attempts to ensure stability e.g., with respect to voltage deviations, bounded reactive power outputs, and system losses e.g., see Sohani Fig. 9, [0059, 0060, 0014, 0040-0041, 0043, 0064, 0067]; see also Li pages 2 and 6.
Applicant further alleges that claims 2-8, 10-16 and 18-20 are allowable in view of their dependency on claims 1, 9 and 17. Claims 2-8, 10-16 and 18-20 are rejected as being taught by Li, Sohani, Jaisson and/or Sinay.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Miller (US 2025/045439) teaches that it is known to have a ML model yield a function F that is arranged to be monotonically non-increasing in its inputs, where inputs are expected to decrease the output/score e.g., see Miller [0087].
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC YOON whose telephone number is (408)918-7581. The examiner can normally be reached on 9 am to 5 pm ET Monday through Friday.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott Baderman, can be reached at telephone number 571-272-3644. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/ERIC J YOON/Primary Examiner, Art Unit 2118