Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-10 are presented for examination.
Drawings
The drawings in Figures 1-14C are not of sufficient quality to permit examination. Accordingly, replacement drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to this Office action. The replacement sheet(s) should be labeled “Replacement Sheet” in the page header (as per 37 CFR 1.84(c)) so as not to obstruct any portion of the drawing figures. If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action.
Applicant is given a shortened statutory period of TWO (2) MONTHS to submit new drawings in compliance with 37 CFR 1.81. Extensions of time may be obtained under the provisions of 37 CFR 1.136(a) but in no case can any extension carry the date for reply to this letter beyond the maximum period of SIX MONTHS set by statute (35 U.S.C. 133). Failure to timely submit replacement drawing sheets will result in ABANDONMENT of the application.
Specification
The disclosure is objected to because of the following informalities:
The specification contains two sets of paragraphs [0001] - [0027]. Appropriate correction is required.
Claim Objections
Claim(s) 1 is/are objected to because of the following informalities:
Claim 1, “wherein the framework is programmed provide a trained GCN-integrated reinforcement learning model” should be “wherein the framework is programmed to provide a trained GCN-integrated reinforcement learning model”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 4, 7-10, is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim(s) 4 recite(s) “the first layer of the neural network”. There is lack of antecedent basis for this limitation in these claim(s).
Claim(s) 7, recite(s) “perform an analysis of distribution of a commodity through the UDN based on simulation for a sequence of recovery actions for one or more components of the UDN, wherein the sequence of recovery actions defines a respective episode, wherein the performance calculator is programmed to compute the measurement of the performance as a deep Q function, which has a Q value based on an instant reward component and a future reward component, the Q value being used to train the neural network”.
It is unclear whether the “which” in “which has a Q value based on an instant reward component and a future reward component, the Q value being used to train the neural network” refers to the calculator, measurement, performance or deep Q function, rendering the claim(s) indefinite.
For examination purposes the examiner has interpreted “which has a Q value based on an instant reward component and a future reward component, the Q value being used to train the neural network” to be
“wherein the deep Q function has a Q value based on an instant reward component and a future reward component, the Q value being used to train the neural network”.
Claim(s) 8 recite(s) “providing, by the neural network, a sequence of recovery actions based on a current state space of the WDN model data and a measure of performance”. It is unclear whether the claim is to be interpreted as
“providing, by the neural network, (i) a sequence of recovery actions based on a current state space of the WDN model data and (ii) a measure of performance”, or
“providing, by the neural network, a sequence of recovery actions based on (i) a current state space of the WDN model data and (ii) a measure of performance”,
rendering the claim(s) indefinite.
For examination purposes the examiner has interpreted “providing, by the neural network, a sequence of recovery actions based on a current state space of the WDN model data and a measure of performance” to be
“providing, by the neural network, (i) a sequence of recovery actions based on a current state space of the WDN model data and (ii) a measure of performance”.
Claim(s) 9-10 do not contain claim limitations that cure the indefiniteness of claim(s) 8, and therefore are also indefinite under 35 U.S.C. 112(b).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhao et al “Learning Sequential Distribution System Restoration via Graph-Reinforcement Learning” IEEE Transactions on Power Systems, Vol. 37, NO. 2, March 2022, (Pages 1601-1611), in view of Cussonneau (US 20180181111 A1).
Regarding claim 1, Zhao teaches a system to facilitate repair decisions for a utility distribution network (UDN), comprising… instructions to provide a reinforcement learning framework, comprising (Zhao Abstract utility distribution network has algorithms for service restoration):
… data including UDN model data representative of a structure of the UDN having a plurality of nodes and parameter data characterizing features and connectivity associated with each node of the UDN structure (Zhao Pg 102, 2nd Para, Sec III-A, Sec III-B-1st Para, UDN may be represented as feature matrix representing nodes, edges and node observation(s) (parameters));
a graph convolutional neural network (GCN) programmed to encode the structure of the UDN, in which the GCN is programmed to project nodes of the UDN structure into a multi-dimensional state space according to the UDN model data and the parameter data and to provide GCN output data responsive to an input representative of at least a current state of the UDN and one or more actions (Zhao Pg1604-RtCol, , GCN encodes UDN feature matrix into lower dimension (projection) node-level feature matrix and vector based on current node observation(s));
a neural network, connected to the GCN, including an input layer and an output layer, in which the input layer is programmed to receive the GCN output, and the output layer is programmed to provide a sequence of recovery actions based on a current state space of the UDN model data; (Zhao Sec II-B, Pg1604-First and Last Paras, Pg1605, GCN output is used by input layer of neural network agent(s) where output layer determinate repair/reconfiguration (recovery) actions to take based on observations and agent states, network performance is predicted based on performing the determined actions),
a performance calculator programmed to determine a measurement of the performance of the UDN in response to each of a plurality of recovery actions applied to the UDN model data for a current state space of the UDN over time , wherein the measurement of performance for each recovery action is applied to train the neural network, and wherein the framework is programmed provide a trained GCN-integrated reinforcement learning model that is programmed to generate recovery output data representing a sequence of recovery actions for the UDN in response to input UDN state data representative of a current state of the UDN (Zhao Sec II-B, Sec III-B-3 performance is predicted based on performing the determined actions and is used to maximize Q-value and minimize loss for policy, reinforcement learning is employed using performance measurements to train model to determine repair actions).
Zhao does not specifically teach a system to facilitate repair decisions for a utility distribution network (UDN), comprising: non-transitory computer-readable memory programmed to store data and instructions, the data including UDN model data representative of a structure of the UDN …; one or more processors configured to access the memory and execute the instructions to provide a reinforcement learning framework, comprising
However Cussonneau teaches a system to facilitate repair decisions for a utility distribution network (UDN), comprising: non-transitory computer-readable memory programmed to store data and instructions, the data including UDN model data representative of a structure of the UDN …; one or more processors configured to access the memory and execute the instructions to provide a reinforcement learning framework, comprising (Cussonneau [Abstract 26, 31, 32] system processor executes instructions stored in memory to perform operations for a UDN, Cussonneau [31, 52-58, 70-72] control parameters data may be stored, control parameters may include information regarding UDN topology (nodes and edges)).
It would have been obvious to one of an ordinary skill in the art before the effective filing date of the claimed invention, to have incorporated the concept taught by Cussonneau of a system to facilitate repair decisions for a utility distribution network (UDN), comprising: non-transitory computer-readable memory programmed to store data and instructions, the data including UDN model data representative of a structure of the UDN …; one or more processors configured to access the memory and execute the instructions to provide a reinforcement learning framework, comprising, into the invention suggested by Zhao; since both inventions are directed towards correcting issues in a UDN, and incorporating the teaching of Cussonneau into the invention suggested by Zhao would provide the added advantage of allowing UDN topology to be stored so it can be retrieved as needed, and the combination would perform with a reasonable expectation of success (Cussonneau [26, 31, 32, 52-58, 70-72]).
Regarding claim 2, Zhao and Cussonneau teach(es) the invention as claimed in claim 1 above.
Zhao further teaches wherein the performance calculator is further programmed to determine the measure of performance of the UDN responsive to each of a plurality of respective recovery actions for a respective episode of recovery actions based on the current state-space and a next-state space for the UDN (Zhao Sec II-B, Pg 1606-1607 Sec IV-A-1,2 Algorithm 1, performance is measured for UDN for each action based on current and next states, actions may be for multiple episodes).
Regarding claim 3, Zhao and Cussonneau teach(es) the invention as claimed in claim 1 above.
Zhao further teaches the GCN is configured to encode structural information for the …utility distribution network (Zhao Pg1604-RtCol, , GCN encodes UDN feature matrix into lower dimension (projection) node-level feature matrix and vector based on current node observation(s)).
Zhao does not specifically teach wherein the UDN is a water distribution network
However Cussonneau teaches wherein the UDN is a water distribution network (Cussonneau Abstract detecting anomalies in a water distribution system comprises a network of nodes).
Regarding claim 4, Zhao and Cussonneau teach(es) the invention as claimed in claim 1 above.
Zhao further teaches wherein the GCN is programmed to provide the GCN output as a matrix representing at least one state space value for respective parameters of each node of the UDN model, the framework is further programmed to convert the matrix into a corresponding vector that is received by the first layer of the neural network (Zhao Pg 1604 Sec III-B 1st and 3rd Paras, GCN converts observation feature matrix for graph into node-level feature matrix which can be converted to feature vector(s) before providing it to convolution layer).
Regarding claim 5, Zhao and Cussonneau teach(es) the invention as claimed in claim 1 above.
Zhao further teaches wherein the GCN output data is provided by aggregating an output for each node dimension of the GCN (Zhao Pg 1604 Last Para- Pg 1605 First Para, observations for each node may be encoded into lower dimension and efficiently encapsulates complex observation information in a low dimension (aggregate)).
Regarding claim 6, Zhao and Cussonneau teach(es) the invention as claimed in claim 1 above.
Zhao further teaches wherein the reinforcement learning framework is further programmed to at least: perform an analysis of distribution of a commodity through the UDN based on … a sequence of recovery actions for one or more components of the UDN, wherein the sequence of recovery actions defines a respective episode, wherein the performance calculator is programmed to compute the measurement of the performance as a deep Q function, which has a Q value based on an instant reward component and a future reward component, the Q value being used to train the neural network, wherein the training is repeated over a number of episodes, in which the state space parameters for the UDN are updated for each of the episodes (Zhao Abstract, Sec II-B, Pg 1606-1607 Sec IV-A-1,2 Algorithm 1, Pg1604 last sentence, UDN may be for power (commodity), deep Q function based on instant and future rewards may be used, Zhao Sec II-B, Pg 1606-1607 Sec IV-A-1,2 Algorithm 1, learning is conducted and performance is measured for UDN for each action based on current and next states, actions may be for multiple episodes).
Zhao does not specifically teach simulation for a sequence of recovery actions for one or more components of the UDN.
However Cussonneau teaches simulation for a sequence of recovery actions for one or more components of the UDN (Cussonneau [107] analysis may be based on simulating corrective actions).
Regarding claim 7, Zhao and Cussonneau teach(es) the invention as claimed in claim 6 above.
Zhao further teaches wherein the performance calculator is further programmed to feed the updated state of UDN into a deep Q function, which includes the GCN and the neural network, and the deep Q functions provides the future reward component as a maximum future reward based on the updated state of the UDN (Zhao Abstract, Sec II-B, Pg 1606-1607 Sec IV-A-1,2 Algorithm 1, Pg1604 last sentence, UDN may be for power (commodity), deep Q function based on instant and future rewards may be used, Zhao Secs IIIA,B, GCN provides up to date information regarding observations which is used as input for Q functions)..
Claim 8, is directed towards a method performing steps similar in scope to the instructions executed by the system of claim 3, and is rejected under the same rationale.
Zhao further teaches providing, by the neural network, a sequence of recovery actions based on a current state space of the WDN model data and a measure of performance (Zhao Sec II-B, Pg1604-First and Last Paras, Pg1605, GCN output is used by input layer of neural network agent(s) where output layer determinate repair/reconfiguration (recovery) actions to take based on observations and agent states, network performance is predicted based on performing the determined actions).
Regarding claim 9, Zhao and Cussonneau teach(es) the invention as claimed in claim 8 above.
Zhao further teaches wherein the input representative of at least the current state of the WDN includes graph structure data representative of the WDN structure, features for each node, and node satisfactory degree representative of system performance for the WDN (Zhao Pg 102, 2nd Para, Sec III-A, Sec III-B-1st Para, UDN may be represented as feature matrix representing nodes, edges and node observation(s) (parameters), Zhao Table I parameters can include allowed values/ranges for nodes), and
the GCN output data is a matrix of vectors representative of the current WDN, in which each node dimension of the matrix is aggregated to provide the GCN output as a one- dimensional vector space Zhao Pg 1604 Sec III-B 1st and 3rd Paras, GCN converts observation feature matrix for graph into node-level feature matrix which can be converted to feature vector(s) before providing it to convolution layer, Zhao Pg 1604 Last Para- Pg 1605 First Para, observations for each node may be encoded into lower dimension and efficiently encapsulates complex observation information in a low dimension (aggregate)).
Regarding claim 10, Zhao and Cussonneau teach(es) the invention as claimed in claim 8 above.
Zhao further teaches wherein the GCN is trained by a deep reinforcement learning framework programmed to perform a method comprising: performing an analysis of … distribution through the WDN based … a sequence of recovery actions for one or more components of the WDN, wherein the sequence of recovery actions defines a respective episode and the measure of performance represents a Q determined by a Q function based on an instant reward component and a future reward component, the Q value being used to train the neural network, wherein the training is repeated over a number of episodes, in which the state space parameters for the WDN are updated for respective actions implemented in each of the episodes (Zhao Abstract, Sec II-B, Pg 1606-1607 Sec IV-A-1,2 Algorithm 1, Pg1604 last sentence, UDN may be for power (commodity), deep Q function based on instant and future rewards may be used, Zhao Sec II-B, Pg 1606-1607 Sec IV-A-1,2 Algorithm 1, deep reinforcement learning may be conducted and performance is measured for UDN for each action based on current and next states, actions may be for multiple episodes).
Zhao does not specifically teach hydraulic distribution …simulation for a sequence of recovery actions for one or more components of the UDN.
However Cussonneau teaches hydraulic distribution …simulation for a sequence of recovery actions for one or more components of the UDN (Cussonneau Abstract, [107] analysis may be based on simulating corrective actions for hydraulic distribution).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SANCHITA ROY whose telephone number is (571)272-5310. The examiner can normally be reached Monday-Friday 12-8.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Usmaan Saeed can be reached at (571) 272-4046. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
SANCHITA ROY
Primary Examiner
Art Unit 2146
/SANCHITA ROY/Primary Examiner, Art Unit 2146
Prosecution Insights