DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-5, 7-8, 10, 13-14, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Rousselet US 20210180891 A1 in view of MAITRA US 20220113049 A1.
Regarding claim 10, Rousselet teaches a system comprising:
a memory; and at least one processor communicatively coupled to the memory (Fig. 1A [0039] [0040] [0046] – [0050] controller 52 analyzes current environmental and operating conditions of the cooling subsystem 14 and makes predict and recommend parameters, and implements the parameters to control the cooling subsystem), wherein the at least one processor is configured to:
deploy a trained machine learning agent in a water sourced heat pump (WSHP) system comprising a plurality of WSHPs (Fig. 1A [0039] [0040] [0046] – [0050] cooling subsystem with multiple WSHPs and cooling tower, Fig. 2 [0054] – [0060] a trained machining learning model is loaded to controller 52 and updated in real-time for real-time control of the cooling subsystem);
analyze one or more state variables associated with the WSHP system in real-time, using the trained machine learning agent, wherein the one or more state variables corresponds to a current state of the plurality of WSHPs, external heat sources of the plurality of WSHPs, or water loop temperature in the WSHP system (Fig. 2 [0054] – [0060] [0065] live and historical sensor data for operating variables and defined setpoints are collected and analyzed using the trained machine learning model for optimal setpoints setting, the operating variables including variable 108 leaving process fluid temperature i.e. “water loop temperature in the WSHP system”).
Rousselet does not explicitly further teach:
the trained machine learning is trained reinforcement learning;
generate one or more action variables using the trained RL agent based at least on the analyzed one or more state variables associated with the WSHP system, wherein the one or more action variables comprises at least one of water loop temperature and water loop flow rate;
generate at least one reward function based on the generated one or more action variables, wherein the at least one reward function corresponds to at least one of real time energy cost for operating the WSHP system, thermal discomfort within an operating area of the WSHP system, stability or degradation information of heat in water loop of the WSHP system; and
optimize at least one of the water loop temperature and the water loop flow rate of the WSHP system based on the generated at least one reward function.
MAITRA explicitly teaches in an analogous art that:
the trained machine learning is trained reinforcement learning ([0027] the trained deep reinforcement learning model);
generate one or more action variables using the trained RL agent based at least on the analyzed one or more state variables associated with the WSHP system, wherein the one or more action variables comprises at least one of water loop temperature and water loop flow rate ([0027] [0029] the trained machine learning model using simulations and deep reinforcement learning to control the system at the supervisor level, control actions are generated using the trained DRL agent based on the collected action and state data, the action data related to chilled water temperature setpoint and chilled water flow GPM STPT);
generate at least one reward function based on the generated one or more action variables, wherein the at least one reward function corresponds to at least one of real time energy cost for operating the WSHP system, thermal discomfort within an operating area of the WSHP system, stability or degradation information of heat in water loop of the WSHP system ([0029] [0043] simulator generated reward function based on the actions, the reward function is power consumption); and
optimize at least one of the water loop temperature and the water loop flow rate of the WSHP system based on the generated at least one reward function ([0043] the action is optimized to maximize the reward based on the reward function).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Rousselet to incorporate the teachings of MAITRA, because they all directed to HVAC control, to make the system wherein the trained machine learning is trained reinforcement learning; generate one or more action variables using the trained RL agent based at least on the analyzed one or more state variables associated with the WSHP system, wherein the one or more action variables comprises at least one of water loop temperature and water loop flow rate; generate at least one reward function based on the generated one or more action variables, wherein the at least one reward function corresponds to at least one of real time energy cost for operating the WSHP system, thermal discomfort within an operating area of the WSHP system, stability or degradation information of heat in water loop of the WSHP system; and optimize at least one of the water loop temperature and the water loop flow rate of the WSHP system based on the generated at least one reward function. One of ordinary skill in the art would have been motivated to do this modification so as to provide control at the supervisory level, as MAITRA teaches in [0027].
Regarding claim 13, Rousselet further teaches the water loop flow rate comprises at least water flow rate, water pump speed, or water circuit delta pressure set point ([0027] chilled water flow GPM).
Regarding claim 14, Rousselet further teaches determine the real time energy cost for operating the WSHP system using a utility tariff module (Figs. 3A&B [0098] [0099] real time operation cost including the energy cost of the cooling subsystem using estimated energy consumption and cost of energy in pricing data 119).
Regarding claim 16, Rousselet further teaches the external heat sources having one or more parameters such as electricity consumption of a cooling tower, fan speed ([0036] fan speed), tower delta temperature, steam or hot water consumption, heat exchanger delta temperature, gas consumption, supply or return delta temperature, aggregated WSHP cooling intensity, aggregated WSHP heating intensity, aggregated WSHP electricity consumption, aggregated zone delta temperature, or aggregated occupancy level.
Regarding claim 18, it is directed to a non-transitory machine-readable information storage medium comprising one or more instructions of carrying out the system with similar limitations as set forth in claim 10. Since Rousselet and MAITRA teach the claimed system, they teach the instructions for implementing the system.
Regarding claim 1, it is directed to a method of carrying out the system with similar limitations as set forth in claim 10. Since Rousselet and MAITRA teach the claimed system, they teach the method steps for implementing the system.
Regarding claims 4-5 and 7, they are directed to a method of carrying out the system with similar limitations as set forth in claims 13-14 and 16, respectively. Since Rousselet and MAITRA teach the claimed system, they teach the method steps for implementing the system, respectively.
Regarding claim 8, Rousselet further teaches the real time energy cost for operating the WSHP system corresponds to energy costs of the external heat sources and energy cost for the operating area of the WSHP system (Figs. 3A&B [0098] [0099] real time operation cost including the energy cost of the cooling subsystem using estimated energy consumption and cost of energy in pricing data 119, [0065] energy consumption for each individual components of the cooling subsystem including cooling tower i.e. “the external heat sources”, chiller, compressor, etc. i.e. “the operating area of the WSHP system”).
Claims 2, 11 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Rousselet in view of MAITRA as applied to claims 1, 4-5, 7-8, 10, 13-14, 16 and 18 above, further in view of FU CN 116880169 A.
Regarding claims 2, 11 and 19, neither Rousselet nor MAITRA explicitly further teaches receive the one or more state variables associated with the WSHP system, from one or more sensors, over a predefined period of time; and train the RL agent for the WSHP system based at least on the received one or more state variables.
FU explicitly teaches in an analogous art that receive the one or more state variables associated with the WSHP system, from one or more sensors, over a predefined period of time; and train the RL agent for the WSHP system based at least on the received one or more state variables (Abstract: obtaining data sample within a time period as training data for deep reinforcement learning).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Rousselet and MAITRA to incorporate the teachings of FU, because they all directed to HVAC control, to make the system/method/ non-transitory machine-readable information storage medium wherein receive the one or more state variables associated with the WSHP system, from one or more sensors, over a predefined period of time; and train the RL agent for the WSHP system based at least on the received one or more state variables. One of ordinary skill in the art would have been motivated to do this modification so as to provide peak power demand prediction control, as FU teaches in Abstract.
Claims 3, 12 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Rousselet in view of MAITRA as applied to claims 1, 4-5, 7-8, 10, 13-14, 16 and 18 above, further in view of Liu US 20070023534 A1.
Regarding claims 3 and 12, Rousselet further teaches the optimization of the water loop temperature is performed by defining a water cooling temperature set point ([0099] leaving water temperature setpoint of chiller).
Neither Rousselet nor MAITRA explicitly further teaches the optimization of the water loop temperature is performed by defining a water heating temperature set point;
Liu explicitly teaches in an analogous art that the optimization of the water loop temperature is performed by defining a water heating temperature set point ([0004] boiler temperature setpoint).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Rousselet and MAITRA to incorporate the teachings of Liu, because they all directed to HVAC control, to make the system/method wherein the optimization of the water loop temperature is performed by defining a water heating temperature set point. One of ordinary skill in the art would have been motivated to do this modification so as to provide heating using heat pumps, as Liu teaches in [0003].
Claim 20 recites similar limitations to that of claims 12 and 13 therefore is rejected on the same basis.
Claims 6 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Rousselet in view of MAITRA as applied to claims 1, 4-5, 7-8, 10, 13-14, 16 and 18 above, further in view of Varghese US 20240142116 A1 and NAGARATHINAM US 20210200163 A1.
Regarding claims 6 and 15, Rousselet further teaches the current state of the plurality of WSHPs comprises at least one of cooling intensity ([0057] cooling load) in a facility and occupancy level ([0119] live occupancy data).
Neither Rousselet nor MAITRA explicitly further teaches the current state of the plurality of WSHPs comprises at least one of heating intensity and comfort state;
Varghese explicitly teaches in an analogous art that the current state of the plurality of WSHPs comprises at least one of and heating intensity ([0041] heating load in heating mode); and
NAGARATHINAM explicitly teaches in an analogous art that the current state of the plurality of WSHPs comprises comfort state ([0038] thermal discomfort).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Rousselet and MAITRA to incorporate the teachings of Varghese and NAGARATHINAM, because they all directed to HVAC control, to make the system/method wherein the current state of the plurality of WSHPs comprises at least one of heating intensity and comfort state. One of ordinary skill in the art would have been motivated to achieve the goal of conditioning to comfort requirements, as NAGARATHINAM teaches in [0034].
Claims 9 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Rousselet in view of MAITRA as applied to claims 1, 4-5, 7-8, 10, 13-14, 16 and 18 above, further in view of NAGARATHINAM.
Regarding claims 9 and 17, neither Rousselet nor MAITRA explicitly further teaches the at least one reward function comprises an energy component and zero or more penalties, wherein the zero or more penalties depends upon the thermal discomfort within the operating area of the WSHP system, and the stability or degradation information of heat in the water loop of the WSHP system.
NAGARATHINAM explicitly teaches in an analogous art that the at least one reward function comprises an energy component and zero or more penalties, wherein the zero or more penalties depends upon the thermal discomfort within the operating area of the WSHP system, and the stability or degradation information of heat in the water loop of the WSHP system ([0038] reward function comprises energy component and penalty, penalty comprises one of thermal discomfort, stability or degradation information on the equipment).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Rousselet and MAITRA to incorporate the teachings of NAGARATHINAM, because they all directed to HVAC control, to make the system/method wherein the at least one reward function comprises an energy component and zero or more penalties, wherein the zero or more penalties depends upon the thermal discomfort within the operating area of the WSHP system, and the stability or degradation information of heat in the water loop of the WSHP system. One of ordinary skill in the art would have been motivated to achieve the goal of conditioning to comfort requirements, as NAGARATHINAM teaches in [0034].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
THIELMANN US 20200049381 A1 teaches adjusting loop fluid temperature for heat pump heating and cooling.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael Tang whose telephone number is (571)272-7437. The examiner can normally be reached M-F 7:30-4 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kamini Shah can be reached on (571)272-2279. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/M.T./ Examiner, Art Unit 2115
/KAMINI S SHAH/ Supervisory Patent Examiner, Art Unit 2115
Prosecution Insights