DETAILED ACTION
Acknowledgments
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in reply to the application filed on 04/29/2024.
Claims 1-13 are currently pending and have been examined.
Information Disclosure Statement
The Information Disclosure Statements filed 04/29/2024 and 02/20/2025 have been considered. Initialed copies of the Form 1449 are enclosed herewith.
Allowable Subject Matter
Claims 5, 6, 10, and 11 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Objections
Claim 7 is objected to. The limitations refer to two separate and district statutory categories. Subsequently, it becomes unclear if claim 7 is independent or depends from claim 1.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 7, and 8 are rejected under U.S.C. 103 as being unpatentable over Yeh et al. (US 12,192,820 B2) hereinafter YEH, in view of Pressman et al. (USPGP 2022/0289030 A1), hereinafter PRESSMAN.
Claims 1, 7:
YEH as shown below discloses the following limitations:
loading a multi-agent reinforcement learning model pre-trained and optimized through a simulation environment; (see at least column 21, line 65 to column 22, line 4; Figures 2, 8-10 as well as associated and related text)
receiving state observation data at a current time instant; (see at least column 4, line 3-28)
inputting the state observation data into the multi-agent reinforcement learning model for reinforcement learning and reasoning to output multi-control action information; (see at least Figures 2 as well as associated and related text)
YEH does not specifically disclose:
generating a control action information instruction based on the multi-control action information, transmitting the control action information instruction to a cooling system in the energy storage system, and performing, by the cooling system, thermal management on the energy storage system in response to the control action information instruction.
PRESSMAN, however, in at least paragraphs 0065 and 0115 does. In this case, each of the elements claimed are all shown by the prior art of record but not combined as claimed. However, the technical ability exists to combine the elements as claimed and the results of the combination are predictable. Therefore, when combined, the elements perform the same function as they did separately. (KSR v. Teleflex, 127 S. Ct. 1727 (2007)). Consequently, it would have been obvious to one of ordinary skill in the art at the effective filing date to combine/modify the method of YEH with the technique of PRESSMAN because, “Failure of a cell and/or current-carrying joint may be detrimental to the entire system. In vehicular applications, the apparatuses may experience large impacts due to travel, coupling and decoupling of a vehicle with another vehicle, etc. These and other forces may loosen poor connections and may cause a loose component that is at a battery potential to make contact with or near ground potential which may eventually lead to a thermal event. Current flow through a poor electrical joint may also lead to excessive heating and a potential thermal event. Not all unexpected heating comes from a poor joint, however. Unexpected heating may come from a failed thermal management circuit (e.g., a cooling system that directs coolant along conduits forming the thermal management circuit) or from the battery cells. Some failure modes may also yield unexpected cooling. Early detection of a cell failure may prevent a thermal event from occurring. But, the large number of cells and/or joints may make early detection extremely difficult.” (PRESSMAN: paragraphs 0004-0005). Additionally, there is a recognized problem or need in the art including market pressure, design need, etc., and there are a finite number of identified predictable solutions. Accordingly, those in the art could have pursued known solutions with reasonable expectation of success. (KSR v. Teleflex, 127 S. Ct. 1727 (2007)). Fundamentally, in the competitive business climate, there is a profit-driven motive to maximize the profitability of goods and services that are provided or marketed to customers. Enterprises typically use business planning to make decisions in order to maximize profits.
Claim 2:
The combination of YEH/PRESSMAN discloses the limitations as shown in the rejections above. YEH further discloses the following limitations:
wherein after the receiving state observation data at a current time instant, the thermal management method further comprises:
determining an observation time step of the state observation data;
determining whether the observation time step reaches a preset length;
acquiring historical observation data, iteratively training the multi-agent reinforcement learning model by using the historical observation data and updating a parameter of the multi-agent reinforcement learning model in a case that the observation time step reaches the preset length, or, continuing to the process of receiving state observation data at a current time instant in a case that the observation time step does not reach the preset length.
See at least column 9 line 32; column 4, lines 3-28, column 15, lines 11-28, column 132, lines 19-34.
Claim 3:
The combination of YEH/PRESSMAN discloses the limitations as shown in the rejections above. PRESSMAN further discloses the following limitations:
wherein the performing, by the cooling system, thermal management on the energy storage system comprises: performing, by the cooling system, the thermal management on a battery system in the energy storage system;
or performing, by the cooling system, the thermal management on a battery system and an electric energy conversion unit in the energy storage system.
See at least paragraphs 0065. In this case, each of the elements claimed are all shown by the prior art of record but not combined as claimed. However, the technical ability exists to combine the elements as claimed and the results of the combination are predictable. Therefore, when combined, the elements perform the same function as they did separately. (KSR v. Teleflex, 127 S. Ct. 1727 (2007)). Consequently, it would have been obvious to one of ordinary skill in the art at the effective filing date to combine/modify the method of YEH with the technique of PRESSMAN because, “Failure of a cell and/or current-carrying joint may be detrimental to the entire system. In vehicular applications, the apparatuses may experience large impacts due to travel, coupling and decoupling of a vehicle with another vehicle, etc. These and other forces may loosen poor connections and may cause a loose component that is at a battery potential to make contact with or near ground potential which may eventually lead to a thermal event. Current flow through a poor electrical joint may also lead to excessive heating and a potential thermal event. Not all unexpected heating comes from a poor joint, however. Unexpected heating may come from a failed thermal management circuit (e.g., a cooling system that directs coolant along conduits forming the thermal management circuit) or from the battery cells. Some failure modes may also yield unexpected cooling. Early detection of a cell failure may prevent a thermal event from occurring. But, the large number of cells and/or joints may make early detection extremely difficult.” (PRESSMAN: paragraphs 0004-0005). Additionally, there is a recognized problem or need in the art including market pressure, design need, etc., and there are a finite number of identified predictable solutions. Accordingly, those in the art could have pursued known solutions with reasonable expectation of success. (KSR v. Teleflex, 127 S. Ct. 1727 (2007)). Fundamentally, in the competitive business climate, there is a profit-driven motive to maximize the profitability of goods and services that are provided or marketed to customers. Enterprises typically use business planning to make decisions in order to maximize profits.
Claim 4:
The combination of YEH/PRESSMAN discloses the limitations as shown in the rejections above. YEH further discloses the following limitations:
wherein the multi-agent reinforcement learning model is pre-trained and optimized by:
acquiring a state parameter and a control parameter of the energy storage system;
determining a state observation space, an action space, a constraint space and a reward function of the multi-agent reinforcement learning model based on the state parameter and the control parameter;
constructing the multi-agent reinforcement learning model based on the state observation space, the action space, the constraint space and the reward function;
receiving state observation data at a time instant t, inputting the state observation data into the multi-agent reinforcement learning model for training, to output an action a(t+1), a reward r(t) and a state s(t+1) at a time instant t+1;
calculating a state value function and a dominance function based on the action a(t+1), the reward r(t) and the state s(t+1); storing, in a data buffer pool, a sequence formed by the action a(t+1), the reward r(t), the state s(t+1), the state value function and the dominance function;
sampling N sequences randomly from the data buffer pool as training data, wherein N is a positive integer;
calculating, based on the sampled batch of sequences, a parameter gradient of a neural network in the multi-agent reinforcement learning model;
updating a parameter of the neural network in the multi-agent reinforcement learning model by using the parameter gradient of the neural network.
See at least column 56, line 60 to column 6, line 65.
Claim 8:
The combination of YEH/PRESSMAN discloses the limitations as shown in the rejections above. YEH further discloses the following limitations:
wherein the electric energy conversion unit comprises a direct current (DC)-alternating current (AC) unit and a plurality of DC-DC units;
a direct current side of the DC-AC unit is connected to the plurality of DC-DC units via a direct current bus; the DC-AC unit is communicatively connected to the EMS via a communication side of the DC-AC unit;
the plurality of DC-DC units each are communicatively connected to the intelligent battery thermal management unit.
See at least column 45, lines 10-20.
Claims 9, 12, and 13 are rejected under U.S.C. 103 as being unpatentable over YEH/PRESSMAN and further in view of Examiner’s OFFICIAL NOTICE.
Claims 9, 12, 13:
The combination of YEH/PRESSMAN discloses the limitations as shown in the rejections above. YEH/PRESSMAN does not specifically disclose:
wherein the cooling system is a coolant system and is configured to perform thermal management on the battery system,
wherein the cooling system comprises a cell liquid cooling plate, a plate heat exchanger, a compressor, a condenser, an air-water exchanger, a first heater, a first circulation pump and a first electromagnetic three-way valve;
a first end of the cell liquid cooling plate is connected to a first input end of the plate heat exchanger, and a first output end of the plate heat exchanger is connected to a first end of the first electromagnetic three-way valve;
a second end of the first electromagnetic three-way valve is connected to a second end of the cell liquid cooling plate through the first circulation pump and the first heater sequentially;
a third end of the first electromagnetic three-way valve is connected to a second end of the air-water exchanger, and a first end of the air-water exchanger is connected to the first input end of the plate heat exchanger;
a second output end of the plate heat exchanger is connected to a second input end of the plate heat exchanger through the condenser and the compressor sequentially.
wherein the cooling system is further configured to perform thermal management on the electric energy conversion unit, and the cooling system further comprises a second heater, a second circulation pump and a second electromagnetic three-way valve;
a third end of the air-water exchanger is connected to a first end of the second electromagnetic three-way valve and a first end of the electric energy conversion unit;
a fourth end of the air-water exchanger is connected to a second end of the second electromagnetic three-way valve;
a third end of the second electromagnetic three-way valve is connected to a second end of the electric energy conversion unit through the second circulation pump and the second heater sequentially.
wherein the cooling system is configured to perform thermal management on the electric energy conversion unit through an internal circulation of a coolant,
wherein in the internal circulation of the coolant, the coolant flows through the second electromagnetic three-way valve, the second circulation pump, the second heater and the electric energy conversion unit;
the cooling system is configured to perform thermal management on the electric energy conversion unit through an external circulation of a coolant,
wherein in the external circulation of the coolant, the coolant flows through the air-water exchanger, the second electromagnetic three-way valve, the second circulation pump, the second heater and the electric energy conversion unit.
However, the Examiner takes OFFICIAL NOTICE that it is old and well known in the thermal cooling arts to utilize standard components, configurations, and layouts for thermal heat exchangers and liquid cooling systems. Therefore, it would have been obvious to one of ordinary skill in the art at the effective filing date to combine/modify the method of YEH/PRESSMAN with the system of heat exchangers because there is a recognized problem or need in the art including market pressure, design need, etc., and there are a finite number of identified predictable solutions. Consequently, those in the art could have pursued known solutions with reasonable expectation of success. (KSR v. Teleflex, 127 S. Ct. 1727 (2007)). Additionally, there is a recognized problem or need in the art including market pressure, design need, etc., and there are a finite number of identified predictable solutions. Accordingly, those in the art could have pursued known solutions with reasonable expectation of success. (KSR v. Teleflex, 127 S. Ct. 1727 (2007)). In the competitive business climate, there is a profit-driven motive to maximize the profitability of goods and services that are provided or marketed to customers. Enterprises typically use business planning to make decisions in order to maximize profits.
CONCLUSION
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Non-Patent Literature:
Yuanlong Li et al. “Transforming Cooling Optimization for Green Data Center via Deep Reinforcement Learning.” (18 July 2018). Retrieved online 07/19/2026. https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwi-lvKc_96VAxX1MlkFHakNBX0QFnoECAsQAQ&url=https%3A%2F%2Farxiv.org%2Fpdf%2F1709.05077&usg=AOvVaw1cPjTcNbFPZbhCppXb4Cok&opi=89978449
Relevancy: “Cooling system plays a critical role in a modern data center (DC). Developing an optimal control policy for DC cooling system is a challenging task. The prevailing approaches often rely on approximating system models that are built upon the knowledge of mechanical cooling, electrical and thermal management, which is difficult to design and may lead to sub optimal or unstable performances. In this paper, we propose utilizing the large amount of monitoring data in DC to optimize the control policy. To do so, we cast the cooling control policy design into an energy cost minimization problem with temperature constraints, and tap it into the emerging deep reinforcement learning (DRL) framework. Specifically, we propose an end-to end cooling control algorithm (CCA) that is based on the actor critic framework and an off-policy offline version of the deep deterministic policy gradient (DDPG) algorithm. In the proposed CCA, an evaluation network is trained to predict an energy cost counter penalized by the cooling status of the DC room, and a policy network is trained to predict optimized control settings when given the current load and weather information. The proposed algorithm is evaluated on the EnergyPlus simulation platform and on a real data trace collected from the National Super Computing Centre (NSCC) of Singapore. Our results show that the proposed CCA can achieve about 11% cooling cost saving on the simulation platform compared with a manually configured baseline control algorithm. In the trace-based study, we propose a de-underestimation validation mechanism as we cannot directly test the algorithm on a real DC. Even though with DUE the results are conservative, we can still achieve about 15% cooling energy saving on the NSCC data trace if we set the inlet temperature threshold at 26.6 degree Celsius. (ABSTRACT)
Man Li et al. “Machine Learning for Harnessing Thermal Energy: From Materials Discovery to System Optimization.” (September 1, 2022). Retrieved online 07/19/2026. Machine Learning for Harnessing Thermal Energy: From Materials Discovery to System Optimization | ACS Energy Letters.
Relevancy: “Recent advances in machine learning (ML) have impacted research communities based on statistical perspectives and uncovered invisibles from conventional standpoints. Though the field is still in the early stage, this progress has driven the thermal science and engineering communities to apply such cutting-edge toolsets for analyzing complex data, unraveling abstruse patterns, and discovering non-intuitive principles. In this work, we present a holistic overview of the applications and future opportunities of ML methods on crucial topics in thermal energy research, from bottom-up materials discovery to top-down system design across atomistic levels to multi-scales. In particular, we focus on a spectrum of impressive ML endeavors investigating the state-of-the-art thermal transport modeling (density functional theory, molecular dynamics, and Boltzmann transport equation), different families of materials (semiconductors, polymers, alloys, and composites), assorted aspects of thermal properties (conductivity, emissivity, stability, and thermoelectricity), and engineering prediction and optimization (devices and systems). We discuss the promises and challenges of current ML approaches and provide perspectives for future directions and new algorithms that could make further impacts on thermal energy research.” (ABSTRACT)
Davide Coraci et al. “Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings.” (January 2023). Retrieved online 07/19/2026. https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwi-lvKc_96VAxX1MlkFHakNBX0QFnoECAwQAQ&url=https%3A%2F%2Fescholarship.org%2Fcontent%2Fqt6z85s9xn%2Fqt6z85s9xn.pdf&usg=AOvVaw1ATgkyg4LZu7Qxpdt9quVP&opi=89978449
Relevancy:” In recent years, advanced control strategies based on Deep Reinforcement Learning (DRL) proved to be effective in optimizing the management of integrated energy systems in buildings, reducing energy costs and improving indoor comfort conditions when compared to traditional reactive controllers. However, the scalability and implementation of DRL controllers are still limited since they require a considerable amount of time before converging to a near-optimal solution. This issue is currently addressed in literature through the offline pre-training of the DRL agent. However this solution results in two main critical issues: (1) the need to develop a building surrogate model to perform the training task, and (2) the need to perform a fine-tuning process over several training episodes to obtain a near-optimal control policy.” (ABSTRACT)
Foreign Art:
UKUMORI. “Electrical Storage Element Evaluation Apparatus Equipped With Electrical Storage Element Evaluation System, Has Evaluating Unit For Implementing Action, And For Evaluating State Of Electricity Storage Element Correspondingly.” (WO 2020/090949 A1)
Relevancy: “NOVELTY - The electrical storage element evaluation apparatus has an action selecting unit (63) for selecting an action including changing a loading state of an electricity storage element, on the basis of action evaluation information. A state acquiring unit is used for acquiring the state of the electricity storage element, when the selected action is implemented. A reward acquiring unit is used for acquiring a reward, when the selected action is implemented. An updating unit is used for updating the action evaluation information on the basis of the acquired state and reward. An evaluating unit is used for implementing the action based on the updated action evaluation information, and for evaluating the state of the electricity storage element.” (ABSTRACT)
BAKER et al. “Electrical Energy Storage Device For E.g. Battery-operated Forklift, Has Heating Module Functioning As Internal Control Mechanism To Regulate Temperature Of Electrochemical Cells Within Predetermined Temperature Range.” (WO 2025/114779 A1)
Relevancy: NOVELTY - The device (100) has a heating module (130) coupled to regulate temperature of electrochemical cells (120), where the heating module is configured to function as an internal control mechanism to regulate the temperature of the electrochemical cells within predetermined temperature range with minimum preset temperature and maximum preset temperature. The heating module comprises a temperature sensor arranged approximate to the electrochemical cells, where the temperature sensor is configured to monitor temperature of the electrochemical cells. An enclosure (150) is configured to house the electrochemical cells and the heating module, where the heating module comprises a bus bar (134) physically coupled to a heating element (136) comprising a resistive heating wire. A relay switch unit is configured to control a current flow through the heating element.” (ABSTRACT)
BELLESCHI et al. “Method For Collecting Training Data For Artificial Intelligence/machine Learning Models, Involves Sending Set Of Conditions To Radio Access Network, And Training AI/ML Models Based On Received MDT Measurements.” (WO 2025/168737 A1)
Relevancy: “NOVELTY - The method involves determining a set of conditions related to selection of a user equipment (UE) (540) for minimization of drive testing (MDT) measurements to be used for training artificial intelligence/machine learning (AI/ML) models. The set of the conditions is sent to a radio access network (RAN) node (530) that serves the UE, where the conditions include conditions on one of radio-related measurements, traffic-related measurements, UE mobility state, UE location, UE traffic type, and status for one of the AI/ML models or model functionalities. The AI/ML models are trained based on received MDT measurements. The MDT measurements are received.” (ABSTRACT)
Any inquiry of a general nature or relating to the status of this application or concerning this communication or earlier communications from the Examiner should be directed to James A. Reagan (james.reagan@uspto.gov) whose telephone number is 571.272.6710. The Examiner can normally be reached Monday through Friday from 9 AM to 5 PM. If attempts to reach the examiner by telephone are unsuccessful, the Examiner’s supervisor, John Hayes, can be reached at 571.272.6708.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://portal.uspto.gov/external/portal/pair . Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.217.9197 (toll-free).
Any response to this action should be mailed to:
Commissioner for Patents
PO Box 1450
Alexandria, Virginia 22313-1450
or faxed to 571-273-8300.
Hand delivered responses should be brought to the United States Patent and Trademark Office Customer Service Window:
Randolph Building
401 Dulany Street
Alexandria, VA 22314.
/JAMES A REAGAN/Primary Examiner, Art Unit 3697
james.reagan@uspto.gov
571.272.6710 (Office)
571.273.6710 (Desktop Fax)
Prosecution Insights