DETAILED ACTION
Claims 1-20 are presented for examination. This office action is response to the submission on 5/28/2024.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 5/28/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
The drawings filed on 5/28/2024 are acceptable for examination proceedings.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3-4, 7, 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Walczyk et al. (US20240068692A1) in view of Brown et al. (US20230392813A1).
Claim 1:
Walczyk teaches “and a reinforcement learning (RL) agent configured to receive and process at least real time environmental conditions, occupancy patterns, and a PMV value associated with an installation premises of the HVAC system,” (Walczyk teaches a edge controller 602 that uses a predicted mean vote, mixed air temperature i.e. real time environmental conditions, and occupancy information in Walczyk [0096] "The sensors/meters 606, the occupancy system 610, the weather service 608, and/or the BMS controller 366 combine to provide current state information and a disturbance forecast to the edge controller 602. In some embodiments, the current state includes one or more of the following variables: indoor air temperature setpoint, cooling setpoint, heating setpoint, coil power, HVAC power, actual indoor air temperature, occupant comfort rating (e.g., predicted mean vote, percent of persons with local discomfort), mixed air mass flow, mixed air temperature, outdoor air mass flow rate (e.g., into an air handling unit), outdoor air temperature, return air flow rate, return air temperature, supply air flow rate, supply air temperature, damper position(s), valve position(s), actuator status, fan speed, compressor speed, etc. In some embodiments, the disturbance forecast includes one or more of solar radiation (e.g., direct and/or diffused solar radiation), outdoor relative humidity, outdoor air temperature, wind direction, wind speed, and occupancy information."; Walczyk teaches the edge controller 602 uses reinforcement learning to control actuators based on information from sensors in Walczyk [0083] "The edge controller 602 is configured to (e.g., programed to) control the actuator(s) 604 based on an indoor air temperature setpoint, state information from the sensor(s)/meter(s) 606, and disturbance information from the weather service 610 and/or occupancy system 608 using an artificial intelligence algorithm, for example using a reinforcement learning model. As shown, the edge controller 602 can determine one or more internal control values for a unit of building equipment to be provided to actuator(s) 604. For example, the edge controller 602 can determine a desired change in mixed air temperature for an air handling unit 302 (e.g., an amount by which the mixed air temperature should be changed from a current time step to a subsequent time step). The edge controller 602 and/or the AHU controller 330 can use the desired change in the mixed air temperature for controlling actuators 324-328 which operate dampers 316-320 of the air handling unit 302 to affect the mixed air temperature."),
“identifying one or more optimal actions to adjust one or more HVAC parameters of the HVAC system,” (Walczyk teaches the edge controller 602 may adjust dampers to affect the mixed air temperature i.e. it identifies optimal actions to adjust HVAC parameters in Walczyk [0083] "The edge controller 602 is configured to (e.g., programed to) control the actuator(s) 604 based on an indoor air temperature setpoint, state information from the sensor(s)/meter(s) 606, and disturbance information from the weather service 610 and/or occupancy system 608 using an artificial intelligence algorithm, for example using a reinforcement learning model. As shown, the edge controller 602 can determine one or more internal control values for a unit of building equipment to be provided to actuator(s) 604. For example, the edge controller 602 can determine a desired change in mixed air temperature for an air handling unit 302 (e.g., an amount by which the mixed air temperature should be changed from a current time step to a subsequent time step). The edge controller 602 and/or the AHU controller 330 can use the desired change in the mixed air temperature for controlling actuators 324-328 which operate dampers 316-320 of the air handling unit 302 to affect the mixed air temperature."), and
“and wherein the RL agent optimizes operations of the HVAC system based on the identified optimal actions, ensuring adaptive thermal comfort and energy efficiency management within the installation premises.” (Walczyk teaches the edge controller 602 may adjust dampers to affect the mixed air temperature i.e. it optimizes operations of the HVAC system in Walczyk [0083] "The edge controller 602 is configured to (e.g., programed to) control the actuator(s) 604 based on an indoor air temperature setpoint, state information from the sensor(s)/meter(s) 606, and disturbance information from the weather service 610 and/or occupancy system 608 using an artificial intelligence algorithm, for example using a reinforcement learning model. As shown, the edge controller 602 can determine one or more internal control values for a unit of building equipment to be provided to actuator(s) 604. For example, the edge controller 602 can determine a desired change in mixed air temperature for an air handling unit 302 (e.g., an amount by which the mixed air temperature should be changed from a current time step to a subsequent time step). The edge controller 602 and/or the AHU controller 330 can use the desired change in the mixed air temperature for controlling actuators 324-328 which operate dampers 316-320 of the air handling unit 302 to affect the mixed air temperature.").
Walczyk does not appear to explicitly teach “A system for HVAC (Heating, Ventilation, and Air Conditioning) optimization, the system comprising: a digital twin of an HVAC system, wherein the digital twin is simulated based on at least design parameters, operational data, and PMV (Predicted Mean Vote) based thermal comfort analysis;” or “wherein the digital twin, driven by the identified optimal actions, dynamically simulates the HVAC system,” However, Brown does teach these claim limitations.
Brown teaches “A system for HVAC (Heating, Ventilation, and Air Conditioning) optimization, the system comprising: a digital twin of an HVAC system, wherein the digital twin is simulated based on at least design parameters, operational data, and PMV (Predicted Mean Vote) based thermal comfort analysis;” (Brown teaches a digital twin that executes the processes described in Figs. 19-30 and includes representations of all systems connected to the integration engine 1404 including HVAC 1432 in Brown [0210] "The digital twin 1604 in general is structured to replicate the physical systems associated with the building digitally or virtually. That is, the digital twin 1604 provides a virtual representation of the building and how the inputs affect operation of the building and environments within the building. The digital twin 1604 can includes programmed representations of the integration engine 1404 and all other components, systems, and subsystems connected to the integration engine 1404. The digital twin 1604 allows for testing and information sampling in a digital environment without the need for physical testing and manipulation. In other words, the digital twin 1604 allows an operator to observe potential changes to a subsystem or system without physically changing the real world environment. Below, a number of use cases for the digital twin 1604 and the system architecture 1600 are described. The system architecture can be used to execute any of the processes shown in FIGS. 19-30 , including any combinations thereof."; Brown teaches that sensors may provide inputs i.e. operational data to the digital twin in Brown [0208] "Similar to the intelligent code blue system architecture 1400 discussed above, the system architecture 1600 communicates with building systems and sensors. The building systems and sensors can be used as inputs of the integration engine 1404 and/or the digital twin 1604 and used for determination of actions. In some embodiments, the inputs of the integration engine 1404 and/or the digital twin 1604 include a lighting system 1608, a shade system 1612, an HVAC system 1618 (e.g., the HVAC system 440), an entertainment system 1622, a camera system 1626, a microphone system 1630, a Real time location system (RTLS) 1634, a security system 1638, and/or an elevator system."; Brown teaches using an occupant's preferences to provide comfort adjustments i.e. comfort analysis in Brown [0138] "In some embodiments, the patient room 1002 provides sensor data for one or more manipulated variables to the data collector 1016. The data collector 1016 may also receive patient preferences from the user profiles 704. The data collector may then send this information to the control signal generator, so that the control signal generator 1018 can provide comfort to the patient without having the patient request the adjustments to the manipulated variable. In some embodiments, this information is combined with the processed digital video feed to determine when the patient has arrived in the patient room 1002. In some embodiments (not shown), the command center engine 502 can perform facial recognition to determine which person (i.e., the patient) has arrived in the patient room 1002, and provide comfortability adjustments for the patient."), and
“wherein the digital twin, driven by the identified optimal actions, dynamically simulates the HVAC system,” (Brown teaches the digital twin can be used to automatically react to situations in a building i.e. it dynamically simulates the HVAC system in Brown [0217] "The ability of the digital twin 1604 to learn the specific response tendencies of the actual people operating the building or facility allows the digital twin 1604 to accurately represent the real world responses and controls of the building and staff within the building. Once the base digital twin policy is fully customized and operating within a predetermined tolerance band or threshold accuracy, then the fully trained customized digital twin policy can be instituted on the integration engine 1404 and can be used to automatically react to situations in the room or building to control building systems and subsystems. The digital twin 1604 can continue to run and update in the background for use the integration engine 1404 or for any other use case or desired implementation.").
Walczyk and Brown are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk and Brown before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk to include the digital twin simulation of Brown because adding the Building equipment and environmental control system for a healthcare facility of Brown would allow for testing and information sampling without the need for physical testin as described in Brown [0210] “The digital twin 1604 in general is structured to replicate the physical systems associated with the building digitally or virtually. That is, the digital twin 1604 provides a virtual representation of the building and how the inputs affect operation of the building and environments within the building. The digital twin 1604 can includes programmed representations of the integration engine 1404 and all other components, systems, and subsystems connected to the integration engine 1404. The digital twin 1604 allows for testing and information sampling in a digital environment without the need for physical testing and manipulation. In other words, the digital twin 1604 allows an operator to observe potential changes to a subsystem or system without physically changing the real world environment.”
Claim 3:
Walczyk in view of Brown teaches “The system of claim 1, further comprising one or more sensors and IoT (Internet of Things) devices placed within the installation premises to capture real-time data including at least the environmental conditions, occupancy patterns, and operational data.” (Walczyk teaches a control system 600 including temperature sensors i.e. they environmental conditions, airflow sensors i.e. that capture operational data, and occupancy system 608 that provides state information i.e. occupancy patterns in Walczyk [0082-0083] "In some embodiments, the control system 600 operates with or as part of the airside system 300 described with reference to FIG. 3 . For example, the sensors/meters 606 can include temperature sensors for measuring air temperature at various locations within the air handling unit 302 or upstream or downstream of the air handling unit 302. For example, sensors/meters 606 may include the temperature sensor 362 arranged to measure the temperature of the supply air 310, a temperature sensor 361 arranged to measure a mixed air temperature (i.e., the temperature of mixed air created by mixing the outside air 314 with the return air 304 downstream of the outside air damper 320 and the mixing damper 318, which may be equivalent to the temperature of the air entering the fan 338 upstream of the heating coil 336 and the cooling coil 334), etc. The sensors/meters 606 may also include the sensor 364 shown in FIG. 3 for measuring an indoor air temperature of a building zone 306, a meter for measuring or estimating an amount of energy provided by and/or consumed by the cooling coil 334 and/or heating coil 336, humidity sensors, airflow sensors, air quality sensors, etc. The edge controller 602 is configured to (e.g., programed to) control the actuator(s) 604 based on an indoor air temperature setpoint, state information from the sensor(s)/meter(s) 606, and disturbance information from the weather service 610 and/or occupancy system 608 using an artificial intelligence algorithm, for example using a reinforcement learning model.").
Claim 4:
Walczyk in view of Brown teaches “The system of claim 1, wherein the digital twin employs cloud-based storage and processing, enabling seamless collaboration and data sharing among multiple stakeholders involved in the premises management.” (Brown teaches the digital twin may receive a base policy from the cloud and receive information about other buildings from the cloud in Brown [0211] "As shown in FIG. 19 , the system architecture 1600 can train the digital twin 1604 over time using a self-supervised training method 1700 that utilizes real life response of a medical team as a training input. In some embodiments, the digital twin 1604 include a machine learning or artificial intelligence engine and is initialized with a base digital twin policy. In some embodiments, the base digital twin policy is maintained in the cloud 1456 or in a remote portion of the digital twin 1604 and the base digital twin policy is updated over time based on aggregated information from multiple buildings connected to the cloud 1456. For example, all hospitals or healthcare facilities that are connected to the cloud 1456 provide information to the digital twin 1604 to be used for training the base digital twin policy. In this way, the base digital twin policy can improve over time and require less training within a specific system or building.").
Claim 7:
Walczyk in view of Brown teaches “The system of claim 5, wherein the PMV calculation integrates occupant feedback data, gathered through user interfaces, to refine accuracy of thermal comfort predictions.” (Brown teaches patients providing feedback about actions implemented by the system in Brown [0139] "For example, the system 1000 may receive feedback from the patient about actions implemented by the system 1000 (e.g., thumbs up or thumbs down in response to an implemented change).").
Claim 17:
Walczyk teaches “A method for optimizing HVAC operations in a premises, the method comprising… receiving and processing, by a reinforcement learning (RL) agent, at least real time environmental conditions, occupancy patterns, and a Predicted Mean Vote (PMV) value associated with an installation premises of the HVAC system,” (Walczyk teaches a edge controller 602 that uses a predicted mean vote, mixed air temperature i.e. real time environmental conditions, and occupancy information in Walczyk [0096] "The sensors/meters 606, the occupancy system 610, the weather service 608, and/or the BMS controller 366 combine to provide current state information and a disturbance forecast to the edge controller 602. In some embodiments, the current state includes one or more of the following variables: indoor air temperature setpoint, cooling setpoint, heating setpoint, coil power, HVAC power, actual indoor air temperature, occupant comfort rating (e.g., predicted mean vote, percent of persons with local discomfort), mixed air mass flow, mixed air temperature, outdoor air mass flow rate (e.g., into an air handling unit), outdoor air temperature, return air flow rate, return air temperature, supply air flow rate, supply air temperature, damper position(s), valve position(s), actuator status, fan speed, compressor speed, etc. In some embodiments, the disturbance forecast includes one or more of solar radiation (e.g., direct and/or diffused solar radiation), outdoor relative humidity, outdoor air temperature, wind direction, wind speed, and occupancy information."; Walczyk teaches the edge controller 602 uses reinforcement learning to control actuators based on information from sensors in Walczyk [0083] "The edge controller 602 is configured to (e.g., programed to) control the actuator(s) 604 based on an indoor air temperature setpoint, state information from the sensor(s)/meter(s) 606, and disturbance information from the weather service 610 and/or occupancy system 608 using an artificial intelligence algorithm, for example using a reinforcement learning model. As shown, the edge controller 602 can determine one or more internal control values for a unit of building equipment to be provided to actuator(s) 604. For example, the edge controller 602 can determine a desired change in mixed air temperature for an air handling unit 302 (e.g., an amount by which the mixed air temperature should be changed from a current time step to a subsequent time step). The edge controller 602 and/or the AHU controller 330 can use the desired change in the mixed air temperature for controlling actuators 324-328 which operate dampers 316-320 of the air handling unit 302 to affect the mixed air temperature."),
“identifying one or more optimal actions to adjust one or more HVAC parameters of the HVAC system;” (Walczyk teaches the edge controller 602 may adjust dampers to affect the mixed air temperature i.e. it identifies optimal actions to adjust HVAC parameters in Walczyk [0083] "The edge controller 602 is configured to (e.g., programed to) control the actuator(s) 604 based on an indoor air temperature setpoint, state information from the sensor(s)/meter(s) 606, and disturbance information from the weather service 610 and/or occupancy system 608 using an artificial intelligence algorithm, for example using a reinforcement learning model. As shown, the edge controller 602 can determine one or more internal control values for a unit of building equipment to be provided to actuator(s) 604. For example, the edge controller 602 can determine a desired change in mixed air temperature for an air handling unit 302 (e.g., an amount by which the mixed air temperature should be changed from a current time step to a subsequent time step). The edge controller 602 and/or the AHU controller 330 can use the desired change in the mixed air temperature for controlling actuators 324-328 which operate dampers 316-320 of the air handling unit 302 to affect the mixed air temperature."), and
“and adjusting HVAC parameters based on the identified optimal actions, ensuring adaptive thermal comfort and energy efficiency management within the installation premises.” (Walczyk teaches the edge controller 602 may adjust dampers to affect the mixed air temperature i.e. it optimizes operations of the HVAC system in Walczyk [0083] "The edge controller 602 is configured to (e.g., programed to) control the actuator(s) 604 based on an indoor air temperature setpoint, state information from the sensor(s)/meter(s) 606, and disturbance information from the weather service 610 and/or occupancy system 608 using an artificial intelligence algorithm, for example using a reinforcement learning model. As shown, the edge controller 602 can determine one or more internal control values for a unit of building equipment to be provided to actuator(s) 604. For example, the edge controller 602 can determine a desired change in mixed air temperature for an air handling unit 302 (e.g., an amount by which the mixed air temperature should be changed from a current time step to a subsequent time step). The edge controller 602 and/or the AHU controller 330 can use the desired change in the mixed air temperature for controlling actuators 324-328 which operate dampers 316-320 of the air handling unit 302 to affect the mixed air temperature.").
Walczyk does not appear to explicitly teach “A method for optimizing HVAC operations in a premises, the method comprising: creating a digital twin of an HVAC system based on design parameters, operational data, and PMV-based thermal comfort analysis;” or “dynamically simulating HVAC operations using the digital twin driven by the identified optimal actions;” However, Brown does teach these claim limitations.
Brown teaches “A method for optimizing HVAC operations in a premises, the method comprising: creating a digital twin of an HVAC system based on design parameters, operational data, and PMV-based thermal comfort analysis;” (Brown teaches a digital twin that executes the processes described in Figs. 19-30 and includes representations of all systems connected to the integration engine 1404 including HVAC 1432 in Brown [0210] "The digital twin 1604 in general is structured to replicate the physical systems associated with the building digitally or virtually. That is, the digital twin 1604 provides a virtual representation of the building and how the inputs affect operation of the building and environments within the building. The digital twin 1604 can includes programmed representations of the integration engine 1404 and all other components, systems, and subsystems connected to the integration engine 1404. The digital twin 1604 allows for testing and information sampling in a digital environment without the need for physical testing and manipulation. In other words, the digital twin 1604 allows an operator to observe potential changes to a subsystem or system without physically changing the real world environment. Below, a number of use cases for the digital twin 1604 and the system architecture 1600 are described. The system architecture can be used to execute any of the processes shown in FIGS. 19-30 , including any combinations thereof."; Brown teaches that sensors may provide inputs i.e. operational data to the digital twin in Brown [0208] "Similar to the intelligent code blue system architecture 1400 discussed above, the system architecture 1600 communicates with building systems and sensors. The building systems and sensors can be used as inputs of the integration engine 1404 and/or the digital twin 1604 and used for determination of actions. In some embodiments, the inputs of the integration engine 1404 and/or the digital twin 1604 include a lighting system 1608, a shade system 1612, an HVAC system 1618 (e.g., the HVAC system 440), an entertainment system 1622, a camera system 1626, a microphone system 1630, a Real time location system (RTLS) 1634, a security system 1638, and/or an elevator system."; Brown teaches using an occupant's preferences to provide comfort adjustments i.e. comfort analysis in Brown [0138] "In some embodiments, the patient room 1002 provides sensor data for one or more manipulated variables to the data collector 1016. The data collector 1016 may also receive patient preferences from the user profiles 704. The data collector may then send this information to the control signal generator, so that the control signal generator 1018 can provide comfort to the patient without having the patient request the adjustments to the manipulated variable. In some embodiments, this information is combined with the processed digital video feed to determine when the patient has arrived in the patient room 1002. In some embodiments (not shown), the command center engine 502 can perform facial recognition to determine which person (i.e., the patient) has arrived in the patient room 1002, and provide comfortability adjustments for the patient."), and
“dynamically simulating HVAC operations using the digital twin driven by the identified optimal actions;” (Brown teaches the digital twin can be used to automatically react to situations in a building i.e. it dynamically simulates the HVAC system in Brown [0217] "The ability of the digital twin 1604 to learn the specific response tendencies of the actual people operating the building or facility allows the digital twin 1604 to accurately represent the real world responses and controls of the building and staff within the building. Once the base digital twin policy is fully customized and operating within a predetermined tolerance band or threshold accuracy, then the fully trained customized digital twin policy can be instituted on the integration engine 1404 and can be used to automatically react to situations in the room or building to control building systems and subsystems. The digital twin 1604 can continue to run and update in the background for use the integration engine 1404 or for any other use case or desired implementation.").
Walczyk and Brown are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk and Brown before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk to include the digital twin simulation of Brown because adding the Building equipment and environmental control system for a healthcare facility of Brown would allow for testing and information sampling without the need for physical testing as described in Brown [0210] “The digital twin 1604 in general is structured to replicate the physical systems associated with the building digitally or virtually. That is, the digital twin 1604 provides a virtual representation of the building and how the inputs affect operation of the building and environments within the building. The digital twin 1604 can includes programmed representations of the integration engine 1404 and all other components, systems, and subsystems connected to the integration engine 1404. The digital twin 1604 allows for testing and information sampling in a digital environment without the need for physical testing and manipulation. In other words, the digital twin 1604 allows an operator to observe potential changes to a subsystem or system without physically changing the real world environment.”
Claim 19:
Walczyk in view of Brown teaches “The method of claim 17, further comprising predicting nationality of one or more individuals and using this to compute nationality-specific thermal comfort indices for PMV calculation.” (Brown teaches that historical demographic information may be used to predict a base profile for a patient including nationality, including adjusting temperature in Brown [0139] "The system 1000 is structured to recognize using the sensor arrays and the video feeds the patient's needs without input. The system 1000 is also integrated with the scheduling system so that procedures and appointments are recognized by the system 1000 and integrated into the care provided by the sentient patient room. For example, the system 1000 may utilize historical demographic information to predict a base profile policy for the patient (e.g., the average individual matching the age, gender, nationality, etc. of the patient defines a base profile of preferences), receives inputs and preferences from the patient before a stay in the sentient patient room (e.g., favorite sports team, favorite color, pictures from a past vacation, favorite authors, normal sleep temperature, favorite scents, etc.) that allow the system 1000 to update the base profile policy to provide a customized profile policy in view of patient inputs, and continue to update the customized profile policy using machine learning or artificial intelligence (e.g. neural networks, reinforcement learning, etc.) to improve a response of the sentient patient room by the system 1000 to the patients activities and actions. For example, the system 1000 may receive feedback from the patient about actions implemented by the system 1000 (e.g., thumbs up or thumbs down in response to an implemented change)."; Brown teaches that profiles may send information to the control signal generator to provide comfort, adjusting variables in Brown [0138] "In some embodiments, the patient room 1002 provides sensor data for one or more manipulated variables to the data collector 1016. The data collector 1016 may also receive patient preferences from the user profiles 704. The data collector may then send this information to the control signal generator, so that the control signal generator 1018 can provide comfort to the patient without having the patient request the adjustments to the manipulated variable.").
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Walczyk et al. (US20240068692A1) in view of Brown et al. (US20230392813A1), further in view of Cheng et al. (US20190163215A1).
Claim 2:
Walczyk in view of Brown teaches “The system of claim 1,” as described above. Neither Walczyk or Brown appear to explicitly teach “wherein the digital twin is simulated by using a physics-based model, a mathematics-based model, and a data-based model.” However, Cheng does teach this claim limitation (Cheng teaches a Modeler 406 that trains model 408 using simulator 410 to form a digital twin of a building in Cheng [0038] "The modeler 406 takes APM input regarding actual building properties in conjunction with historical input regarding energy usage information and trains the model using the simulator 410. The modeler 406 is built using the simulator 410 through arithmetic, knowledge, etc., to form a model, digital twin, and/or other digital version of a target building 102-106 (e.g., with corresponding scale size, location, HVAC infrastructure, settings, etc.)."; Cheng teaches using physics and historical data to simulate in Cheng [0039] "Given past energy usage, for example, the model 408 can be trained to use properties of the simulator 410 and/or other model to react to future conditions based on a knowledge of how the model 408 has reacted to energy use in the past. The model 408 can be trained on historical data and tuned based on historical parameter information, user input, scenario characteristics, etc., to prepare the model for deployment and simulation."; Cheng teaches predicted energy use is calculated using model 408 i.e. mathematics model in Cheng [0067] "At block 806, a value associated with each simulated scenario is calculated. For example, a predicted energy usage is generated for each scenario by simulating the scenario using the model 408 and the simulator 410. The predicted energy usage can be combined with the site's average energy cost or tariff, bounded, in some examples, by a benefit of the HVAC settings (e.g., comfort, equipment and/or personnel operating environment, safety, etc.) from the scenario."; Cheng teaches the model may be a digital twin in Cheng [0051-0052] "At block 802, a model of the facility 102-106 energy systems (e.g., systems controlled by the HVAC control 110-114, etc.) is generated. For example, APM service 702 information such as facility 102-106 state, average energy price, etc., HVAC control parameters, performance metrics, weather data, user input, etc., can be combined to form a model of building energy systems, usage, control, etc. The building energy model 408 can be used simulate the building system's capabilities, responses, adjustments, other infrastructure, etc., and can be applied to energy conservation scenarios to model outcome(s) of such scenario(s). In some examples, the model is a digital twin of the building's environmental control systems.").
Walczyk, Brown, and Cheng are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk, Brown, and Cheng before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk modified to include the digital twin simulation of Brown to include the simulation using the model which uses historical data, physics, and mathematics of Cheng because adding the Building energy modeling tool systems and methods of Cheng would allow for further context for simulating the model and allow for savings across sites as described in Cheng [0100] “While manual trial and error is inefficient, ineffective, and impractical across a large enterprise, certain examples enable value determinations across multiple scenarios to selection among HVAC control configuration options according to best cost/benefit and/or other value. Energy conservation savings can be obtained across an enterprise, at multiple sites, at a single site, etc. For example, a chain store can simulate across thousands of stores to determine a solution that provides HVAC settings to improve energy usage at most or all sites.” And in Cheng [0072] “At 1110, the modeler 406 requests historical usage data from the data store 502. At 1112, the data store 502 transmits historical usage data for the site 102-106 to the modeler 406. For example, historical data regarding usage of systems at the site 102-106 can help to expand the model 408 and provide further context and probability for simulating the model 408 according to various scenarios.”
Claims 5-6 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Walczyk et al. (US20240068692A1) in view of Brown et al. (US20230392813A1), further in view of Wenzel et al. (US20200355391A1).
Claim 5:
Walczyk in view of Brown teaches “The system of claim 1,” as described above. Neither Walczyk or Brown appear to explicitly teach “wherein the PMV value is calculated based on at least real-time monitoring of air temperature, radiant temperature, air humidity, air speed, clothing insulation, and metabolic activity, utilizing sensors and IoT devices distributed throughout the installation premises,” or “and wherein the PMV value is adjusted based on the occupancy patterns to reflect anticipated thermal comfort needs of one or more individuals.” However, Cheng does teach these claim limitations.
Cheng teaches “wherein the PMV value is calculated based on at least real-time monitoring of air temperature, radiant temperature, air humidity, air speed, clothing insulation, and metabolic activity, utilizing sensors and IoT devices distributed throughout the installation premises,” (Wenzel teaches calculating PMV using temperature, humidity, radiant temperature, air velocity, metabolic rate, and cloth index in Wenzel [0140] "As described in greater detail below, comfort controller 9000 can perform a zone simulation to generate data used to train a neural network. To appropriately structure a zone simulation, an occupant's comfort with regards to current zone conditions should be quantified. A predicted mean vote (PMV) index can be used as a measure of thermal comfort, and is the primary method used to asses comfort in the aforementioned zone simulation. PMV is adopted by the ISO 7730 standard. The ISO recommends maintaining PMV at level 0 with a tolerance of 0.5 as the best thermal comfort. PMV can be based on theoretical model and includes results from experimental results with approximately 1300 subjects and includes four variables: temperature T, relative humidity Φ, mean radiant temperature Trm, and air velocity ν. The index may also include two individual parameters: metabolic rate M and cloth index Icl. Based on the variables and parameters, the PMV can be calculated using Fanger's Equation as given by: PMV=G(t bs ,Φ,T rm ,ν,M,I cl)"), and
“and wherein the PMV value is adjusted based on the occupancy patterns to reflect anticipated thermal comfort needs of one or more individuals.” (Wenzel teaches a specific occupant having comfort preferences i.e. if that occupant is detected, the PMV is adjusted in Wenzel [0155] "Another important aspect of the zone simulation performed by zone simulator 9012 is a response of occupants to the indoor air temperature and humidity. In is sense, a four variable comfort control approach can be integrated to allow the simulator to detect whether or not certain occupants would be uncomfortable given current conditions. In some embodiments, the occupant simulation includes carbon dioxide generation and/or heat effects on the zone by occupants. The variables that occupant simulation may depend on include metabolic rate, clothing insulation, and the specific occupant's comfort preferences (in terms of the PMV index). Said conditions can be combined to determine whether or not an occupant is comfortable.").
Walczyk, Brown, and Wenzel are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk, Brown, and Wenzel before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk modified to include the digital twin simulation of Brown to include the determination of PMV value using the six standard variables of Wenzel because adding the Predictive building control system with neural network based comfort prediction of Wenzel would allow for determining whether the occupant is comfortable, which could prevent the occupant from overriding the setpoint as described in Wenzel [0155] “Another important aspect of the zone simulation performed by zone simulator 9012 is a response of occupants to the indoor air temperature and humidity. In is sense, a four variable comfort control approach can be integrated to allow the simulator to detect whether or not certain occupants would be uncomfortable given current conditions. In some embodiments, the occupant simulation includes carbon dioxide generation and/or heat effects on the zone by occupants. The variables that occupant simulation may depend on include metabolic rate, clothing insulation, and the specific occupant's comfort preferences (in terms of the PMV index). Said conditions can be combined to determine whether or not an occupant is comfortable.” And Wenzel [0160] “If an occupant is uncomfortable, that is, the occupant's PMV value is outside of their comfort bounds, the occupant may want to override the setpoint (e.g., move the setpoint up or down). In detail, the metabolic rate, clothing insulation, and zone temperature and humidity simulations are fed into the PMV calculation performed by zone simulator 9012, which can output an index value for the occupant in question. The PMV index can be compared to the minimum and maximum PMV comfort bounds that describe the specific occupant's preferences. If the PMV value is below the minimum comfort bound, the occupant may increase the setpoint to a temperature that results in a PMV value within their comfort bounds. Similarly, if the PMV is above the maximum comfort bound, the occupant may decrease the setpoint to a temperature that results in a PMV value within their bounds. This process can be carried out in each time-step of the simulation performed by zone simulator 9012. Every time PMV is calculated, the instantaneous zone conditions can be used to determine an occupant's comfort.”
Claim 6:
Walczyk in view of Brown, further in view of Wenzel teaches “The system of claim 5, further comprising a nationality prediction device for predicting nationality of the one or more individuals, wherein the predicted nationality is used for real time computation of nationality-specific thermal comfort indices, enabling the PMV calculation to account for cultural preferences in the thermal comfort.” (Brown teaches that historical demographic information may be used to predict a base profile for a patient including nationality, including adjusting temperature in Brown [0139] "The system 1000 is structured to recognize using the sensor arrays and the video feeds the patient's needs without input. The system 1000 is also integrated with the scheduling system so that procedures and appointments are recognized by the system 1000 and integrated into the care provided by the sentient patient room. For example, the system 1000 may utilize historical demographic information to predict a base profile policy for the patient (e.g., the average individual matching the age, gender, nationality, etc. of the patient defines a base profile of preferences), receives inputs and preferences from the patient before a stay in the sentient patient room (e.g., favorite sports team, favorite color, pictures from a past vacation, favorite authors, normal sleep temperature, favorite scents, etc.) that allow the system 1000 to update the base profile policy to provide a customized profile policy in view of patient inputs, and continue to update the customized profile policy using machine learning or artificial intelligence (e.g. neural networks, reinforcement learning, etc.) to improve a response of the sentient patient room by the system 1000 to the patients activities and actions. For example, the system 1000 may receive feedback from the patient about actions implemented by the system 1000 (e.g., thumbs up or thumbs down in response to an implemented change)."; Brown teaches that profiles may send information to the control signal generator to provide comfort, adjusting variables in Brown [0138] "In some embodiments, the patient room 1002 provides sensor data for one or more manipulated variables to the data collector 1016. The data collector 1016 may also receive patient preferences from the user profiles 704. The data collector may then send this information to the control signal generator, so that the control signal generator 1018 can provide comfort to the patient without having the patient request the adjustments to the manipulated variable.").
Claim 18:
Walczyk in view of Brown teaches “The method of claim 17,” as described above. Neither of Walczyk or Brown appear to explicitly teach “wherein calculating the PMV value is based on air temperature, radiant temperature, air humidity, air speed, clothing insulation, and metabolic activity, and wherein the PMV value is adjusted based on the occupancy patterns to reflect anticipated thermal comfort needs of one or more individuals.” However, Wenzel does teach this claim limitation (Wenzel teaches calculating PMV using temperature, humidity, radiant temperature, air velocity, metabolic rate, and cloth index in Wenzel [0140] "As described in greater detail below, comfort controller 9000 can perform a zone simulation to generate data used to train a neural network. To appropriately structure a zone simulation, an occupant's comfort with regards to current zone conditions should be quantified. A predicted mean vote (PMV) index can be used as a measure of thermal comfort, and is the primary method used to asses comfort in the aforementioned zone simulation. PMV is adopted by the ISO 7730 standard. The ISO recommends maintaining PMV at level 0 with a tolerance of 0.5 as the best thermal comfort. PMV can be based on theoretical model and includes results from experimental results with approximately 1300 subjects and includes four variables: temperature T, relative humidity Φ, mean radiant temperature Trm, and air velocity ν. The index may also include two individual parameters: metabolic rate M and cloth index Icl. Based on the variables and parameters, the PMV can be calculated using Fanger's Equation as given by: PMV=G(t bs ,Φ,T rm ,ν,M,I cl)"; Wenzel teaches a specific occupant having comfort preferences i.e. if that occupant is detected, the PMV is adjusted in Wenzel [0155] "Another important aspect of the zone simulation performed by zone simulator 9012 is a response of occupants to the indoor air temperature and humidity. In is sense, a four variable comfort control approach can be integrated to allow the simulator to detect whether or not certain occupants would be uncomfortable given current conditions. In some embodiments, the occupant simulation includes carbon dioxide generation and/or heat effects on the zone by occupants. The variables that occupant simulation may depend on include metabolic rate, clothing insulation, and the specific occupant's comfort preferences (in terms of the PMV index). Said conditions can be combined to determine whether or not an occupant is comfortable.").
Walczyk, Brown, and Wenzel are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk, Brown, and Wenzel before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk modified to include the digital twin simulation of Brown to include the determination of PMV value using the six standard variables of Wenzel because adding the Predictive building control system with neural network based comfort prediction of Wenzel would allow for determining whether the occupant is comfortable, which could prevent the occupant from overriding the setpoint as described in Wenzel [0155] “Another important aspect of the zone simulation performed by zone simulator 9012 is a response of occupants to the indoor air temperature and humidity. In is sense, a four variable comfort control approach can be integrated to allow the simulator to detect whether or not certain occupants would be uncomfortable given current conditions. In some embodiments, the occupant simulation includes carbon dioxide generation and/or heat effects on the zone by occupants. The variables that occupant simulation may depend on include metabolic rate, clothing insulation, and the specific occupant's comfort preferences (in terms of the PMV index). Said conditions can be combined to determine whether or not an occupant is comfortable.” And Wenzel [0160] “If an occupant is uncomfortable, that is, the occupant's PMV value is outside of their comfort bounds, the occupant may want to override the setpoint (e.g., move the setpoint up or down). In detail, the metabolic rate, clothing insulation, and zone temperature and humidity simulations are fed into the PMV calculation performed by zone simulator 9012, which can output an index value for the occupant in question. The PMV index can be compared to the minimum and maximum PMV comfort bounds that describe the specific occupant's preferences. If the PMV value is below the minimum comfort bound, the occupant may increase the setpoint to a temperature that results in a PMV value within their comfort bounds. Similarly, if the PMV is above the maximum comfort bound, the occupant may decrease the setpoint to a temperature that results in a PMV value within their bounds. This process can be carried out in each time-step of the simulation performed by zone simulator 9012. Every time PMV is calculated, the instantaneous zone conditions can be used to determine an occupant's comfort.”
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Walczyk et al. (US20240068692A1) in view of Brown et al. (US20230392813A1), further in view of Wenzel et al. (US20200355391A1), further in view of G. Gao, J. Li and Y. Wen, "DeepComfort: Energy-Efficient Thermal Comfort Control in Buildings Via Reinforcement Learning," in IEEE Internet of Things Journal, vol. 7, no. 9, pp. 8472-8484, Sept. 2020, doi: 10.1109/JIOT.2020.2992117. (hereinafter referred to as “Gao”).
Claim 8:
Walczyk in view of Brown, further in view of Wenzel teaches “The system of claim 5,” as described above. None of Walczyk, Brown, or Wenzel appear to explicitly teach “wherein the PMV calculation employs one or more machine learning algorithms to analyze at least the monitored data and historical PMV data, enhancing the system's ability to predict thermal comfort requirements under varying conditions.” However, Gao does teach this claim limitation (Gao teaches calculating a thermal comfort value using a neural network with the inputs being temperature, humidity, radiant temperature, air speed, metabolic rate, and clothing insulation in Gao [Page 8476 last paragraph - page 8477 first paragraph] "We adopt the deep feedforward neural network for predicting thermal comfort. The structure of the neural network for predicting thermal comfort is illustrated in Fig. 4. The inputs of the neural network include air temperature, humidity, mean radiant temperature, air speed, metabolic rate, and clothing insulation. All of these values are numerical. The hidden layer of the neural network has two layers, and the output layer has one neuron. The output of the neural network is the predicted thermal comfort value. The activation function of the hidden layer is a sigmoid function, and the activation function of the output layer is a linear function." and in Gao Fig. 4
[AltContent: rect]
PNG
media_image1.png
495
799
media_image1.png
Greyscale
).
Walczyk, Brown, Wenzel, and Gao are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk, Brown, Wenzel, and Gao before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk modified to include the digital twin simulation of Brown, further modified to include the determination of PMV value using the six standard variables of Wenzel to include the calculation of a thermal comfort value using a neural network of Gao because adding the Thermal comfort control in buildings via reinforcement learning of Gao would allow for an improvement in prediction accuracy of thermal comfort as described in Gao [Page 8473, second paragraph] “We design a deep FNN with Bayesian regularization for predicting thermal comfort. Our method can improve the prediction accuracy by 14.5% in terms of mean-square error (MSE).
Claims 9-14 are rejected under 35 U.S.C. 103 as being unpatentable over Walczyk et al. (US20240068692A1) in view of Brown et al. (US20230392813A1), further in view of Barnes (US20220268475A1).
Claim 9:
Walczyk in view of Brown teaches “The system of claim 1, wherein the RL agent utilizes the PMV value, energy consumption, (Walczyk teaches a predicted mean vote being provided to the BMS controller in Walczyk [0096] "The sensors/meters 606, the occupancy system 610, the weather service 608, and/or the BMS controller 366 combine to provide current state information and a disturbance forecast to the edge controller 602. In some embodiments, the current state includes one or more of the following variables: indoor air temperature setpoint, cooling setpoint, heating setpoint, coil power, HVAC power, actual indoor air temperature, occupant comfort rating (e.g., predicted mean vote, percent of persons with local discomfort),"; Walczyk teaches an objective being energy usage in Walczyk [0111] "At step 860, learning of the model is reinforced, for example based on a building setpoint and a corresponding measurement. The building setpoint may be an indoor air temperature setpoint and the corresponding measurement may be a measurement of the indoor air temperature. Reinforcing learning of the model can include adjusting one or more parameters of the model to improve a value of a reward function, for example where the reward function includes a term based on a difference between the building setpoint and the corresponding measurement. The reward function may also include a term based on energy usage of the equipment and/or a proxy for energy usage. For example, in some embodiments the control value can be used as a proxy for energy usage. Reinforcing learning of the model by adjusting parameters based on such a reward function can improve the model's ability over time to generate a control value which reduces error between the building setpoint and the corresponding measurement while also reducing energy usage or other objective.").
Neither Walczyk or Brown appear to explicitly teach “The system of claim 1, wherein the RL agent utilizes the However, Barnes does teach this claim limitation (Barnes teaches a model 392-1 that uses an objective function 452 to meet desired outcomes for air with energy usage needing to be within tolerance in Barnes [0074] "Model 392-1 is applied in an attempt to optimize for an objective function 452 defined by a desired air handling outcome. More specifically, in step 410 (FIG. 4A), the prediction model 392-1 (having been trained on historical data) is now run using the most current (that is, subsequently received) sensed data (including, e.g., air quality sensor data 454 and/or ambient weather data 456), applying error minimization based on the previously-set desired outcomes for air in the facility. FIG. 4C illustrates this process where, once the behavior of the air in the facility can be simulated, error minimization (module 460) is applied over the trained model 445 (in step 412) to generate optimal air-handling control instructions (465) that work within the tolerances of defined optimization targets 452, such as energy usage, particulate concentration, temperature, humidity, and pressure at every point in the facility, those targets being identified in step 408. The output of the model 445 are air-handler control instructions 465, which instructions are then sent by the gateway 150 to the appropriate air-handling and distribution units via the mesh network 220."; Barnes teaches using pressure gradients to calculate flow of air and it is used in air quality prediction model Barnes [0086] "The calculation of the relevant gradient values may be performed by the environment modeling logic 175 or in some embodiments, by control logic 350 (or exception handling logic 354) at the gateway 150, in a data gathering stage. In one embodiment, the gateway 150 may (in step 406 of FIG. 4) calculate at least gradient such as a temperature gradient, a contaminant level gradient, and/or an air pressure gradient based on received sensor measurements. An unexpected gradient (e.g., a high rate of change) may trigger exception handling actions (step 420). In one embodiment, the gateway 150 may send the calculated gradient(s) (or the data from which calculation of gradient can be performed) to the server 170, where logic 175 may use such data to model air flow. For instance, logic 175 may perform thermal stratification, air pressure stratification, and/or any other type of grouping or classification of various physical locations or ranges within the facility 110. These calculations, and/or the modeled flow of air, may be weighted and applied in the execution of the generated air quality prediction model in the manner described above with reference to FIGS. 3A-5B.").
Walczyk, Brown, and Barnes are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk, Brown, and Barnes before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk modified to include the digital twin simulation of Brown to include the model using pressure gradients of Barnes because adding the Machine learning systems for modeling and balancing the activity of air quality devices in industrial applications of Barnes would allow for calculation of gradients in order to determine actions to be taken and employ solutions more efficiently as described in Barnes [0089] “In some cases, calculated gradients may be used to sense certain conditions for which a corrective action is desired. As an example, if the gradient between two temperatures, such as a temperature measured by a sensor at a lower altitude and temperature measured by a sensor at a high altitude in the same room or area of the facility exceeds a threshold, a particular action may be taken, such as activation of a fan or other air handling equipment or a change to one or more setpoints. The action taken may be selected to reduce or other change the measured gradient. Similarly, an action may be taken to reduce or otherwise change a measured pressure gradient between the pressures measured by different sensors in different areas of the facility or between a pressure measured inside of the facility and atmospheric pressure measured outside of the facility.” And in Barnes [0093] “The systems and methods described above allow for automated intelligent control of multiple individually-controlled air handling units in support of overall environmental health and safety while optimizing for the goals of the site administrator. More specifically, the systems and methods described herein apply machine learning to learn how the facility as a whole behaviors as a cohesive unit. Further, the described machine learning algorithms that optimize for any of several different goals, a facility-specific and industry-specific set of rules can be sustained. Because of this, air handling systems that use a computer system employing the solutions described herein perform more efficiently, faster (with reduced execution time) and with less wasted computing resources, and in safer and more sustainable, energy-efficient manners. Accordingly, the uptime of the both the computing systems and the overall machine systems within the facility is improved.”
Claim 10:
Walczyk in view of Brown, further in view of Barnes teaches “The system of claim 9, wherein the RL agent adapts its strategies based on historical PMV convergence, ensuring continuous improvement in balancing the thermal comfort and energy efficiency.” (Walczyk teaches adjusting the model overtime as building dynamics change due to seasonal changes i.e. the historical seasonal data may be used in future years in Walczyk [0092] "The model creation computing system 614 can then provide the trained model to the edge controller 602. The model may be received the edge controller 602 in a sufficiently-trained state to initiate online operations of the edge controller 602. The edge controller 602 may then perform reinforcement learning during online operations to fine-tune the model to actual building dynamics and to adjust the model overtime as building dynamics change (due to seasonal changes, space reconfigurations, utilization and occupancy changes, etc.)."; Walczyk teaches that control values converge to an ideal control approach over time in Walczyk [0113] "As shown in the fourth graph 902 of control values over time, the control value determined by the edge model has deviations from an expert simulated control value at the beginning of reinforcement learning but converges over time such that, towards the right end of the graph (later in time), performance of the edge model substantially imitates performance of a theoretical/ideal (e.g., simulated) expert control approach.").
Claim 11:
Walczyk in view of Brown, further in view of Barnes teaches “The system of claim 9, wherein the RL agent incorporates PMV mapping data to dynamically adjust temperature and airflow settings in different zones of the installation premises, responding to diverse thermal comfort requirements.” (Walczyk teaches the control system 600 includes components of BMS 400 and 500 in Walczyk [0080] "Referring now to FIGS. 6-7 , block diagrams of a control system 600 are shown, according to some embodiments. In some embodiments, the control system 600 is part of a building management system and/or includes components which are included in a building management system (e.g., BMS 400, BMS 500)."; Walczyk teaches zone controllers controlling their zone using reinforcement learning in Walczyk [0079] "Each zone controller 524, 530-532, 536, and 548-550 can be configured to monitor and control a different building zone. Zone controllers 524, 530-532, 536, and 548-550 can use the inputs and outputs provided via their SA busses to monitor and control various building zones. For example, a zone controller 536 can use a temperature input received from networked sensors 538 via SA bus 566 (e.g., a measured temperature of a building zone) as feedback in a temperature control algorithm. Zone controllers 524, 530-532, 536, and 548-550 can use various types of control algorithms (e.g., state-based algorithms, extremum seeking control (ESC) algorithms, proportional-integral (PI) control algorithms, proportional-integral-derivative (PID) control algorithms, model predictive control (MPC) algorithms, feedback control algorithms, etc.) to control a variable state or condition (e.g., temperature, humidity, airflow, lighting, etc.) in or around building 10."; Walczyk teaches a current state which is used by the edge controller being PMV in Walczyk [0096] "The sensors/meters 606, the occupancy system 610, the weather service 608, and/or the BMS controller 366 combine to provide current state information and a disturbance forecast to the edge controller 602. In some embodiments, the current state includes one or more of the following variables: indoor air temperature setpoint, cooling setpoint, heating setpoint, coil power, HVAC power, actual indoor air temperature, occupant comfort rating (e.g., predicted mean vote, percent of persons with local discomfort)").
Claim 12:
Walczyk in view of Brown, further in view of Barnes teaches “The system of claim 9, wherein the RL agent integrates the PMV value with the occupancy patterns, optimizing HVAC operations to accommodate movement of one or more individuals within the installation premises, ensuring consistent thermal comfort levels.” (Walczyk teaches an edge controller 602 that uses a predicted mean vote and occupancy information to optimize HVAC operations in Walczyk [0096] "The sensors/meters 606, the occupancy system 610, the weather service 608, and/or the BMS controller 366 combine to provide current state information and a disturbance forecast to the edge controller 602. In some embodiments, the current state includes one or more of the following variables: indoor air temperature setpoint, cooling setpoint, heating setpoint, coil power, HVAC power, actual indoor air temperature, occupant comfort rating (e.g., predicted mean vote, percent of persons with local discomfort), mixed air mass flow, mixed air temperature, outdoor air mass flow rate (e.g., into an air handling unit), outdoor air temperature, return air flow rate, return air temperature, supply air flow rate, supply air temperature, damper position(s), valve position(s), actuator status, fan speed, compressor speed, etc. In some embodiments, the disturbance forecast includes one or more of solar radiation (e.g., direct and/or diffused solar radiation), outdoor relative humidity, outdoor air temperature, wind direction, wind speed, and occupancy information.").
Claim 13:
Walczyk in view of Brown, further in view of Barnes teaches “The system of claim 9, wherein the RL agent generates the optimal actions to converge to the relevant PMV and reduce power consumption by simulating various HVAC scenarios based on the real time environmental data, occupancy patterns, and PMV values.” (Walczyk teaches that control values converge to an ideal control approach over time in Walczyk [0113] "As shown in the fourth graph 902 of control values over time, the control value determined by the edge model has deviations from an expert simulated control value at the beginning of reinforcement learning but converges over time such that, towards the right end of the graph (later in time), performance of the edge model substantially imitates performance of a theoretical/ideal (e.g., simulated) expert control approach."; Walczyk teaches reinforcement learning is used to improve energy usage in Walczyk [0111] "At step 860, learning of the model is reinforced, for example based on a building setpoint and a corresponding measurement. The building setpoint may be an indoor air temperature setpoint and the corresponding measurement may be a measurement of the indoor air temperature. Reinforcing learning of the model can include adjusting one or more parameters of the model to improve a value of a reward function, for example where the reward function includes a term based on a difference between the building setpoint and the corresponding measurement. The reward function may also include a term based on energy usage of the equipment and/or a proxy for energy usage."; Walczyk teaches the edge controller determining a mixed air temperature by using a simulation in Walczyk [0090] "For example, in an example where the edge controller 602 determines a mixed air temperature for controlling an air handling unit, the mode creation computing system 614 can execute, within the simulation and using a model predictive control algorithm, a simulated mixed air temperatures which minimize a simulated objective associated with running a simulated air handling unit in the simulation."; Walczyk teaches a edge controller 602 that uses a predicted mean vote, mixed air temperature i.e. real time environmental conditions, and occupancy information in Walczyk [0096] "The sensors/meters 606, the occupancy system 610, the weather service 608, and/or the BMS controller 366 combine to provide current state information and a disturbance forecast to the edge controller 602. In some embodiments, the current state includes one or more of the following variables: indoor air temperature setpoint, cooling setpoint, heating setpoint, coil power, HVAC power, actual indoor air temperature, occupant comfort rating (e.g., predicted mean vote, percent of persons with local discomfort), mixed air mass flow, mixed air temperature, outdoor air mass flow rate (e.g., into an air handling unit), outdoor air temperature, return air flow rate, return air temperature, supply air flow rate, supply air temperature, damper position(s), valve position(s), actuator status, fan speed, compressor speed, etc. In some embodiments, the disturbance forecast includes one or more of solar radiation (e.g., direct and/or diffused solar radiation), outdoor relative humidity, outdoor air temperature, wind direction, wind speed, and occupancy information.").
Claim 14:
Walczyk in view of Brown, further in view of Barnes teaches “The system of claim 13, wherein the optimal actions include adaptive HVAC parameter suggestions, providing instant recommendations for adjusting temperature, airflow, and operation of one or more HVAC equipment of the HVAC system to achieve the optimal thermal comfort and energy efficiency.” (Walczyk teaches that the examples referring to temperature may be used for airflow as well in Walczyk [0084] "The edge controller 602 can receive values of the indoor air temperature setpoint for multiple time steps (e.g., for a few hours ahead) and such that the indoor air temperature setpoint changes over time. The examples herein referring to indoor air temperature may be adapted for other states or conditions of a building, such as humidity, pressure, air quality, airflow, etc."; Walczyk teaches the edge controller determines an action which changes the state of the system to adjust temperature in Walczyk [0086-0087] "The edge controller 602 is programmed to determine an action, shown as the internal control value in FIG. 6 . In some embodiments, the action is denoted as Ut=u and is determined given the current state Xt=x based on a policy πθ. Operations executed by the edge controller 602 in such embodiments can be expressed as: u=πθ(x) to determine the control action to take. In some embodiments, disturbances d are also inputs to the model, such that u=πθ(x, d). The policy πθ can be provided as an artificial intelligence model, for example a neural network model with parameters θ. Execution of action u via control of actuators 604 leads to a change in the state to Xt+1=x′ based on the system dynamics (e.g., thermodynamics of a building space) and leads to a reward Rt+1=r. The reward R may be based on an objective function (cost function), for example based on a deviation of a measured indoor temperature from the indoor temperature setpoint and/or an energy consumption of the equipment unit (e.g., r=−C(x,u) with the reward representing a negative of cost C such that the reward r is maximized by the edge controller 602). ").
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Walczyk et al. (US20240068692A1) in view of Brown et al. (US20230392813A1), further in view of Barnes (US20220268475A1), further in view of F. Cicirelli, A. Guerrieri, C. Mastroianni, G. Spezzano and A. Vinci, "Thermal comfort management leveraging deep reinforcement learning and human-in-the-loop," 2020 IEEE International Conference on Human-Machine Systems (ICHMS), Rome, Italy, 2020, pp. 1-6, doi: 10.1109/ICHMS49158.2020.9209555. (hereinafter referred to as “Cicirelli”).
Claim 15:
Walczyk in view of Brown, further in view of Barnes teaches “The system of claim 14, whereinof temperature, fan speed, and other operational parameters in response to changing conditions.” (Walczyk teaches the edge controller 602 uses reinforcement learning to control actuators based on information from sensors i.e. it makes automated adjustments to temperature and operational parameters in Walczyk [0083] "The edge controller 602 is configured to (e.g., programed to) control the actuator(s) 604 based on an indoor air temperature setpoint, state information from the sensor(s)/meter(s) 606, and disturbance information from the weather service 610 and/or occupancy system 608 using an artificial intelligence algorithm, for example using a reinforcement learning model. As shown, the edge controller 602 can determine one or more internal control values for a unit of building equipment to be provided to actuator(s) 604. For example, the edge controller 602 can determine a desired change in mixed air temperature for an air handling unit 302 (e.g., an amount by which the mixed air temperature should be changed from a current time step to a subsequent time step). The edge controller 602 and/or the AHU controller 330 can use the desired change in the mixed air temperature for controlling actuators 324-328 which operate dampers 316-320 of the air handling unit 302 to affect the mixed air temperature.").
None of Walczyk, Brown, or Barnes appear to explicitly teach “The system of claim 14, wherein the optimal actions are validated over the digital twin” However, Cicirelli does teach this claim limitation (Cicirelli teaches a deep reinforcement learning controller that may use a digital twin to predict a result of an action by simulation in Cicirelli [Page 2, Section III, second paragraph] "Figure 2 shows how the approach is specialized for the management of a cognitive building. The approach is founded on a controller that takes a set of environmental and energy parameters as inputs and, by using a DRL-based controller, operates on the environment through a set of actuators, thus creating a cycle that learns from experience how to reduce energy consumption and improve people comfort. The control cycle interacts both with a real environment and with its digital counterpart, in accordance with the digital twin paradigm. The presence of the digital twin is helpful both in the learning phase, when the neural network needs to be trained with a large number of experiments, and in the operating phase, since it can be useful to predict the result of a given action by simulation. In particular, Figure 2 shows the two components of the digital counterpart, i.e., the simulator that models the evolution of the environment and a robot that simulates the human interaction." and in Cicirelli [Page 3, second paragraph] "The DRL-based controller uses a reward function, which weighs the human and the environmental rewards and computes the overall reward. The controller uses a Deep Q-network [6] to choose the action that is expected to maximize the value of the overall reward. The chosen action consists of the settings of the HVAC system, of the doors, and of the windows. The action can be operated either on the physical environment or on its digital twin, thus closing the cycle.").
Walczyk, Brown, Barnes, and Cicirelli are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk, Brown, Barnes, and Cicirelli before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk modified to include the digital twin simulation of Brown, further modified to include the model using pressure gradients of Barnes to include the reinforcement learning controller that uses a digital twin to predict a result of an action by simulation of Cicirelli because adding the thermal comfort management leveraging deep reinforcement learning of Cicirelli would allow for prediction of a result during the operating phase as described in Cicirelli [Page 2, Section III, second paragraph] "Figure 2 shows how the approach is specialized for the management of a cognitive building. The approach is founded on a controller that takes a set of environmental and energy parameters as inputs and, by using a DRL-based controller, operates on the environment through a set of actuators, thus creating a cycle that learns from experience how to reduce energy consumption and improve people comfort. The control cycle interacts both with a real environment and with its digital counterpart, in accordance with the digital twin paradigm. The presence of the digital twin is helpful both in the learning phase, when the neural network needs to be trained with a large number of experiments, and in the operating phase, since it can be useful to predict the result of a given action by simulation. In particular, Figure 2 shows the two components of the digital counterpart, i.e., the simulator that models the evolution of the environment and a robot that simulates the human interaction."
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Walczyk et al. (US20240068692A1) in view of Brown et al. (US20230392813A1), further in view of Scholten et al. (US20250110454A1).
Claim 16:
Walczyk in view of Brown teaches “The system of claim 1… and wherein the environmental conditions include at least air temperature, (Walczyk teaches that the edge controller 602 uses a mixed air temperature in Walczyk [0096] "The sensors/meters 606, the occupancy system 610, the weather service 608, and/or the BMS controller 366 combine to provide current state information and a disturbance forecast to the edge controller 602. In some embodiments, the current state includes one or more of the following variables: indoor air temperature setpoint, cooling setpoint, heating setpoint, coil power, HVAC power, actual indoor air temperature, occupant comfort rating (e.g., predicted mean vote, percent of persons with local discomfort), mixed air mass flow, mixed air temperature, outdoor air mass flow rate (e.g., into an air handling unit), outdoor air temperature, return air flow rate, return air temperature, supply air flow rate, supply air temperature, damper position(s), valve position(s), actuator status, fan speed, compressor speed, etc. In some embodiments, the disturbance forecast includes one or more of solar radiation (e.g., direct and/or diffused solar radiation), outdoor relative humidity, outdoor air temperature, wind direction, wind speed, and occupancy information."), and
“wherein the operational data includes real-time data related to at least one of (Brown teaches the HVAC subsystem 440 including temperature sensors in Brown [0073] "Each of building subsystems 428 can include any number of devices, controllers, and connections for completing its individual functions and control activities. HVAC subsystem 440 can include many of the same components as HVAC system 100, as described with reference to FIGS. 1-3 . For example, HVAC subsystem 440 can include a chiller, a boiler, any number of air handling units, economizers, field controllers, supervisory controllers, actuators, temperature sensors, and other devices for controlling the temperature, humidity, airflow, or other variable conditions within building 10.”).
Neither Walczyk or Brown appear to explicitly teach “wherein the design parameters include specifications of at least one of an air conditioner, a compressor, a condenser, a thermal expansion valve, an air handling unit, air filter and a chiller unit of the HVAC system,” However, Scholten does teach this claim limitation (Scholten teaches a digital twin which includes using data on the specifications of a chiller unit in Scholten [0022-0023] "FIG. 3 shows an example data collection process 300 according to some embodiments of the disclosure. System 100 can perform process 300 to gather data descriptive of equipment 10 operations and prepare the data for subsequent processing to build a digital twin of equipment 10. For example, system 100 can perform process 300 at 202 of process 200. At 302, system 100 can assemble data from the various sources described above (e.g., equipment 10 sensor(s) 20, energy meter(s) 30, and/or controller 40) in data store 110. This can take place over a long period of time to ensure the data set is rich and detailed, for example over the course of a year or some other length of time. Once the data has been assembled in data store 110, ML processing 120 can read the data and load the data into a dataframe for subsequent processing. Data assembled by system 100 may vary according to specific equipment 10 type being modeled, or even by specific piece of equipment 10, but as an example, the following data may be gathered for a chiller in some embodiments: entering temperatures, leaving temperatures, compressor discharge pressure, condenser refrigeration pressure, condenser section pressure, condenser discharge pressure, evaporator entering temperature, condenser entering temperature, condenser leaving temperature, compressor suction temperature, chiller apparent power, chiller primary leaving temperature, compressor apparent power, compressor current, and/or compressor voltage.").
Walczyk, Brown, and Scholten are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk, Brown, and Scholten before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk modified to include the digital twin simulation of Brown to include the design parameters being included with the digital twin of Scholten because adding the Digital twin systems management of Scholten would allow for analysis of operational scenarios enabling changes to the equipment as described in Scholten [0041] “FIG. 5A shows an example simulation process 500, and FIG. 5B shows an example simulation UI 550, according to some embodiments of the disclosure. System 100 can perform process 500 to simulate equipment 10 operations using a digital twin of equipment 10 and display the simulation. At least some tests performed within simulation process 500 can be “what-if” analyses where operational scenarios are simulated and digital twin performance in such scenarios is evaluated, enabling changes to the real equipment 10 to realize performance and/or efficiency improvements expected in view of digital twin performance. For example, sequence of operations and/or settings may be altered for the digital twin, changes due to these alterations may be evaluated, and real-world sequences of operations and/or settings may be specified for equipment 10 according to the evaluation. In some embodiments, system 100 can perform process 500 at 206-208 of process 200.”
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Walczyk et al. (US20240068692A1) in view of Brown et al. (US20230392813A1), further in view of F. Cicirelli, A. Guerrieri, C. Mastroianni, G. Spezzano and A. Vinci, "Thermal comfort management leveraging deep reinforcement learning and human-in-the-loop," 2020 IEEE International Conference on Human-Machine Systems (ICHMS), Rome, Italy, 2020, pp. 1-6, doi: 10.1109/ICHMS49158.2020.9209555. (hereinafter referred to as “Cicirelli”).
Claim 20:
Walczyk in view of Brown teaches “The method of claim 17, further comprising (Walczyk teaches the edge controller 602 uses reinforcement learning to control actuators based on information from sensors i.e. it makes automated adjustments to temperature and operational parameters in Walczyk [0083] "The edge controller 602 is configured to (e.g., programed to) control the actuator(s) 604 based on an indoor air temperature setpoint, state information from the sensor(s)/meter(s) 606, and disturbance information from the weather service 610 and/or occupancy system 608 using an artificial intelligence algorithm, for example using a reinforcement learning model. As shown, the edge controller 602 can determine one or more internal control values for a unit of building equipment to be provided to actuator(s) 604. For example, the edge controller 602 can determine a desired change in mixed air temperature for an air handling unit 302 (e.g., an amount by which the mixed air temperature should be changed from a current time step to a subsequent time step). The edge controller 602 and/or the AHU controller 330 can use the desired change in the mixed air temperature for controlling actuators 324-328 which operate dampers 316-320 of the air handling unit 302 to affect the mixed air temperature.").
Neither Walczyk or Brown appear to explicitly teach “The method of claim 17, further comprising validating the optimal actions over the digital twin,” However, Cicirelli does teach this claim limitation (Cicirelli teaches a deep reinforcement learning controller that may use a digital twin to predict a result of an action by simulation in Cicirelli [Page 2, Section III, second paragraph] "Figure 2 shows how the approach is specialized for the management of a cognitive building. The approach is founded on a controller that takes a set of environmental and energy parameters as inputs and, by using a DRL-based controller, operates on the environment through a set of actuators, thus creating a cycle that learns from experience how to reduce energy consumption and improve people comfort. The control cycle interacts both with a real environment and with its digital counterpart, in accordance with the digital twin paradigm. The presence of the digital twin is helpful both in the learning phase, when the neural network needs to be trained with a large number of experiments, and in the operating phase, since it can be useful to predict the result of a given action by simulation. In particular, Figure 2 shows the two components of the digital counterpart, i.e., the simulator that models the evolution of the environment and a robot that simulates the human interaction." and in Cicirelli [Page 3, second paragraph] "The DRL-based controller uses a reward function, which weighs the human and the environmental rewards and computes the overall reward. The controller uses a Deep Q-network [6] to choose the action that is expected to maximize the value of the overall reward. The chosen action consists of the settings of the HVAC system, of the doors, and of the windows. The action can be operated either on the physical environment or on its digital twin, thus closing the cycle.").
Walczyk, Brown, and Cicirelli are analogous art because they are from the same field of endeavor of HVAC. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, having teachings of Walczyk, Brown, and Cicirelli before him/her, to modify the teachings of an Equipment edge controller with reinforcement learning of Walczyk modified to include the digital twin simulation of Brown to include the reinforcement learning controller that uses a digital twin to predict a result of an action by simulation of Cicirelli because adding the thermal comfort management leveraging deep reinforcement learning of Cicirelli would allow for prediction of a result during the operating phase as described in Cicirelli [Page 2, Section III, second paragraph] "Figure 2 shows how the approach is specialized for the management of a cognitive building. The approach is founded on a controller that takes a set of environmental and energy parameters as inputs and, by using a DRL-based controller, operates on the environment through a set of actuators, thus creating a cycle that learns from experience how to reduce energy consumption and improve people comfort. The control cycle interacts both with a real environment and with its digital counterpart, in accordance with the digital twin paradigm. The presence of the digital twin is helpful both in the learning phase, when the neural network needs to be trained with a large number of experiments, and in the operating phase, since it can be useful to predict the result of a given action by simulation. In particular, Figure 2 shows the two components of the digital counterpart, i.e., the simulator that models the evolution of the environment and a robot that simulates the human interaction."
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Desage et al. (US20250180242A1) teaches a control device that simulates performance of a BMS in order to evaluate an optimal path closest to a desired outcome in Desage [0206].
Russom et al. (US20240361028A1) teaches using occupancy data to adjust setpoint temperature in Russom [0038], using a reinforcement learning model in Russom [0033], and using a digital twin to train the model in Russom [0051-0053].
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Zachary A Cain whose telephone number is (571)272-4503. The examiner can normally be reached Mon-Fri 7:00-3:30 CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kenneth M Lo can be reached at (571) 272-9774. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Z.A.C./ Examiner, Art Unit 2116
/KENNETH M LO/ Supervisory Patent Examiner, Art Unit 2116
Prosecution Insights