18. Wrappers example

We will see on this notebook what are the wrappers defined by sinergym and how to use them. Currently, we have developed a normalization wrapper, multi-observation wrapper and Logger wrapper

[1]:

import gym
import numpy as np
import sinergym
from sinergym.utils.wrappers import (LoggerWrapper, MultiObsWrapper,
                                     NormalizeObservation)
from sinergym.utils.constants import RANGES_5ZONE

/usr/local/lib/python3.10/dist-packages/gym/spaces/box.py:73: UserWarning: WARN: Box bound precision lowered by casting to float32
  logger.warn(

18.1. Normalization Wrapper

[2]:

#Original env
env=gym.make('Eplus-demo-v1')
obs=env.reset()
print('BEFORE NORMILIZATION: ',obs)

#Normalize env
env = NormalizeObservation(env,ranges=RANGES_5ZONE)
obs=env.reset()
print('AFTER NORMILIZATION: ',obs)
env.close()

[2022-10-07 09:15:34,778] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:34,780] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:34,783] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:34,785] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:34,786] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2022-10-07 09:15:34,789] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:34,805] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res16/Eplus-env-sub_run1
BEFORE NORMILIZATION:  [1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04]
[2022-10-07 09:15:40,486] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2022-10-07 09:15:40,488] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:40,505] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res16/Eplus-env-sub_run2
AFTER NORMILIZATION:  [0.98466104 0.         0.         0.         0.33834645 0.9484536
 0.17748918 0.8111888  0.         0.         0.800001   0.44444445
 0.39061695 0.         0.43193626 0.         0.         0.
 0.         0.56101173]
[2022-10-07 09:15:45,445] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.

18.2. Logger Wrapper

[3]:

env=gym.make('Eplus-demo-v1')
env=LoggerWrapper(env)

[2022-10-07 09:15:46,643] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:46,643] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:46,646] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:46,646] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:46,649] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:46,649] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:46,653] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:46,653] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:46,657] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2022-10-07 09:15:46,657] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...

Now, you can see in Sinergym output folder that you will have available progress.csv file and monitor.csv files in each episode.

18.3. Multi Observation Wrapper

[4]:

#Original environment
env=gym.make('Eplus-demo-v1')
obs=env.reset()
print('BEFORE MULTI OBSERVATION: ',obs)

#Multi Observation environment
env=MultiObsWrapper(env, n=5, flatten=True)
obs=env.reset()
print('AFTER MULTI OBSERVATION: ',obs)
env.close()

[2022-10-07 09:15:48,201] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:48,201] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:48,201] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:48,203] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:48,203] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:48,203] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:48,205] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:48,205] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:48,205] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:48,208] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:48,208] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:48,208] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:48,210] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2022-10-07 09:15:48,210] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2022-10-07 09:15:48,210] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2022-10-07 09:15:48,213] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:48,213] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:48,213] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:48,225] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res18/Eplus-env-sub_run1
[2022-10-07 09:15:48,225] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res18/Eplus-env-sub_run1
[2022-10-07 09:15:48,225] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res18/Eplus-env-sub_run1
BEFORE MULTI OBSERVATION:  [1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04]
[2022-10-07 09:15:53,099] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2022-10-07 09:15:53,099] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2022-10-07 09:15:53,099] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2022-10-07 09:15:53,100] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:53,100] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:53,100] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:53,108] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res18/Eplus-env-sub_run2
[2022-10-07 09:15:53,108] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res18/Eplus-env-sub_run2
[2022-10-07 09:15:53,108] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res18/Eplus-env-sub_run2
AFTER MULTI OBSERVATION:  [1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04
 1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04
 1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04
 1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04
 1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04]
[2022-10-07 09:15:58,144] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2022-10-07 09:15:58,144] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2022-10-07 09:15:58,144] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.

18.4. All Wrappers in one

You can also combine wrappers in order to use all their functionality at the same time.

[5]:

env=gym.make('Eplus-demo-v1')
#Normalization
env=NormalizeObservation(env,ranges=RANGES_5ZONE)
#Logger
env=LoggerWrapper(env)
#MultiObs
env=MultiObsWrapper(env,n=5,flatten=True)

[2022-10-07 09:15:59,535] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:59,535] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:59,535] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:59,535] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-10-07 09:15:59,537] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:59,537] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:59,537] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:59,537] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-10-07 09:15:59,540] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:59,540] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:59,540] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:59,540] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-10-07 09:15:59,543] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:59,543] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:59,543] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:59,543] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-10-07 09:15:59,545] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2022-10-07 09:15:59,545] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2022-10-07 09:15:59,545] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2022-10-07 09:15:59,545] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...

Warning

The order of wrappers if you are going to use several at the same time is really important. The correct order is Normalization - Logger - MultiObs and subsets (for example, Normalization - Multiobs is valid).

Note

For more information about Sinergym Logger, visit Logger documentation and Wrapper documentation

Now we just simply use the environment with the wrappers, for example:

[6]:

for i in range(1):
    obs = env.reset()
    rewards = []
    done = False
    current_month = 0
    while not done:
        a = env.action_space.sample()
        obs, reward, done, info = env.step(a)
        rewards.append(reward)
        if info['month'] != current_month:  # display results every month
            current_month = info['month']
            print('Reward: ', sum(rewards), info)
    print('Episode ', i, 'Mean reward: ', np.mean(
        rewards), 'Cumulative reward: ', sum(rewards))
env.close()

[2022-10-07 09:15:59,744] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:59,744] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:59,744] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:59,744] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-10-07 09:15:59,754] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res19/Eplus-env-sub_run1
[2022-10-07 09:15:59,754] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res19/Eplus-env-sub_run1
[2022-10-07 09:15:59,754] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res19/Eplus-env-sub_run1
[2022-10-07 09:15:59,754] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res19/Eplus-env-sub_run1
Reward:  -0.9027267231826451 {'timestep': 1, 'time_elapsed': 900, 'year': 1991, 'month': 1, 'day': 1, 'hour': 0, 'total_power': 18054.5344636529, 'total_power_no_units': -1.8054534463652903, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.99998833869494], 'out_temperature': 1.8, 'action_': [21, 21]}
Reward:  -2097.069513380396 {'timestep': 2976, 'time_elapsed': 2678400, 'year': 1991, 'month': 2, 'day': 1, 'hour': 0, 'total_power': 4420.417448015461, 'total_power_no_units': -0.44204174480154607, 'comfort_penalty': -1.5592545746684792, 'abs_comfort': 1.5592545746684792, 'temperatures': [18.44074542533152], 'out_temperature': -7.0, 'action_': [17, 28]}
Reward:  -4115.586715117292 {'timestep': 5664, 'time_elapsed': 5097600, 'year': 1991, 'month': 3, 'day': 1, 'hour': 0, 'total_power': 15106.04831627059, 'total_power_no_units': -1.510604831627059, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.73615474040233], 'out_temperature': 8.1, 'action_': [21, 21]}
Reward:  -5458.5534876380125 {'timestep': 8640, 'time_elapsed': 7776000, 'year': 1991, 'month': 4, 'day': 1, 'hour': 0, 'total_power': 7163.35210582586, 'total_power_no_units': -0.7163352105825861, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.99485496779759], 'out_temperature': 7.7, 'action_': [21, 21]}
Reward:  -6378.342330065786 {'timestep': 11520, 'time_elapsed': 10368000, 'year': 1991, 'month': 5, 'day': 1, 'hour': 0, 'total_power': 152.4868953414246, 'total_power_no_units': -0.01524868953414246, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.75661867832585], 'out_temperature': 13.0, 'action_': [16, 29]}
Reward:  -7230.79295371363 {'timestep': 14496, 'time_elapsed': 13046400, 'year': 1991, 'month': 6, 'day': 1, 'hour': 0, 'total_power': 11573.19138929936, 'total_power_no_units': -1.157319138929936, 'comfort_penalty': -1.1447156088359804, 'abs_comfort': 1.1447156088359804, 'temperatures': [21.85528439116402], 'out_temperature': 18.4, 'action_': [22, 23]}
Reward:  -10136.551748683549 {'timestep': 17376, 'time_elapsed': 15638400, 'year': 1991, 'month': 7, 'day': 1, 'hour': 0, 'total_power': 175.779675822106, 'total_power_no_units': -0.0175779675822106, 'comfort_penalty': -2.2201257903301403, 'abs_comfort': 2.2201257903301403, 'temperatures': [20.77987420966986], 'out_temperature': 17.7, 'action_': [18, 27]}
Reward:  -13417.322446742612 {'timestep': 20352, 'time_elapsed': 18316800, 'year': 1991, 'month': 8, 'day': 1, 'hour': 0, 'total_power': 20438.77860937315, 'total_power_no_units': -2.043877860937315, 'comfort_penalty': -1.2761863833358795, 'abs_comfort': 1.2761863833358795, 'temperatures': [21.72381361666412], 'out_temperature': 20.6, 'action_': [22, 22]}
Reward:  -16653.395519627065 {'timestep': 23328, 'time_elapsed': 20995200, 'year': 1991, 'month': 9, 'day': 1, 'hour': 0, 'total_power': 296.4058515355591, 'total_power_no_units': -0.029640585153555915, 'comfort_penalty': -2.0341594895709108, 'abs_comfort': 2.0341594895709108, 'temperatures': [20.96584051042909], 'out_temperature': 18.8, 'action_': [17, 28]}
Reward:  -19473.56028227908 {'timestep': 26208, 'time_elapsed': 23587200, 'year': 1991, 'month': 10, 'day': 1, 'hour': 0, 'total_power': 152.4868953414246, 'total_power_no_units': -0.01524868953414246, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.33729078022685], 'out_temperature': 13.3, 'action_': [20, 25]}
Reward:  -20476.057376697874 {'timestep': 29184, 'time_elapsed': 26265600, 'year': 1991, 'month': 11, 'day': 1, 'hour': 0, 'total_power': 12183.50538957212, 'total_power_no_units': -1.218350538957212, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [21.85073350665203], 'out_temperature': 13.0, 'action_': [22, 22]}
Reward:  -21654.559762066197 {'timestep': 32064, 'time_elapsed': 28857600, 'year': 1991, 'month': 12, 'day': 1, 'hour': 0, 'total_power': 152.6211799169361, 'total_power_no_units': -0.01526211799169361, 'comfort_penalty': -1.027243301837089, 'abs_comfort': 1.027243301837089, 'temperatures': [18.97275669816291], 'out_temperature': 5.1, 'action_': [17, 28]}
Reward:  -23670.370352311853 {'timestep': 35040, 'time_elapsed': 31536000, 'year': 1992, 'month': 1, 'day': 1, 'hour': 0, 'total_power': 23172.08056052774, 'total_power_no_units': -2.317208056052774, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [21.13047508039367], 'out_temperature': -12.0, 'action_': [22, 22]}
Episode  0 Mean reward:  -0.6755242680454563 Cumulative reward:  -23670.370352311853
[2022-10-07 09:16:15,289] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2022-10-07 09:16:15,289] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2022-10-07 09:16:15,289] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2022-10-07 09:16:15,289] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.