Wrappers example
We will see on this notebook what are the wrappers defined by sinergym and how to use them. Currently, we have developed a normalization wrapper, multi-observation wrapper and Logger wrapper
[1]:
import gym
import numpy as np
import sinergym
from sinergym.utils.wrappers import (LoggerWrapper, MultiObsWrapper,
NormalizeObservation)
from sinergym.utils.constants import RANGES_5ZONE
/usr/local/lib/python3.10/dist-packages/gym/spaces/box.py:73: UserWarning: WARN: Box bound precision lowered by casting to float32
logger.warn(
Normalization Wrapper
[2]:
#Original env
env=gym.make('Eplus-demo-v1')
obs=env.reset()
print('BEFORE NORMILIZATION: ',obs)
#Normalize env
env = NormalizeObservation(env,ranges=RANGES_5ZONE)
obs=env.reset()
print('AFTER NORMILIZATION: ',obs)
[2022-08-24 09:25:01,984] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:01,986] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:01,990] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:01,993] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-08-24 09:25:01,997] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:02,013] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res7/Eplus-env-sub_run1
BEFORE NORMILIZATION: [1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9522228e+01
0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 8.0098755e+03]
[2022-08-24 09:25:07,442] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2022-08-24 09:25:07,443] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:07,451] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res7/Eplus-env-sub_run2
AFTER NORMILIZATION: [0.98466104 0. 0. 0. 0.33834645 0.9484536
0.17748918 0.8111888 0. 0. 0.800001 0.44444445
0.39061695 0. 0.42973474 0. 0. 0.
0. 0.24169508]
Logger Wrapper
[3]:
env=gym.make('Eplus-demo-v1')
env=LoggerWrapper(env)
[2022-08-24 09:25:08,780] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:08,780] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:08,784] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:08,784] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:08,788] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:08,788] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:08,792] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-08-24 09:25:08,792] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
Now, you can see in Sinergym output folder that you will have available progress.csv
file and monitor.csv
files in each episode.
Multi Observation Wrapper
[4]:
#Original environment
env=gym.make('Eplus-demo-v1')
obs=env.reset()
print('BEFORE MULTI OBSERVATION: ',obs)
#Multi Observation environment
env=MultiObsWrapper(env, n=5, flatten=True)
obs=env.reset()
print('AFTER MULTI OBSERVATION: ',obs)
[2022-08-24 09:25:10,424] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:10,424] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:10,424] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:10,427] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:10,427] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:10,427] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:10,430] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:10,430] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:10,430] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:10,433] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-08-24 09:25:10,433] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-08-24 09:25:10,433] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-08-24 09:25:10,436] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:10,436] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:10,436] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:10,452] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res9/Eplus-env-sub_run1
[2022-08-24 09:25:10,452] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res9/Eplus-env-sub_run1
[2022-08-24 09:25:10,452] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res9/Eplus-env-sub_run1
BEFORE MULTI OBSERVATION: [1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9522228e+01
0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 8.0098755e+03]
[2022-08-24 09:25:15,441] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2022-08-24 09:25:15,441] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2022-08-24 09:25:15,441] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2022-08-24 09:25:15,442] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:15,442] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:15,442] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:15,451] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res9/Eplus-env-sub_run2
[2022-08-24 09:25:15,451] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res9/Eplus-env-sub_run2
[2022-08-24 09:25:15,451] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res9/Eplus-env-sub_run2
AFTER MULTI OBSERVATION: [1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9522228e+01
0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 8.0098755e+03
1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9522228e+01
0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 8.0098755e+03
1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9522228e+01
0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 8.0098755e+03
1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9522228e+01
0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 8.0098755e+03
1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9522228e+01
0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 8.0098755e+03]
All Wrappers in one
You can also combine wrappers in order to use all their functionality at the same time.
[5]:
env=gym.make('Eplus-demo-v1')
#Normalization
env=NormalizeObservation(env,ranges=RANGES_5ZONE)
#Logger
env=LoggerWrapper(env)
#MultiObs
env=MultiObsWrapper(env,n=5,flatten=True)
[2022-08-24 09:25:16,978] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:16,978] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:16,978] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:16,978] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2022-08-24 09:25:16,981] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:16,981] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:16,981] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:16,981] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2022-08-24 09:25:16,985] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:16,985] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:16,985] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:16,985] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2022-08-24 09:25:16,990] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-08-24 09:25:16,990] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-08-24 09:25:16,990] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2022-08-24 09:25:16,990] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
Warning
The order of wrappers if you are going to use several at the same time is really important. The correct order is Normalization - Logger - MultiObs and subsets (for example, Normalization - Multiobs is valid).
Note
For more information about Sinergym Logger, visit Logger documentation and Wrapper documentation
Now we just simply use the environment with the wrappers, for example:
[6]:
for i in range(1):
obs = env.reset()
rewards = []
done = False
current_month = 0
while not done:
a = env.action_space.sample()
obs, reward, done, info = env.step(a)
rewards.append(reward)
if info['month'] != current_month: # display results every month
current_month = info['month']
print('Reward: ', sum(rewards), info)
print('Episode ', i, 'Mean reward: ', np.mean(
rewards), 'Cumulative reward: ', sum(rewards))
env.close()
[2022-08-24 09:25:17,309] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:17,309] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:17,309] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:17,309] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2022-08-24 09:25:17,326] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res10/Eplus-env-sub_run1
[2022-08-24 09:25:17,326] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res10/Eplus-env-sub_run1
[2022-08-24 09:25:17,326] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res10/Eplus-env-sub_run1
[2022-08-24 09:25:17,326] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res10/Eplus-env-sub_run1
Reward: -0.6725497890372522 {'timestep': 1, 'time_elapsed': 900, 'year': 1991, 'month': 1, 'day': 1, 'hour': 0, 'total_power': 3133.716390651042, 'total_power_no_units': -0.3133716390651042, 'comfort_penalty': -1.0317279390094, 'abs_comfort': 1.0317279390094, 'temperatures': [18.9682720609906], 'out_temperature': 1.8, 'action_': [18, 27]}
Reward: -1831.9846868749223 {'timestep': 2976, 'time_elapsed': 2678400, 'year': 1991, 'month': 2, 'day': 1, 'hour': 0, 'total_power': 9625.868884432568, 'total_power_no_units': -0.9625868884432568, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.99962719644226], 'out_temperature': -7.0, 'action_': [21, 21]}
Reward: -3572.1163979019448 {'timestep': 5664, 'time_elapsed': 5097600, 'year': 1991, 'month': 3, 'day': 1, 'hour': 0, 'total_power': 3381.835883111865, 'total_power_no_units': -0.3381835883111865, 'comfort_penalty': -0.04845287512533858, 'abs_comfort': 0.04845287512533858, 'temperatures': [19.95154712487466], 'out_temperature': 8.1, 'action_': [20, 25]}
Reward: -4744.85182426822 {'timestep': 8640, 'time_elapsed': 7776000, 'year': 1991, 'month': 4, 'day': 1, 'hour': 0, 'total_power': 1059.217058241587, 'total_power_no_units': -0.1059217058241587, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [21.02705837058395], 'out_temperature': 7.7, 'action_': [21, 21]}
Reward: -5541.648792015279 {'timestep': 11520, 'time_elapsed': 10368000, 'year': 1991, 'month': 5, 'day': 1, 'hour': 0, 'total_power': 186.5934720667916, 'total_power_no_units': -0.018659347206679163, 'comfort_penalty': -0.6904987523763602, 'abs_comfort': 0.6904987523763602, 'temperatures': [19.30950124762364], 'out_temperature': 13.0, 'action_': [19, 26]}
Reward: -6265.521158571141 {'timestep': 14496, 'time_elapsed': 13046400, 'year': 1991, 'month': 6, 'day': 1, 'hour': 0, 'total_power': 2706.140914592288, 'total_power_no_units': -0.2706140914592288, 'comfort_penalty': -1.0741537654487203, 'abs_comfort': 1.0741537654487203, 'temperatures': [21.92584623455128], 'out_temperature': 18.4, 'action_': [22, 23]}
Reward: -9267.105328861617 {'timestep': 17376, 'time_elapsed': 15638400, 'year': 1991, 'month': 7, 'day': 1, 'hour': 0, 'total_power': 215.8190761707732, 'total_power_no_units': -0.02158190761707732, 'comfort_penalty': -1.709210011679879, 'abs_comfort': 1.709210011679879, 'temperatures': [21.29078998832012], 'out_temperature': 17.7, 'action_': [16, 29]}
Reward: -12581.239530466879 {'timestep': 20352, 'time_elapsed': 18316800, 'year': 1991, 'month': 8, 'day': 1, 'hour': 0, 'total_power': 4303.552629721591, 'total_power_no_units': -0.43035526297215915, 'comfort_penalty': -3.137288347910051, 'abs_comfort': 3.137288347910051, 'temperatures': [19.86271165208995], 'out_temperature': 20.6, 'action_': [17, 28]}
Reward: -15889.752598388217 {'timestep': 23328, 'time_elapsed': 20995200, 'year': 1991, 'month': 9, 'day': 1, 'hour': 0, 'total_power': 2547.043642249929, 'total_power_no_units': -0.2547043642249929, 'comfort_penalty': -1.0062198564813798, 'abs_comfort': 1.0062198564813798, 'temperatures': [21.99378014351862], 'out_temperature': 18.8, 'action_': [22, 23]}
Reward: -18686.835610714406 {'timestep': 26208, 'time_elapsed': 23587200, 'year': 1991, 'month': 10, 'day': 1, 'hour': 0, 'total_power': 1683.24724933671, 'total_power_no_units': -0.168324724933671, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.98601762250864], 'out_temperature': 13.3, 'action_': [21, 24]}
Reward: -19616.273201273543 {'timestep': 29184, 'time_elapsed': 26265600, 'year': 1991, 'month': 11, 'day': 1, 'hour': 0, 'total_power': 1971.816419607761, 'total_power_no_units': -0.19718164196077612, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.98439137803774], 'out_temperature': 13.0, 'action_': [21, 24]}
Reward: -20664.183622138895 {'timestep': 32064, 'time_elapsed': 28857600, 'year': 1991, 'month': 12, 'day': 1, 'hour': 0, 'total_power': 4794.553408394407, 'total_power_no_units': -0.4794553408394407, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.80737243763421], 'out_temperature': 5.1, 'action_': [21, 24]}
Reward: -22425.618844122902 {'timestep': 35040, 'time_elapsed': 31536000, 'year': 1992, 'month': 1, 'day': 1, 'hour': 0, 'total_power': 12081.03798343912, 'total_power_no_units': -1.208103798343912, 'comfort_penalty': -0.0, 'abs_comfort': 0.0, 'temperatures': [20.61093146586074], 'out_temperature': -12.0, 'action_': [22, 22]}
Episode 0 Mean reward: -0.6400005377888734 Cumulative reward: -22425.618844122902
[2022-08-24 09:25:33,119] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2022-08-24 09:25:33,119] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2022-08-24 09:25:33,119] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2022-08-24 09:25:33,119] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.