20. Wrappers example

We will see on this notebook what are the wrappers defined by Sinergym and how to use them. You can also implement your own wrappers inheriting from gym.Wrapper or some of its variants.

[1]:

import gymnasium as gym
import numpy as np
import sinergym
from sinergym.utils.wrappers import *

20.1. Multi-Objective Wrapper

MO-Gymnasium is an open source Python library for developing and comparing multi-objective reinforcement learning algorithms. These environments return a reward vector instead of a scalar value; one for each objective.

In order to be more general as possible, it could be interesting that Sinergym would give that reward vector too. In this way, Sinergym would have compatibility with both; multi-objective algorithms and algorithms that work with a traditional reward value.

We can transform reward returned in a vector using the next wrapper:

[2]:

env=gym.make('Eplus-5zone-hot-discrete-v1')
env=MultiObjectiveReward(env,reward_terms=['energy_term','comfort_term'])

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER MultiObjectiveReward] (INFO) : wrapper initialized.

You have to ensure that reward_terms are available in info dict returned in step method of the environment. Otherwise, we will encounter an execution error. By default, Sinergym environments return in info dict all reward terms specified in reward class used, so if the objective exists in reward term you shouldn’t have any problem.

[3]:

env.reset()
action = env.action_space.sample()
obs, reward, terminated, truncated, info = env.step(action)
env.close()
print(reward)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1/output]

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]
[-0.05542625239599594, -0.857887333002763]

20.2. Previous Observations Wrappers

This Wrapper will add observation values from previous timestep to current environment observation. It is possible to select the variables you want to track its previous observation values. Observation space will be updated with the new dimension.

[4]:

env=gym.make('Eplus-5zone-hot-discrete-v1')
env = PreviousObservationWrapper(env, previous_variables=[
        'htg_setpoint',
        'clg_setpoint',
        'air_temperature'])

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER PreviousObservationWrapper] (INFO) : Wrapper initialized.

We can see the observation values has been updated:

[5]:

env.reset()
obs,_,_,_,_=env.step(env.action_space.sample())
obs_dict=dict(zip(env.get_wrapper_attr('observation_variables'),obs))
env.close()
print('NEW OBSERVATION: ',obs_dict)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]
NEW OBSERVATION:  {'month': 1.0, 'day_of_month': 1.0, 'hour': 0.0, 'outdoor_temperature': 4.8, 'outdoor_humidity': 61.0, 'wind_speed': 4.65, 'wind_direction': 160.0, 'diffuse_solar_radiation': 0.0, 'direct_solar_radiation': 0.0, 'htg_setpoint': 12.8, 'clg_setpoint': 40.0, 'air_temperature': 18.284225, 'air_humidity': 26.9228, 'people_occupant': 0.0, 'co2_emission': 0.0, 'HVAC_electricity_demand_rate': 1108.525, 'total_electricity_HVAC': 997672.56, 'htg_setpoint_previous': 12.8, 'clg_setpoint_previous': 40.0, 'air_temperature_previous': 18.384174}

20.3. Datetime Wrapper

This wrapper will substitute day value by is_weekend flag, and hour and month by sin and cos values. Observation space is also updated automatically.

[6]:

env=gym.make('Eplus-5zone-hot-discrete-v1')
env = DatetimeWrapper(env)

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER DatetimeWrapper] (INFO) : Wrapper initialized.

Concretely, this wrapper deletes the observation variables month, day and hour, and adds month_sin, month_cos, is_weekend, hour_sin and hour_cos instead:

[7]:

env.reset()
obs,_,_,_,_=env.step(env.action_space.sample())
obs_dict=dict(zip(env.get_wrapper_attr('observation_variables'),obs))
env.close()
print('NEW OBSERVATION: ',obs_dict)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]
NEW OBSERVATION:  {'month_cos': 1.0, 'month_sin': 0.0, 'is_weekend': 0.0, 'hour_cos': 1.0, 'hour_sin': 0.0, 'outdoor_temperature': 4.800000190734863, 'outdoor_humidity': 61.0, 'wind_speed': 4.650000095367432, 'wind_direction': 160.0, 'diffuse_solar_radiation': 0.0, 'direct_solar_radiation': 0.0, 'htg_setpoint': 12.800000190734863, 'clg_setpoint': 40.0, 'air_temperature': 18.284225463867188, 'air_humidity': 26.922800064086914, 'people_occupant': 0.0, 'co2_emission': 0.0, 'HVAC_electricity_demand_rate': 1108.5250244140625, 'total_electricity_HVAC': 997672.5625}

20.4. Environment Action Normalization Wrapper

This is an example about how to normalize a previous continuos environment action space. If we don’t define the range values, it will be constructed with the range [-1,1] by default:

[8]:

# We will create a continuous environment
env=gym.make('Eplus-5zone-hot-continuous-v1')
print('ORIGINAL ACTION SPACE: ',env.get_wrapper_attr('action_space'))
# NORMALIZATION
# Apply the normalize action wrapper
env=NormalizeAction(env,normalize_range=(-1.0,1.0))
print('WRAPPED ACTION SPACE: ',env.get_wrapper_attr('action_space'))
env.reset()
for i in range(5):
    action=env.action_space.sample()
    print('Normalized action: ',action)
    _,_,_,_,info=env.step(action)
    print('Action done in simulator: ', info['action'])
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
ORIGINAL ACTION SPACE:  Box([15.  22.5], [22.5 30. ], (2,), float32)
[WRAPPER NormalizeAction] (INFO) : New normalized action Space: Box(-1.0, 1.0, (2,), float32)
[WRAPPER NormalizeAction] (INFO) : Wrapper initialized
WRAPPED ACTION SPACE:  Box(-1.0, 1.0, (2,), float32)
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Normalized action:  [ 0.8381089  -0.15050839]
Action done in simulator:  [21.892908, 25.685593]
Normalized action:  [-0.20605884  0.00121972]
Action done in simulator:  [17.97728, 26.254574]
Normalized action:  [ 0.20422778 -0.9112268 ]
Action done in simulator:  [19.515854, 22.8329]
Normalized action:  [-0.6123266  -0.04220473]
Action done in simulator:  [16.453775, 26.091732]
Normalized action:  [ 0.46484488 -0.32572523]
Action done in simulator:  [20.493168, 25.02853]
Progress: |*****----------------------------------------------------------------------------------------------| 5%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

20.5. Environment Discretization Wrapper

This is an example about how to discretize a previous continuos environment. We will require the new discrete action space and an action mapping function whose output matches with the original unwrapped environment action space:

[9]:

# We will create a continuous environment
env=gym.make('Eplus-5zone-hot-continuous-v1')
print('ORIGINAL ACTION SPACE: ',env.get_wrapper_attr('action_space'))
print('IS DISCRETE?: ',env.get_wrapper_attr('is_discrete'))
# DISCRETIZATION
# Defining new discrete space and action mapping function
new_discrete_space = gym.spaces.Discrete(10) # Action values [0,9]
def action_mapping_function(action):
    mapping = {
        0: [15, 30], # These lists matches with original action space
        1: [16, 29],
        2: [17, 28],
        3: [18, 27],
        4: [19, 26],
        5: [20, 25],
        6: [21, 24],
        7: [22, 23],
        8: [22, 22.5],
        9: [21, 22.5]
    }

    return mapping[action]
# Apply the discretize wrapper
env=DiscretizeEnv(env,discrete_space=new_discrete_space,action_mapping=action_mapping_function)
print('WRAPPED ACTION SPACE: ',env.get_wrapper_attr('action_space'))
print('IS DISCRETE?: ',env.get_wrapper_attr('is_discrete'))
env.reset()
for i in range(5):
    action=env.action_space.sample()
    print('ACTION DISCRETE: ',action)
    _,_,_,_,info=env.step(action)
    print('Action done in simulator: ', info['action'])
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
ORIGINAL ACTION SPACE:  Box([15.  22.5], [22.5 30. ], (2,), float32)
IS DISCRETE?:  False
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
WRAPPED ACTION SPACE:  Discrete(10)
IS DISCRETE?:  True
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
ACTION DISCRETE:  2
Action done in simulator:  [17, 28]
ACTION DISCRETE:  3
Action done in simulator:  [18, 27]
ACTION DISCRETE:  8
Action done in simulator:  [22, 22.5]
ACTION DISCRETE:  5
Action done in simulator:  [20, 25]
ACTION DISCRETE:  6
Action done in simulator:  [21, 24]
Progress: |*****----------------------------------------------------------------------------------------------| 5%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

As we can see in output, the input is an int, but a list with original action space is done in the simulator.

20.6. Discrete Incremental Wrapper

A wrapper for an incremental setpoint action space environment. This wrapper will update an environment, converting it in a discrete environment with an action mapping function and action space depending on the step and delta specified. The action will be sum with current setpoint values instead of overwrite the latest action. Then, the action is the current setpoint values with the increase instead of the discrete value action whose purpose is to define the increment/decrement itself.

[10]:

env=gym.make('Eplus-5zone-hot-continuous-v1')
print('ORIGINAL ACTION SPACE: ',env.get_wrapper_attr('action_space'))
env = DiscreteIncrementalWrapper(
        env,initial_values=[21.0,25.0], delta_temp=2, step_temp=0.5)
print('WRAPPED ACTION SPACE: ',env.get_wrapper_attr('action_space'))
print('WRAPPED ACTION MAPPING: ',env.get_wrapper_attr('action_mapping'))

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
ORIGINAL ACTION SPACE:  Box([15.  22.5], [22.5 30. ], (2,), float32)
[WRAPPER DiscreteIncrementalWrapper] (INFO) : New incremental action mapping: 17
[WRAPPER DiscreteIncrementalWrapper] (INFO) : {0: [0.0, 0.0], 1: [0.5, 0.0], 2: [1.0, 0.0], 3: [1.5, 0.0], 4: [2.0, 0.0], 5: [-0.5, 0.0], 6: [-1.0, 0.0], 7: [-1.5, 0.0], 8: [-2.0, 0.0], 9: [0.0, 0.5], 10: [0.0, 1.0], 11: [0.0, 1.5], 12: [0.0, 2.0], 13: [0.0, -0.5], 14: [0.0, -1.0], 15: [0.0, -1.5], 16: [0.0, -2.0]}
[WRAPPER DiscreteIncrementalWrapper] (INFO) : Wrapper initialized
WRAPPED ACTION SPACE:  Discrete(17)
WRAPPED ACTION MAPPING:  <bound method DiscreteIncrementalWrapper.action_mapping of <DiscreteIncrementalWrapper<OrderEnforcing<PassiveEnvChecker<EplusEnv<Eplus-5zone-hot-continuous-v1>>>>>>

Maximum and minimum values to create the action mapping is read from environment action space, so that the setpoint increments and decrements do not go outside the agreed limits. The delta and step values are used to determine how the discrete space of these increments and decrements will be constructed. An example of how it works is shown below:

[11]:

env.reset()
print('CURRENT SETPOINTS VALUES: ', env.get_wrapper_attr('current_setpoints'))
for i in range(5):
    action=env.action_space.sample()
    _,_,_,_,info=env.step(action)
    print('Action number ',i,': ',env.get_wrapper_attr('action_mapping')(action))
    print('Setpoints update: ', info['action'])
env.close()

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
CURRENT SETPOINTS VALUES:  [21.0, 25.0]
Action number  0 :  [-2.0, 0.0]
Setpoints update:  [19.0, 25.0]
Action number  1 :  [0.0, 0.5]
Setpoints update:  [19.0, 25.5]
Action number  2 :  [0.0, -0.5]
Setpoints update:  [19.0, 25.0]
Action number  3 :  [0.0, 0.0]
Setpoints update:  [19.0, 25.0]
Action number  4 :  [0.0, -0.5]
Setpoints update:  [19.0, 24.5]
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

20.7. Normalization Wrapper

This wrapper is used to transform observation received from simulator in values between -1 and 1. It is based in the dynamic normalization wrapper of Gymnasium. At the beginning, it is not precise and the values may be out of range usually, so use this wrapper carefully.

[12]:

#Original env
env=gym.make('Eplus-5zone-hot-discrete-v1')
env = NormalizeObservation(
        env=env)

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER NormalizeObservation] (INFO) : wrapper initialized.

We can see in the next code how the variables specified have been normalized correctly:

[13]:

env.reset()
obs,_,_,_,_=env.step(env.action_space.sample())
obs_dict=dict(zip(env.get_wrapper_attr('observation_variables'),obs))
env.close()
print('OBSERVATION WITH NORMALIZATION: ',obs_dict)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]
OBSERVATION WITH NORMALIZATION:  {'month': 0.00499968750430059, 'day_of_month': 0.00499968750430059, 'hour': 0.0, 'outdoor_temperature': 0.9875907976185373, 'outdoor_humidity': -0.9745626607331893, 'wind_speed': 0.9973969815420386, 'wind_direction': 0.9908525387095581, 'diffuse_solar_radiation': 0.0, 'direct_solar_radiation': 0.0, 'htg_setpoint': 0.007049581561774207, 'clg_setpoint': 0.0070688585864955075, 'air_temperature': -0.35264364590376623, 'air_humidity': 0.05800834342804423, 'people_occupant': 0.0, 'co2_emission': 0.0, 'HVAC_electricity_demand_rate': 0.9802888899782684, 'total_electricity_HVAC': 0.9802889755036568}

20.8. Logger Wrapper

Wrapper for logging all interactions between agent and environment. Logger class can be selected in the constructor if other type of logging is required. For more information about Sinergym Logger visit Logger

[14]:

env=gym.make('Eplus-5zone-hot-discrete-v1')
env=LoggerWrapper(env)


env.reset()
terminated = False
current_month = 0
while not terminated:
    a = env.action_space.sample()
    _,_,terminated,_,_=env.step(a)
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER LoggerWrapper] (INFO) : Wrapper initialized.
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[WRAPPER LoggerWrapper] (INFO) : Creating monitor.csv for current episode (episode 1) if logger is active
Progress: |**-------------------------------------------------------------------------------------------------| 2%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[WRAPPER LoggerWrapper] (INFO) : End of episode, recording summary (progress.csv) if logger is active
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

Now, you can see in Sinergym output folder that you will have available progress.csv file and monitor.csv files in each episode.

20.9. Multi Observation Wrapper

This wrapper will stack observation received in a history queue (size can be customized).

[15]:

#Original environment
env=gym.make('Eplus-5zone-hot-discrete-v1')
obs, info=env.reset()
print('BEFORE MULTI OBSERVATION: ',obs)

#Multi Observation environment
env=MultiObsWrapper(env, n=5, flatten=True)
obs, info=env.reset()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
BEFORE MULTI OBSERVATION:  [1.0000000e+00 1.0000000e+00 0.0000000e+00 4.4000001e+00 6.5000000e+01
 3.8750000e+00 1.4500000e+02 0.0000000e+00 0.0000000e+00 1.2800000e+01
 4.0000000e+01 1.8384174e+01 2.6903370e+01 0.0000000e+00 0.0000000e+00
 1.0360292e+03 9.3242619e+05]
[WRAPPER MultiObsWrapper] (INFO) : Wrapper initialized.
Progress: |***************************************************************************************************| 99%
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 2]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 2 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[16]:

print('AFTER MULTI OBSERVATION: ',obs)
env.close()

AFTER MULTI OBSERVATION:  [1.0000000e+00 1.0000000e+00 0.0000000e+00 4.8000002e+00 6.1000000e+01
 4.6500001e+00 1.6000000e+02 0.0000000e+00 0.0000000e+00 1.2800000e+01
 4.0000000e+01 1.8284225e+01 2.6922800e+01 0.0000000e+00 0.0000000e+00
 1.1085250e+03 9.9767256e+05 1.0000000e+00 1.0000000e+00 0.0000000e+00
 4.8000002e+00 6.1000000e+01 4.6500001e+00 1.6000000e+02 0.0000000e+00
 0.0000000e+00 1.2800000e+01 4.0000000e+01 1.8284225e+01 2.6922800e+01
 0.0000000e+00 0.0000000e+00 1.1085250e+03 9.9767256e+05 1.0000000e+00
 1.0000000e+00 0.0000000e+00 4.8000002e+00 6.1000000e+01 4.6500001e+00
 1.6000000e+02 0.0000000e+00 0.0000000e+00 1.2800000e+01 4.0000000e+01
 1.8284225e+01 2.6922800e+01 0.0000000e+00 0.0000000e+00 1.1085250e+03
 9.9767256e+05 1.0000000e+00 1.0000000e+00 0.0000000e+00 4.8000002e+00
 6.1000000e+01 4.6500001e+00 1.6000000e+02 0.0000000e+00 0.0000000e+00
 1.2800000e+01 4.0000000e+01 1.8284225e+01 2.6922800e+01 0.0000000e+00
 0.0000000e+00 1.1085250e+03 9.9767256e+05 1.0000000e+00 1.0000000e+00
 0.0000000e+00 4.8000002e+00 6.1000000e+01 4.6500001e+00 1.6000000e+02
 0.0000000e+00 0.0000000e+00 1.2800000e+01 4.0000000e+01 1.8284225e+01
 2.6922800e+01 0.0000000e+00 0.0000000e+00 1.1085250e+03 9.9767256e+05]

[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

20.10. Nesting wrappers

All the wrappers listed in this notebook can be nested in order to combine functionalities added. However, the order in which these wrappers are nested is critical to their proper functionality. We strongly recommend that it be in the same order in which they appear here or in the documentation. In case you want to nest a subset of the total, it is not a problem as long as you continue respecting this order. We see and example:

[17]:

env = gym.make('Eplus-5zone-hot-continuous-v1')
env = MultiObjectiveReward(
    env=env,
    reward_terms=[
        'energy_term',
        'comfort_term'])
env = PreviousObservationWrapper(env, previous_variables=[
    'htg_setpoint',
    'clg_setpoint',
    'air_temperature'])
env = DatetimeWrapper(env)
env = DiscreteIncrementalWrapper(
    env,initial_values=[21.0,25.0], delta_temp=2, step_temp=0.5)
env = NormalizeObservation(
    env=env)
env = LoggerWrapper(env=env, flag=True)
env = MultiObsWrapper(env=env, n=5, flatten=True)

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER MultiObjectiveReward] (INFO) : wrapper initialized.
[WRAPPER PreviousObservationWrapper] (INFO) : Wrapper initialized.
[WRAPPER DatetimeWrapper] (INFO) : Wrapper initialized.
[WRAPPER DiscreteIncrementalWrapper] (INFO) : New incremental action mapping: 17
[WRAPPER DiscreteIncrementalWrapper] (INFO) : {0: [0.0, 0.0], 1: [0.5, 0.0], 2: [1.0, 0.0], 3: [1.5, 0.0], 4: [2.0, 0.0], 5: [-0.5, 0.0], 6: [-1.0, 0.0], 7: [-1.5, 0.0], 8: [-2.0, 0.0], 9: [0.0, 0.5], 10: [0.0, 1.0], 11: [0.0, 1.5], 12: [0.0, 2.0], 13: [0.0, -0.5], 14: [0.0, -1.0], 15: [0.0, -1.5], 16: [0.0, -2.0]}
[WRAPPER DiscreteIncrementalWrapper] (INFO) : Wrapper initialized
[WRAPPER NormalizeObservation] (INFO) : wrapper initialized.
[WRAPPER LoggerWrapper] (INFO) : Wrapper initialized.
[WRAPPER MultiObsWrapper] (INFO) : Wrapper initialized.

Now we just simply use the environment with the wrappers, for example:

[18]:

for i in range(1):
    obs, info = env.reset()
    terminated = False
    current_month = 0
    while not terminated:
        a = env.action_space.sample()
        obs, reward, terminated, truncated, info = env.step(a)
        if info['month'] != current_month:  # display results every month
            current_month = info['month']
            print('Reward: ', reward, info)
env.close()

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[WRAPPER LoggerWrapper] (INFO) : Creating monitor.csv for current episode (episode 1) if logger is active
Reward:  [-0.05542625239599594, -0.857887333002763] {'time_elapsed(hours)': 0.5, 'month': 1, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [21.5, 25.0], 'timestep': 2, 'reward': -0.913313585398759, 'energy_term': -0.05542625239599594, 'comfort_term': -0.857887333002763, 'reward_weight': 0.5, 'abs_energy': 1108.5250479199187, 'abs_comfort': 1.715774666005526, 'energy_values': [1108.5250479199187], 'temp_values': [18.284225333994474]}
Progress: |*--------------------------------------------------------------------------------------------------| 1%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Reward:  [-0.00538809382535063, -0.0] {'time_elapsed(hours)': 744.375, 'month': 2, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [19.0, 22.5], 'timestep': 2977, 'reward': -0.00538809382535063, 'energy_term': -0.00538809382535063, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 107.76187650701259, 'abs_comfort': 0.0, 'energy_values': [107.76187650701259], 'temp_values': [20.961309048938297]}
Reward:  [-0.006935757308931089, -0.0] {'time_elapsed(hours)': 1416.25, 'month': 3, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [18.0, 30.0], 'timestep': 5665, 'reward': -0.006935757308931089, 'energy_term': -0.006935757308931089, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 138.71514617862178, 'abs_comfort': 0.0, 'energy_values': [138.71514617862178], 'temp_values': [20.093696177204187]}
Reward:  [-0.0, -0.0] {'time_elapsed(hours)': 2160.25, 'month': 4, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [15.5, 28.5], 'timestep': 8641, 'reward': -0.0, 'energy_term': -0.0, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.0, 'energy_values': [0.0], 'temp_values': [22.11540370667002]}
Reward:  [-0.0, -0.4961122594044092] {'time_elapsed(hours)': 2880.25, 'month': 5, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [18.5, 24.0], 'timestep': 11521, 'reward': -0.4961122594044092, 'energy_term': -0.0, 'comfort_term': -0.4961122594044092, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.9922245188088183, 'energy_values': [0.0], 'temp_values': [24.49222451880882]}
Reward:  [-0.0, -0.0] {'time_elapsed(hours)': 3624.25, 'month': 6, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [22.5, 22.5], 'timestep': 14497, 'reward': -0.0, 'energy_term': -0.0, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.0, 'energy_values': [0.0], 'temp_values': [25.96360689704416]}
Reward:  [-0.0, -0.9700266488211167] {'time_elapsed(hours)': 4344.375, 'month': 7, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [20.0, 24.5], 'timestep': 17377, 'reward': -0.9700266488211167, 'energy_term': -0.0, 'comfort_term': -0.9700266488211167, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 1.9400532976422333, 'energy_values': [0.0], 'temp_values': [27.940053297642233]}
Reward:  [-0.0, -0.006542366753496509] {'time_elapsed(hours)': 5088.25, 'month': 8, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [16.0, 27.5], 'timestep': 20353, 'reward': -0.006542366753496509, 'energy_term': -0.0, 'comfort_term': -0.006542366753496509, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.013084733506993018, 'energy_values': [0.0], 'temp_values': [26.013084733506993]}
Reward:  [-0.0, -0.0] {'time_elapsed(hours)': 5832.25, 'month': 9, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [22.5, 27.5], 'timestep': 23329, 'reward': -0.0, 'energy_term': -0.0, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.0, 'energy_values': [0.0], 'temp_values': [25.800150805217598]}
Reward:  [-0.06446283743025183, -0.9667086618754865] {'time_elapsed(hours)': 6552.3125, 'month': 10, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [21.5, 27.5], 'timestep': 26209, 'reward': -1.0311714993057384, 'energy_term': -0.06446283743025183, 'comfort_term': -0.9667086618754865, 'reward_weight': 0.5, 'abs_energy': 1289.2567486050366, 'abs_comfort': 1.933417323750973, 'energy_values': [1289.2567486050366], 'temp_values': [25.433417323750973]}
Reward:  [-0.004536449977537947, -0.0] {'time_elapsed(hours)': 7296.375, 'month': 11, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [15.0, 26.5], 'timestep': 29185, 'reward': -0.004536449977537947, 'energy_term': -0.004536449977537947, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 90.72899955075894, 'abs_comfort': 0.0, 'energy_values': [90.72899955075894], 'temp_values': [22.02925460307228]}
Reward:  [-0.004536449977537947, -0.0] {'time_elapsed(hours)': 8016.25, 'month': 12, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [15.0, 22.5], 'timestep': 32065, 'reward': -0.004536449977537947, 'energy_term': -0.004536449977537947, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 90.72899955075894, 'abs_comfort': 0.0, 'energy_values': [90.72899955075894], 'temp_values': [21.283078053476807]}
Progress: |***************************************************************************************************| 99%
[WRAPPER LoggerWrapper] (INFO) : End of episode, recording summary (progress.csv) if logger is active
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]