21. Wrappers example

In this notebook, we’ll explore the wrappers defined by Sinergym and how to use them. You can also create your own wrappers by inheriting from gym.Wrapper or its variants.

import gymnasium as gym
import numpy as np
import sinergym
from sinergym.utils.wrappers import *

21.1. Multi-Objective Wrapper

MO-Gymnasium is an open-source Python library for developing and comparing multi-objective reinforcement learning algorithms. These environments return a reward vector instead of a scalar value, one for each objective.

To be as general as possible, it could be beneficial for Sinergym to also provide that reward vector. This way, Sinergym would be compatible with both multi-objective algorithms and algorithms that work with a traditional reward value.

We can transform the returned reward into a vector using the following wrapper:

env=gym.make('Eplus-5zone-hot-discrete-v1')
env=MultiObjectiveReward(env,reward_terms=['energy_term','comfort_term'])

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER MultiObjectiveReward] (INFO) : wrapper initialized.

Ensure that reward_terms are available in the info dict returned in the environment’s step method. Otherwise, an execution error will occur. By default, Sinergym environments return all reward terms specified in the used reward class in the info dict, so if the objective exists in the reward term, you shouldn’t encounter any problems.

env.reset()
action = env.action_space.sample()
obs, reward, terminated, truncated, info = env.step(action)
env.close()
print(reward)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1/output]

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35042/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]
[-0.05542625239599594, -0.857887333002763]

21.2. Previous Observations Wrappers

This Wrapper will add the previous timestep’s observation values to the current environment observation. You can select the variables you want to track its previous observation values. The observation space will be updated with the new dimension.

env=gym.make('Eplus-5zone-hot-discrete-v1')
env = PreviousObservationWrapper(env, previous_variables=[
        'htg_setpoint',
        'clg_setpoint',
        'air_temperature'])

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER PreviousObservationWrapper] (INFO) : Wrapper initialized.

You can see that the observation values have been updated:

env.reset()
obs,_,_,_,_=env.step(env.action_space.sample())
obs_dict=dict(zip(env.get_wrapper_attr('observation_variables'),obs))
env.close()
print('NEW OBSERVATION: ',obs_dict)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35043/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]
NEW OBSERVATION:  {'month': 1.0, 'day_of_month': 1.0, 'hour': 0.0, 'outdoor_temperature': 4.8, 'outdoor_humidity': 61.0, 'wind_speed': 4.65, 'wind_direction': 160.0, 'diffuse_solar_radiation': 0.0, 'direct_solar_radiation': 0.0, 'htg_setpoint': 12.8, 'clg_setpoint': 40.0, 'air_temperature': 18.284225, 'air_humidity': 26.9228, 'people_occupant': 0.0, 'co2_emission': 0.0, 'HVAC_electricity_demand_rate': 1108.525, 'total_electricity_HVAC': 997672.56, 'htg_setpoint_previous': 12.8, 'clg_setpoint_previous': 40.0, 'air_temperature_previous': 18.384174}

21.3. Datetime Wrapper

This wrapper will replace the day value with the is_weekend flag, and hour and month with sin and cos values. The observation space is also automatically updated.

env=gym.make('Eplus-5zone-hot-discrete-v1')
env = DatetimeWrapper(env)

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER DatetimeWrapper] (INFO) : Wrapper initialized.

Specifically, this wrapper removes the observation variables month, day, and hour, and adds month_sin, month_cos, is_weekend, hour_sin, and hour_cos instead:

env.reset()
obs,_,_,_,_=env.step(env.action_space.sample())
obs_dict=dict(zip(env.get_wrapper_attr('observation_variables'),obs))
env.close()
print('NEW OBSERVATION: ',obs_dict)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35044/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]
NEW OBSERVATION:  {'month_cos': 1.0, 'month_sin': 0.0, 'is_weekend': 0.0, 'hour_cos': 1.0, 'hour_sin': 0.0, 'outdoor_temperature': 4.800000190734863, 'outdoor_humidity': 61.0, 'wind_speed': 4.650000095367432, 'wind_direction': 160.0, 'diffuse_solar_radiation': 0.0, 'direct_solar_radiation': 0.0, 'htg_setpoint': 12.800000190734863, 'clg_setpoint': 40.0, 'air_temperature': 18.284225463867188, 'air_humidity': 26.922800064086914, 'people_occupant': 0.0, 'co2_emission': 0.0, 'HVAC_electricity_demand_rate': 1108.5250244140625, 'total_electricity_HVAC': 997672.5625}

21.4. Environment Action Normalization Wrapper

Here’s an example of how to normalize a previous continuous environment action space. If we don’t define the range values, it will default to the range [-1,1]:

# We will create a continuous environment
env=gym.make('Eplus-5zone-hot-continuous-v1')
print('ORIGINAL ACTION SPACE: ',env.get_wrapper_attr('action_space'))
# NORMALIZATION
# Apply the normalize action wrapper
env=NormalizeAction(env,normalize_range=(-1.0,1.0))
print('WRAPPED ACTION SPACE: ',env.get_wrapper_attr('action_space'))
env.reset()
for i in range(5):
    action=env.action_space.sample()
    print('Normalized action: ',action)
    _,_,_,_,info=env.step(action)
    print('Action done in simulator: ', info['action'])
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
ORIGINAL ACTION SPACE:  Box([15.  22.5], [22.5 30. ], (2,), float32)
[WRAPPER NormalizeAction] (INFO) : New normalized action Space: Box(-1.0, 1.0, (2,), float32)
[WRAPPER NormalizeAction] (INFO) : Wrapper initialized
WRAPPED ACTION SPACE:  Box(-1.0, 1.0, (2,), float32)
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35045/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Normalized action:  [ 0.8381089  -0.15050839]
Action done in simulator:  [21.892908, 25.685593]
Normalized action:  [-0.20605884  0.00121972]
Action done in simulator:  [17.97728, 26.254574]
Normalized action:  [ 0.20422778 -0.9112268 ]
Action done in simulator:  [19.515854, 22.8329]
Normalized action:  [-0.6123266  -0.04220473]
Action done in simulator:  [16.453775, 26.091732]
Normalized action:  [ 0.46484488 -0.32572523]
Action done in simulator:  [20.493168, 25.02853]
Progress: |*****----------------------------------------------------------------------------------------------| 5%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

21.5. Environment Discretization Wrapper

Here’s an example of how to discretize a previous continuous environment. We’ll need the new discrete action space and an action mapping function whose output matches the original unwrapped environment action space:

# We will create a continuous environment
env=gym.make('Eplus-5zone-hot-continuous-v1')
print('ORIGINAL ACTION SPACE: ',env.get_wrapper_attr('action_space'))
print('IS DISCRETE?: ',env.get_wrapper_attr('is_discrete'))
# DISCRETIZATION
# Defining new discrete space and action mapping function
new_discrete_space = gym.spaces.Discrete(10) # Action values [0,9]
def action_mapping_function(action):
    mapping = {
        0: [15, 30], # These lists matches with original action space
        1: [16, 29],
        2: [17, 28],
        3: [18, 27],
        4: [19, 26],
        5: [20, 25],
        6: [21, 24],
        7: [22, 23],
        8: [22, 22.5],
        9: [21, 22.5]
    }

    return mapping[action]
# Apply the discretize wrapper
env=DiscretizeEnv(env,discrete_space=new_discrete_space,action_mapping=action_mapping_function)
print('WRAPPED ACTION SPACE: ',env.get_wrapper_attr('action_space'))
print('IS DISCRETE?: ',env.get_wrapper_attr('is_discrete'))
env.reset()
for i in range(5):
    action=env.action_space.sample()
    print('ACTION DISCRETE: ',action)
    _,_,_,_,info=env.step(action)
    print('Action done in simulator: ', info['action'])
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
ORIGINAL ACTION SPACE:  Box([15.  22.5], [22.5 30. ], (2,), float32)
IS DISCRETE?:  False
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
WRAPPED ACTION SPACE:  Discrete(10)
IS DISCRETE?:  True
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35046/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
ACTION DISCRETE:  2
Action done in simulator:  [17, 28]
ACTION DISCRETE:  3
Action done in simulator:  [18, 27]
ACTION DISCRETE:  8
Action done in simulator:  [22, 22.5]
ACTION DISCRETE:  5
Action done in simulator:  [20, 25]
ACTION DISCRETE:  6
Action done in simulator:  [21, 24]
Progress: |*****----------------------------------------------------------------------------------------------| 5%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

As seen in the output, the input is an int, but a list with the original action space is created in the simulator.

21.6. Discrete Incremental Wrapper

This wrapper updates an environment for an incremental setpoint action space. It converts the environment into a discrete environment with an action mapping function and action space depending on the step and delta specified. The action is added to the current setpoint values instead of overwriting the latest action. Thus, the action is the current setpoint values with the increase, not the discrete value action that defines the increment/decrement itself.

env=gym.make('Eplus-5zone-hot-continuous-v1')
print('ORIGINAL ACTION SPACE: ',env.get_wrapper_attr('action_space'))
env = DiscreteIncrementalWrapper(
        env,initial_values=[21.0,25.0], delta_temp=2, step_temp=0.5)
print('WRAPPED ACTION SPACE: ',env.get_wrapper_attr('action_space'))
print('WRAPPED ACTION MAPPING: ',env.get_wrapper_attr('action_mapping'))

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
ORIGINAL ACTION SPACE:  Box([15.  22.5], [22.5 30. ], (2,), float32)
[WRAPPER DiscreteIncrementalWrapper] (INFO) : New incremental action mapping: 17
[WRAPPER DiscreteIncrementalWrapper] (INFO) : {0: [0.0, 0.0], 1: [0.5, 0.0], 2: [1.0, 0.0], 3: [1.5, 0.0], 4: [2.0, 0.0], 5: [-0.5, 0.0], 6: [-1.0, 0.0], 7: [-1.5, 0.0], 8: [-2.0, 0.0], 9: [0.0, 0.5], 10: [0.0, 1.0], 11: [0.0, 1.5], 12: [0.0, 2.0], 13: [0.0, -0.5], 14: [0.0, -1.0], 15: [0.0, -1.5], 16: [0.0, -2.0]}
[WRAPPER DiscreteIncrementalWrapper] (INFO) : Wrapper initialized
WRAPPED ACTION SPACE:  Discrete(17)
WRAPPED ACTION MAPPING:  <bound method DiscreteIncrementalWrapper.action_mapping of <DiscreteIncrementalWrapper<OrderEnforcing<PassiveEnvChecker<EplusEnv<Eplus-5zone-hot-continuous-v1>>>>>>

The maximum and minimum values for creating the action mapping are read from the environment action space, ensuring that the setpoint increments and decrements do not exceed the agreed limits. The delta and step values are used to determine how the discrete space of these increments and decrements will be constructed. Here’s an example of how it works:

env.reset()
print('CURRENT SETPOINTS VALUES: ', env.get_wrapper_attr('current_setpoints'))
for i in range(5):
    action=env.action_space.sample()
    _,_,_,_,info=env.step(action)
    print('Action number ',i,': ',env.get_wrapper_attr('action_mapping')(action))
    print('Setpoints update: ', info['action'])
env.close()

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35047/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
CURRENT SETPOINTS VALUES:  [21.0, 25.0]
Action number  0 :  [-2.0, 0.0]
Setpoints update:  [19.0, 25.0]
Action number  1 :  [0.0, 0.5]
Setpoints update:  [19.0, 25.5]
Action number  2 :  [0.0, -0.5]
Setpoints update:  [19.0, 25.0]
Action number  3 :  [0.0, 0.0]
Setpoints update:  [19.0, 25.0]
Action number  4 :  [0.0, -0.5]
Setpoints update:  [19.0, 24.5]
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

21.7. Normalization Wrapper

This wrapper is used to transform the observation received from the simulator into values between -1 and 1. It’s based on the dynamic normalization wrapper of Gymnasium. Initially, it may not be precise, and the values may often be out of range, so use this wrapper with caution.

#Original env
env=gym.make('Eplus-5zone-hot-discrete-v1')
env = NormalizeObservation(
        env=env)

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER NormalizeObservation] (INFO) : wrapper initialized.

In the following code, you can see how the specified variables have been correctly normalized:

env.reset()
obs,_,_,_,_=env.step(env.action_space.sample())
obs_dict=dict(zip(env.get_wrapper_attr('observation_variables'),obs))
env.close()
print('OBSERVATION WITH NORMALIZATION: ',obs_dict)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35048/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |****-----------------------------------------------------------------------------------------------| 4%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]
OBSERVATION WITH NORMALIZATION:  {'month': 0.00499968750430059, 'day_of_month': 0.00499968750430059, 'hour': 0.0, 'outdoor_temperature': 0.9875907976185373, 'outdoor_humidity': -0.9745626607331893, 'wind_speed': 0.9973969815420386, 'wind_direction': 0.9908525387095581, 'diffuse_solar_radiation': 0.0, 'direct_solar_radiation': 0.0, 'htg_setpoint': 0.007049581561774207, 'clg_setpoint': 0.0070688585864955075, 'air_temperature': -0.35264364590376623, 'air_humidity': 0.05800834342804423, 'people_occupant': 0.0, 'co2_emission': 0.0, 'HVAC_electricity_demand_rate': 0.9802888899782684, 'total_electricity_HVAC': 0.9802889755036568}

21.8. Logger Wrapper

This wrapper logs all interactions between the agent and the environment. You can select the Logger class in the constructor if a different type of logging is required. For more information about the Sinergym Logger, visit Logger

env=gym.make('Eplus-5zone-hot-discrete-v1')
env=LoggerWrapper(env)


env.reset()
truncated = terminated = False
current_month = 0
while not (terminated or truncated):
    a = env.action_space.sample()
    _,_,terminated,truncated,_=env.step(a)
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
[WRAPPER LoggerWrapper] (INFO) : Wrapper initialized.
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35049/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[WRAPPER LoggerWrapper] (INFO) : Creating monitor.csv for current episode (episode 1) if logger is active
Progress: |**-------------------------------------------------------------------------------------------------| 2%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Progress: |***************************************************************************************************| 99%
[WRAPPER LoggerWrapper] (INFO) : End of episode, recording summary (progress.csv) if logger is active
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

Now, if you check the Sinergym output folder, you’ll find progress.csv and monitor.csv files for each episode.

21.9. Multi Observation Wrapper

This wrapper stacks the observations received in a history queue (the size can be customized).

#Original environment
env=gym.make('Eplus-5zone-hot-discrete-v1')
obs, info=env.reset()
print('BEFORE MULTI OBSERVATION: ',obs)

#Multi Observation environment
env=MultiObsWrapper(env, n=5, flatten=True)
obs, info=env.reset()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER DiscretizeEnv] (INFO) : New Discrete Space and mapping: Discrete(10)
[WRAPPER DiscretizeEnv] (INFO) : Make sure that the action space is compatible and contained in the original environment.
[WRAPPER DiscretizeEnv] (INFO) : Wrapper initialized
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
BEFORE MULTI OBSERVATION:  [1.0000000e+00 1.0000000e+00 0.0000000e+00 4.4000001e+00 6.5000000e+01
 3.8750000e+00 1.4500000e+02 0.0000000e+00 0.0000000e+00 1.2800000e+01
 4.0000000e+01 1.8384174e+01 2.6903370e+01 0.0000000e+00 0.0000000e+00
 1.0360292e+03 9.3242619e+05]
[WRAPPER MultiObsWrapper] (INFO) : Wrapper initialized.
Progress: |***************************************************************************************************| 99%
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 2]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35050/Eplus-env-sub_run2/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 2 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

print('AFTER MULTI OBSERVATION: ',obs)
env.close()

AFTER MULTI OBSERVATION:  [1.0000000e+00 1.0000000e+00 0.0000000e+00 4.8000002e+00 6.1000000e+01
 4.6500001e+00 1.6000000e+02 0.0000000e+00 0.0000000e+00 1.2800000e+01
 4.0000000e+01 1.8284225e+01 2.6922800e+01 0.0000000e+00 0.0000000e+00
 1.1085250e+03 9.9767256e+05 1.0000000e+00 1.0000000e+00 0.0000000e+00
 4.8000002e+00 6.1000000e+01 4.6500001e+00 1.6000000e+02 0.0000000e+00
 0.0000000e+00 1.2800000e+01 4.0000000e+01 1.8284225e+01 2.6922800e+01
 0.0000000e+00 0.0000000e+00 1.1085250e+03 9.9767256e+05 1.0000000e+00
 1.0000000e+00 0.0000000e+00 4.8000002e+00 6.1000000e+01 4.6500001e+00
 1.6000000e+02 0.0000000e+00 0.0000000e+00 1.2800000e+01 4.0000000e+01
 1.8284225e+01 2.6922800e+01 0.0000000e+00 0.0000000e+00 1.1085250e+03
 9.9767256e+05 1.0000000e+00 1.0000000e+00 0.0000000e+00 4.8000002e+00
 6.1000000e+01 4.6500001e+00 1.6000000e+02 0.0000000e+00 0.0000000e+00
 1.2800000e+01 4.0000000e+01 1.8284225e+01 2.6922800e+01 0.0000000e+00
 0.0000000e+00 1.1085250e+03 9.9767256e+05 1.0000000e+00 1.0000000e+00
 0.0000000e+00 4.8000002e+00 6.1000000e+01 4.6500001e+00 1.6000000e+02
 0.0000000e+00 0.0000000e+00 1.2800000e+01 4.0000000e+01 1.8284225e+01
 2.6922800e+01 0.0000000e+00 0.0000000e+00 1.1085250e+03 9.9767256e+05]

[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]

21.10. Nesting wrappers

All the wrappers listed in this notebook can be nested to combine functionalities. However, the order in which these wrappers are nested is critical for their proper functionality. We strongly recommend following the same order as they appear here or in the documentation. If you want to nest a subset of the total, it’s not a problem as long as you respect this order. Here’s an example:

env = gym.make('Eplus-5zone-hot-continuous-v1')
env = MultiObjectiveReward(
    env=env,
    reward_terms=[
        'energy_term',
        'comfort_term'])
env = PreviousObservationWrapper(env, previous_variables=[
    'htg_setpoint',
    'clg_setpoint',
    'air_temperature'])
env = DatetimeWrapper(env)
env = DiscreteIncrementalWrapper(
    env,initial_values=[21.0,25.0], delta_temp=2, step_temp=0.5)
env = NormalizeObservation(
    env=env)
env = LoggerWrapper(env=env, flag=True)
env = MultiObsWrapper(env=env, n=5, flatten=True)

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.
[WRAPPER MultiObjectiveReward] (INFO) : wrapper initialized.
[WRAPPER PreviousObservationWrapper] (INFO) : Wrapper initialized.
[WRAPPER DatetimeWrapper] (INFO) : Wrapper initialized.
[WRAPPER DiscreteIncrementalWrapper] (INFO) : New incremental action mapping: 17
[WRAPPER DiscreteIncrementalWrapper] (INFO) : {0: [0.0, 0.0], 1: [0.5, 0.0], 2: [1.0, 0.0], 3: [1.5, 0.0], 4: [2.0, 0.0], 5: [-0.5, 0.0], 6: [-1.0, 0.0], 7: [-1.5, 0.0], 8: [-2.0, 0.0], 9: [0.0, 0.5], 10: [0.0, 1.0], 11: [0.0, 1.5], 12: [0.0, 2.0], 13: [0.0, -0.5], 14: [0.0, -1.0], 15: [0.0, -1.5], 16: [0.0, -2.0]}
[WRAPPER DiscreteIncrementalWrapper] (INFO) : Wrapper initialized
[WRAPPER NormalizeObservation] (INFO) : wrapper initialized.
[WRAPPER LoggerWrapper] (INFO) : Wrapper initialized.
[WRAPPER MultiObsWrapper] (INFO) : Wrapper initialized.

Now we simply use the environment with the wrappers, for example:

for i in range(1):
    obs, info = env.reset()
    truncated = terminated = False
    current_month = 0
    while not (terminated or truncated):
        a = env.action_space.sample()
        obs, reward, terminated, truncated, info = env.step(a)
        if info['month'] != current_month:  # display results every month
            current_month = info['month']
            print('Reward: ', reward, info)
env.close()

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-discrete-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res35051/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[WRAPPER LoggerWrapper] (INFO) : Creating monitor.csv for current episode (episode 1) if logger is active
Reward:  [-0.05542625239599594, -0.857887333002763] {'time_elapsed(hours)': 0.5, 'month': 1, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [21.5, 25.0], 'timestep': 2, 'reward': -0.913313585398759, 'energy_term': -0.05542625239599594, 'comfort_term': -0.857887333002763, 'reward_weight': 0.5, 'abs_energy': 1108.5250479199187, 'abs_comfort': 1.715774666005526, 'energy_values': [1108.5250479199187], 'temp_values': [18.284225333994474]}
Progress: |*--------------------------------------------------------------------------------------------------| 1%

/usr/local/lib/python3.10/dist-packages/gymnasium/spaces/box.py:240: UserWarning: WARN: Casting input x to numpy array.
  gym.logger.warn("Casting input x to numpy array.")

Reward:  [-0.00538809382535063, -0.0] {'time_elapsed(hours)': 744.375, 'month': 2, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [19.0, 22.5], 'timestep': 2977, 'reward': -0.00538809382535063, 'energy_term': -0.00538809382535063, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 107.76187650701259, 'abs_comfort': 0.0, 'energy_values': [107.76187650701259], 'temp_values': [20.961309048938297]}
Reward:  [-0.006935757308931089, -0.0] {'time_elapsed(hours)': 1416.25, 'month': 3, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [18.0, 30.0], 'timestep': 5665, 'reward': -0.006935757308931089, 'energy_term': -0.006935757308931089, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 138.71514617862178, 'abs_comfort': 0.0, 'energy_values': [138.71514617862178], 'temp_values': [20.093696177204187]}
Reward:  [-0.0, -0.0] {'time_elapsed(hours)': 2160.25, 'month': 4, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [15.5, 28.5], 'timestep': 8641, 'reward': -0.0, 'energy_term': -0.0, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.0, 'energy_values': [0.0], 'temp_values': [22.11540370667002]}
Reward:  [-0.0, -0.4961122594044092] {'time_elapsed(hours)': 2880.25, 'month': 5, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [18.5, 24.0], 'timestep': 11521, 'reward': -0.4961122594044092, 'energy_term': -0.0, 'comfort_term': -0.4961122594044092, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.9922245188088183, 'energy_values': [0.0], 'temp_values': [24.49222451880882]}
Reward:  [-0.0, -0.0] {'time_elapsed(hours)': 3624.25, 'month': 6, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [22.5, 22.5], 'timestep': 14497, 'reward': -0.0, 'energy_term': -0.0, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.0, 'energy_values': [0.0], 'temp_values': [25.96360689704416]}
Reward:  [-0.0, -0.9700266488211167] {'time_elapsed(hours)': 4344.375, 'month': 7, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [20.0, 24.5], 'timestep': 17377, 'reward': -0.9700266488211167, 'energy_term': -0.0, 'comfort_term': -0.9700266488211167, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 1.9400532976422333, 'energy_values': [0.0], 'temp_values': [27.940053297642233]}
Reward:  [-0.0, -0.006542366753496509] {'time_elapsed(hours)': 5088.25, 'month': 8, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [16.0, 27.5], 'timestep': 20353, 'reward': -0.006542366753496509, 'energy_term': -0.0, 'comfort_term': -0.006542366753496509, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.013084733506993018, 'energy_values': [0.0], 'temp_values': [26.013084733506993]}
Reward:  [-0.0, -0.0] {'time_elapsed(hours)': 5832.25, 'month': 9, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [22.5, 27.5], 'timestep': 23329, 'reward': -0.0, 'energy_term': -0.0, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 0.0, 'abs_comfort': 0.0, 'energy_values': [0.0], 'temp_values': [25.800150805217598]}
Reward:  [-0.06446283743025183, -0.9667086618754865] {'time_elapsed(hours)': 6552.3125, 'month': 10, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [21.5, 27.5], 'timestep': 26209, 'reward': -1.0311714993057384, 'energy_term': -0.06446283743025183, 'comfort_term': -0.9667086618754865, 'reward_weight': 0.5, 'abs_energy': 1289.2567486050366, 'abs_comfort': 1.933417323750973, 'energy_values': [1289.2567486050366], 'temp_values': [25.433417323750973]}
Reward:  [-0.004536449977537947, -0.0] {'time_elapsed(hours)': 7296.375, 'month': 11, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [15.0, 26.5], 'timestep': 29185, 'reward': -0.004536449977537947, 'energy_term': -0.004536449977537947, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 90.72899955075894, 'abs_comfort': 0.0, 'energy_values': [90.72899955075894], 'temp_values': [22.02925460307228]}
Reward:  [-0.004536449977537947, -0.0] {'time_elapsed(hours)': 8016.25, 'month': 12, 'day': 1, 'hour': 0, 'is_raining': False, 'action': [15.0, 22.5], 'timestep': 32065, 'reward': -0.004536449977537947, 'energy_term': -0.004536449977537947, 'comfort_term': -0.0, 'reward_weight': 0.5, 'abs_energy': 90.72899955075894, 'abs_comfort': 0.0, 'energy_values': [90.72899955075894], 'temp_values': [21.283078053476807]}
Progress: |***************************************************************************************************| 99%
[WRAPPER LoggerWrapper] (INFO) : End of episode, recording summary (progress.csv) if logger is active
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-discrete-v1]