18. Changing an environment registered in Sinergym

As discussed above, Sinergym has a list of available environments which we can call with gym.make(<environment_id>) as long as the Sinergym package is imported into the python script.

import gymnasium as gym
import numpy as np

import sinergym
env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-stochastic-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-stochastic-v1-res2]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-stochastic-v1 created successfully.

These environment ID’s have a number of components defined, not only the building design (epJSON), but also the reward function, the action and observation spaces, the definition of variables, meters and actuators, etc.

If you want a new environment, you can define it from scratch in our environment list (take the env demo as example) and run it locally.

Another option (recommended) is starting from one of our environments ID and changing the components you want. You can combine different components obviously.

The way to do that is to add in the gym.make(<environment_id>) those parameters of the constructor of the Sinergym environment we want to overwrite.

Many of this updates require the building model changes in order to adapt it to this new features, this will be made by Sinergym automatically. For example, using another weather file requires building location and design days update, using new observation variables requires to update the Output:Variable fields, the same occurs with extra configuration context concerned with simulation directly. This new building file version, is saved in the Sinergym output folder, leaving the original intact.

Let’s see some things we can update starting from any environment:

18.1. Adding a new reward

As mentioned above, simply add the appropriate parameters to gym.make() after specifying the environment ID.

from sinergym.utils.rewards import ExpReward

env = gym.make('Eplus-5zone-hot-continuous-v1', reward=ExpReward, reward_kwargs={
                                                                    'temperature_variables': 'air_temperature',
                                                                    'energy_variables': 'HVAC_electricity_demand_rate',
                                                                    'range_comfort_winter': (20.0, 23.5),
                                                                    'range_comfort_summer': (23.0, 26.0),
                                                                    'energy_weight': 0.1})

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-discrete-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-discrete-v1-res2]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-discrete-v1 created successfully.

You have to specify the reward class you are going to use in the environment. A reward function class has several parameters that can be specified by user like temperature variables, weights, etc. Depending on the reward function you are using. In order to be able to define it, we have reward_kwargs parameter. See reward documentation for more information about reward classes and how to create a new one.

18.2. Updating the weather used in environment

It is possible to substitute the weather file (and the weather variability if you want a stochastic environment) used in the environment by a new one. The methodology is the same than the reward example:

env = gym.make('Eplus-5zone-cool-continuous-stochastic-v1',
                weather_files='ESP_Granada.084190_SWEC.epw',
                weather_variability=(1.0,0.0,0.001))

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-cool-discrete-stochastic-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-cool-discrete-stochastic-v1-res1]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-cool-discrete-stochastic-v1 created successfully.

It is possible to add more than one weather file too. In that case, Sinergym will select one of the available weathers randomly and will adapt the building model to that location in each episode:

env = gym.make('Eplus-5zone-cool-continuous-stochastic-v1',
                weather_files=['ESP_Granada.084190_SWEC.epw','FIN_Helsinki.029740_IWEC.epw','PRT_Lisboa.085360_INETI.epw'])

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-cool-discrete-stochastic-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-cool-discrete-stochastic-v1-res2]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-cool-discrete-stochastic-v1 created successfully.

18.3. Changing observation and action spaces

By default, the IDs of the predefined environments in Sinergym already have the time variables, variables, meters and actuators set up, together observation (time variables, variables and meters) and action (actuators) spaces for those components.

However, it can be overwritten by a new definition of them. On the one hand, we will have to define the name of the time variables, variables, meters or actuators, and on the other hand, the definition of the action space (and an action mapping if it is a discrete environment). Observation space is calculated automatically by Sinergym. It is possible to change a subset of these components too. For more information about the structure definitions shown bellow, visit the documentation.

import gymnasium as gym
import numpy as np

import sinergym

new_time_variables=['month', 'day_of_month', 'hour']

new_variables={
    'outdoor_temperature': ('Site Outdoor Air Drybulb Temperature', 'Environment'),
    'outdoor_humidity': ('Site Outdoor Air Relative Humidity', 'Environment'),
    'wind_speed': ('Site Wind Speed', 'Environment'),
    'wind_direction': ('Site Wind Direction', 'Environment'),
    'diffuse_solar_radiation': ('Site Diffuse Solar Radiation Rate per Area', 'Environment'),
    'direct_solar_radiation': ('Site Direct Solar Radiation Rate per Area', 'Environment'),
    'west_zone_htg_setpoint': ('Zone Thermostat Heating Setpoint Temperature', 'West Zone'),
    'east_zone_htg_setpoint': ('Zone Thermostat Heating Setpoint Temperature', 'East Zone'),
    'west_zone_clg_setpoint': ('Zone Thermostat Cooling Setpoint Temperature', 'West Zone'),
    'east_zone_clg_setpoint': ('Zone Thermostat Cooling Setpoint Temperature', 'East Zone'),
    'west_zone_air_temperature': ('Zone Air Temperature', 'West Zone'),
    'east_zone_air_temperature': ('Zone Air Temperature', 'East Zone'),
    'west_zone_air_humidity': ('Zone Air Relative Humidity', 'West Zone'),
    'east_zone_air_humidity': ('Zone Air Relative Humidity', 'East Zone'),
    'HVAC_electricity_demand_rate': ('Facility Total HVAC Electricity Demand Rate', 'Whole Building')
}

new_meters={
    'east_zone_electricity':'Electricity:Zone:EAST ZONE',
    'west_zone_electricity':'Electricity:Zone:WEST ZONE',
}

new_actuators = {
    'Heating_Setpoint_RL': (
        'Schedule:Compact',
        'Schedule Value',
        'Heating Setpoints'),
    'Cooling_Setpoint_RL': (
        'Schedule:Compact',
        'Schedule Value',
        'Cooling Setpoints')
}

new_action_space = gym.spaces.Box(
    low=np.array([14.0, 22.0], dtype=np.float32),
    high=np.array([22.0, 30.5], dtype=np.float32),
    shape=(2,),
    dtype=np.float32)

env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1',
                time_variables=new_time_variables,
                variables=new_variables,
                meters=new_meters,
                actuators=new_actuators,
                action_space=new_action_space,
            )

print('New environment observation varibles (time variables + variables + meters): {}'.format(env.get_wrapper_attr('observation_variables')))
print('New environment action varibles (actuators): {}'.format(env.get_wrapper_attr('action_variables')))
for i in range(1):
    obs, info = env.reset()
    rewards = []
    truncated = terminated = False
    current_month = 0
    while not (terminated or truncated):
        a = env.action_space.sample()
        obs, reward, terminated, truncated, info = env.step(a)
        rewards.append(reward)
    print(
        'Episode ',
        i,
        'Mean reward: ',
        np.mean(rewards),
        'Cumulative reward: ',
        sum(rewards))
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [datacenter-cool-discrete-stochastic-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res2]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment datacenter-cool-discrete-stochastic-v1 created successfully.
New environment observation varibles (time variables + variables + meters): ['month', 'day_of_month', 'hour', 'outdoor_temperature', 'outdoor_humidity', 'wind_speed', 'wind_direction', 'diffuse_solar_radiation', 'direct_solar_radiation', 'west_zone_htg_setpoint', 'east_zone_htg_setpoint', 'west_zone_clg_setpoint', 'east_zone_clg_setpoint', 'west_zone_air_temperature', 'east_zone_air_temperature', 'west_zone_air_humidity', 'east_zone_air_humidity', 'HVAC_electricity_demand_rate', 'east_zone_electricity', 'west_zone_electricity']
New environment action varibles (actuators): ['Heating_Setpoint_RL', 'Cooling_Setpoint_RL']
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [datacenter-cool-discrete-stochastic-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res2/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Adapting weather to building model. [USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res2/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res2/Eplus-env-sub_run1/USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3_Random_1.0_0.0_0.001.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res2/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res2/Eplus-env-sub_run1/2ZoneDataCenterHVAC_wEconomizer.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |***************************************************************************************************| 99%
Episode  0 Mean reward:  -1.1076129748179115 Cumulative reward:  -38810.758637619416
[ENVIRONMENT] (INFO) : Environment closed. [datacenter-cool-discrete-stochastic-v1]

In case the definition has some inconsistency, such as the spaces do not fit with the variables, the observation variables do not exist, etc. Sinergym will display an error.

The time variables, variables, meters or actuators definition must be correct for Energyplus Engine. Otherwise, an error message will be shown by Sinergym. See documentation for more information.

18.4. Adding more extra configuration

It is possible to update any argument of the environment constructor in Sinergym. Consequently, we can make other changes such as the name of the environment or the maximum number of episodes it will store in its output or the normalization in the gym actions interface.

env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1',
                env_name='new_env_name',
                max_ep_data_store_num=20
                )

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [new_env_name]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-new_env_name-res1]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment new_env_name created successfully.

You can even add a dictionary with extra parameters that update building model with some new context concerned with simulation directly:

extra_conf={
    'timesteps_per_hour':6,
    'runperiod':(1,1,1991,2,1,1991),
}

env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1',
                config_params=extra_conf
                )

env.reset()
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [datacenter-cool-discrete-stochastic-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res3]
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 1, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[MODELING] (INFO) : Model Config is correct.
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment datacenter-cool-discrete-stochastic-v1 created successfully.
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [datacenter-cool-discrete-stochastic-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res3/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3.epw used.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : Extra config: runperiod updated to {'apply_weekend_holiday_rule': 'No', 'begin_day_of_month': 1, 'begin_month': 1, 'begin_year': 1991, 'day_of_week_for_start_day': 'Tuesday', 'end_day_of_month': 2, 'end_month': 1, 'end_year': 1991, 'use_weather_file_daylight_saving_period': 'Yes', 'use_weather_file_holidays_and_special_days': 'Yes', 'use_weather_file_rain_indicators': 'Yes', 'use_weather_file_snow_indicators': 'Yes'}
[MODELING] (INFO) : Updated episode length (seconds): 172800.0
[MODELING] (INFO) : Updated timestep size (seconds): 600.0
[MODELING] (INFO) : Updated timesteps per episode: 288
[MODELING] (INFO) : Adapting weather to building model. [USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res3/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res3/Eplus-env-sub_run1/USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3_Random_1.0_0.0_0.001.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res3/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-datacenter-cool-discrete-stochastic-v1-res3/Eplus-env-sub_run1/2ZoneDataCenterHVAC_wEconomizer.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.

/usr/local/lib/python3.10/dist-packages/opyplus/weather_data/weather_data.py:493: FutureWarning: the 'line_terminator'' keyword is deprecated, use 'lineterminator' instead.
  epw_content = self._headers_to_epw(use_datetimes=use_datetimes) + df.to_csv(

[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Progress: |**************************************************-------------------------------------------------| 50%
[ENVIRONMENT] (INFO) : Environment closed. [datacenter-cool-discrete-stochastic-v1]

For more information about extra configuration parameters, see our Extra configuration documentation