21. Changing an environment registered in Sinergym

To use the available environments in Sinergym, simply import the Sinergym package and call gym.make(<environment_id>) in your Python script. The list of available environments can be found in the Sinergym documentation.

import gymnasium as gym
import numpy as np

import sinergym
env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: 5zone-hot-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

Environment IDs in Sinergym encompass various components, including the building design (epJSON), reward function, action and observation spaces, variable definitions, meters, actuators, and more.

If you wish to create a new environment, you can define it from scratch in our environment list (use the env demo as an example) and run it locally.

Alternatively (and recommended), you can start with one of our environment IDs and modify the components as needed. You can mix and match different components.

To do this, add the parameters you want to overwrite in the gym.make(<environment_id>) constructor of the Sinergym environment.

Many of these updates need changes to the building model to accommodate new features. Sinergym will automatically handle these changes. For instance, using a different weather file requires updating the building location and design days. Similarly, using new observation variables requires updating the Output:Variable fields. The same applies to any additional configuration context directly related to the simulation. Sinergym saves this new building file version in its output folder, leaving the original untouched.

Let’s explore some of the elements we can update from any environment:

21.1. Adding a new reward

As mentioned above, simply add the appropriate parameters to gym.make() after specifying the environment ID.

from sinergym.utils.rewards import ExpReward

env = gym.make('Eplus-5zone-hot-continuous-v1', reward=ExpReward, reward_kwargs={
                                                                    'temperature_variables': 'air_temperature',
                                                                    'energy_variables': 'HVAC_electricity_demand_rate',
                                                                    'range_comfort_winter': (20.0, 23.5),
                                                                    'range_comfort_summer': (23.0, 26.0),
                                                                    'energy_weight': 0.1})

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: 5zone-hot-continuous-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-v1-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

You need to define the reward class for the environment. A reward function class has various user-specifiable parameters, such as temperature variables, weights, etc., depending on the chosen reward function. To define it, we use the reward_kwargs parameter. Refer to the reward documentation for more details on reward classes and how to create a new one.

21.2. Updating the weather used in environment

You can replace the weather file (and weather variability for a stochastic environment) used in the environment with a new one. The approach is the same as the reward example:

env = gym.make('Eplus-5zone-cool-continuous-stochastic-v1',
                weather_files='PRT_Lisboa.085360_INETI.epw',
                weather_variability={
                                        "drybulb": (1.0, 0.0, 0.001),
                                        "windspd": (3.0, 0.0, 0.01)
                                    })

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: 5zone-cool-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-5zone-cool-continuous-stochastic-v1-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

You can also add multiple weather files. In this scenario, Sinergym will randomly select one of the available weather files and adjust the building model to that location for each episode:

env = gym.make('Eplus-5zone-cool-continuous-stochastic-v1',
                weather_files=['ESP_Granada.084190_SWEC.epw','FIN_Helsinki.029740_IWEC.epw','PRT_Lisboa.085360_INETI.epw'])

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: 5zone-cool-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-5zone-cool-continuous-stochastic-v1-res2
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

21.3. Changing observation and action spaces

By default, the IDs of the predefined environments in Sinergym come with time variables, variables, meters, and actuators pre-configured, along with observation (time variables, variables, and meters) and action (actuators) spaces for these components.

However, these can be overwritten with new definitions. We’ll need to define the names of the time variables, variables, meters, or actuators, as well as the action space definition (and an action mapping for discrete environments). Sinergym automatically calculates the observation space. It’s also possible to modify a subset of these components. For more details on the structure definitions shown below, refer to the documentation.

import gymnasium as gym
import numpy as np

import sinergym

new_time_variables=['month', 'day_of_month', 'hour']

new_variables={
    'outdoor_temperature': ('Site Outdoor Air Drybulb Temperature', 'Environment'),
    'outdoor_humidity': ('Site Outdoor Air Relative Humidity', 'Environment'),
    'wind_speed': ('Site Wind Speed', 'Environment'),
    'wind_direction': ('Site Wind Direction', 'Environment'),
    'diffuse_solar_radiation': ('Site Diffuse Solar Radiation Rate per Area', 'Environment'),
    'direct_solar_radiation': ('Site Direct Solar Radiation Rate per Area', 'Environment'),
    'west_zone_htg_setpoint': ('Zone Thermostat Heating Setpoint Temperature', 'West Zone'),
    'east_zone_htg_setpoint': ('Zone Thermostat Heating Setpoint Temperature', 'East Zone'),
    'west_zone_clg_setpoint': ('Zone Thermostat Cooling Setpoint Temperature', 'West Zone'),
    'east_zone_clg_setpoint': ('Zone Thermostat Cooling Setpoint Temperature', 'East Zone'),
    'west_zone_air_temperature': ('Zone Air Temperature', 'West Zone'),
    'east_zone_air_temperature': ('Zone Air Temperature', 'East Zone'),
    'west_zone_air_humidity': ('Zone Air Relative Humidity', 'West Zone'),
    'east_zone_air_humidity': ('Zone Air Relative Humidity', 'East Zone'),
    'HVAC_electricity_demand_rate': ('Facility Total HVAC Electricity Demand Rate', 'Whole Building')
}

new_meters={
    'east_zone_electricity':'Electricity:Zone:EAST ZONE',
    'west_zone_electricity':'Electricity:Zone:WEST ZONE',
}

new_actuators = {
    'Heating_Setpoint_RL': (
        'Schedule:Compact',
        'Schedule Value',
        'Heating Setpoints'),
    'Cooling_Setpoint_RL': (
        'Schedule:Compact',
        'Schedule Value',
        'Cooling Setpoints')
}

new_action_space = gym.spaces.Box(
    low=np.array([14.0, 22.0], dtype=np.float32),
    high=np.array([22.0, 30.5], dtype=np.float32),
    shape=(2,),
    dtype=np.float32)

env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1',
                time_variables=new_time_variables,
                variables=new_variables,
                meters=new_meters,
                actuators=new_actuators,
                action_space=new_action_space,
            )

print('New environment observation varibles (time variables + variables + meters): {}'.format(env.get_wrapper_attr('observation_variables')))
print('New environment action varibles (actuators): {}'.format(env.get_wrapper_attr('action_variables')))
for i in range(1):
    obs, info = env.reset()
    rewards = []
    truncated = terminated = False
    current_month = 0
    while not (terminated or truncated):
        a = env.action_space.sample()
        obs, reward, terminated, truncated, info = env.step(a)
        rewards.append(reward)
    print(
        'Episode ',
        i,
        'Mean reward: ',
        np.mean(rewards),
        'Cumulative reward: ',
        sum(rewards))
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: datacenter-cool-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-datacenter-cool-continuous-stochastic-v1-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.
New environment observation varibles (time variables + variables + meters): ['month', 'day_of_month', 'hour', 'outdoor_temperature', 'outdoor_humidity', 'wind_speed', 'wind_direction', 'diffuse_solar_radiation', 'direct_solar_radiation', 'west_zone_htg_setpoint', 'east_zone_htg_setpoint', 'west_zone_clg_setpoint', 'east_zone_clg_setpoint', 'west_zone_air_temperature', 'east_zone_air_temperature', 'west_zone_air_humidity', 'east_zone_air_humidity', 'HVAC_electricity_demand_rate', 'east_zone_electricity', 'west_zone_electricity']
New environment action varibles (actuators): ['Heating_Setpoint_RL', 'Cooling_Setpoint_RL']
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode.
[ENVIRONMENT] (INFO) : Episode 1: datacenter-cool-continuous-stochastic-v1
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created.
[MODELING] (INFO) : Weather file USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3.epw used.
[MODELING] (INFO) : Adapting weather to building model.
[ENVIRONMENT] (INFO) : Saving episode output path.
[ENVIRONMENT] (INFO) : Episode 1 started.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Simulation Progress [Episode 1]: 100%|██████████| 100/100 [00:26<00:00,  4.07%/s, 100% completed]Episode  0 Mean reward:  -0.6645031249190511 Cumulative reward:  -23284.189497163552
Simulation Progress [Episode 1]: 100%|██████████| 100/100 [00:28<00:00,  3.47%/s, 100% completed]
[ENVIRONMENT] (INFO) : Environment closed. [datacenter-cool-continuous-stochastic-v1]

If the definition contains inconsistencies, such as mismatched spaces and variables, or non-existent observation variables, Sinergym will throw an error.

The definitions for time variables, variables, meters, or actuators must be compatible with the EnergyPlus Engine. If not, Sinergym will display an error message. You can define the spaces using our environment register too. Refer to the documentation for more details.

21.4. Adding more extra configuration

In Sinergym, any argument of the environment constructor can be updated. This allows for modifications such as changing the environment name, adjusting the maximum number of episodes stored in its output, or altering the normalization in the gym actions interface.

env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1',
                env_name='new_env_name',
                max_ep_data_store_num=20
                )

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: new_env_name
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-new_env_name-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

You can also include a dictionary with additional parameters to update the building model directly with new simulation-related context:

extra_conf={
    'timesteps_per_hour':6,
    'runperiod':(1,1,1991,2,1,1991),
}

env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1',
                config_params=extra_conf
                )

env.reset()
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: datacenter-cool-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-datacenter-cool-continuous-stochastic-v1-res2
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Extra config: runperiod updated to {'apply_weekend_holiday_rule': 'No', 'begin_day_of_month': 1, 'begin_month': 1, 'begin_year': 1991, 'day_of_week_for_start_day': 'Tuesday', 'end_day_of_month': 2, 'end_month': 1, 'end_year': 1991, 'use_weather_file_daylight_saving_period': 'Yes', 'use_weather_file_holidays_and_special_days': 'Yes', 'use_weather_file_rain_indicators': 'Yes', 'use_weather_file_snow_indicators': 'Yes'}
[MODELING] (INFO) : Updated episode length (seconds): 172800.0
[MODELING] (INFO) : Updated timestep size (seconds): 600.0
[MODELING] (INFO) : Updated timesteps per episode: 288
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 172800.0
[MODELING] (INFO) : timestep size (seconds): 600.0
[MODELING] (INFO) : timesteps per episode: 288
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode.
[ENVIRONMENT] (INFO) : Episode 1: datacenter-cool-continuous-stochastic-v1
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created.
[MODELING] (INFO) : Weather file USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3.epw used.
[MODELING] (INFO) : Adapting weather to building model.
[ENVIRONMENT] (INFO) : Saving episode output path.
[ENVIRONMENT] (INFO) : Episode 1 started.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[ENVIRONMENT] (INFO) : Environment closed. [datacenter-cool-continuous-stochastic-v1]
Simulation Progress [Episode 1]:  51%|█████     | 51/100 [00:00<00:00, 1534.34%/s, 51% completed]

For more information about extra configuration parameters, see our Extra configuration documentation