Changing an environment registered in Sinergym

To use any available environment in Sinergym , simply import the sinergym package and call gym.make(<environment_id>) in your Python script. The list of available environments can be found in the Sinergym documentation.

import gymnasium as gym
import numpy as np

import sinergym

env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: 5zone-hot-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

Sinergym environments are composed by several components, including a building model (an epJSON file), a reward function, the action and observation spaces, variable definitions, meters, actuators, and more.

If you want to create a new environment, you can add it to the environment list and run it locally.

Alternatively, it is recommended to start with one of the pre-defined environment IDs and modify its components as needed.

To do this, add the parameters you want to overwrite in the gym.make(<environment_id>) constructor of the Sinergym environment.

Many of these updates need changes to the building model to accommodate new features. Sinergym will automatically handle these changes. For instance, using a different weather file requires updating the building location and design days. Similarly, using new observation variables requires updating the Output:Variable fields. The same applies to any additional configuration context directly related to the simulation. Sinergym saves this new building file version in its output folder, leaving the original untouched.

Let’s explore some of the elements that can be updated for any environment:

Adding a new reward

When adding a new reward function, as mentioned above, simply add the appropriate parameters to gym.make() after specifying the environment ID.

First, define the reward class, and then specify the required parameters. A reward function class has several user-specifiable parameters, such as temperature variables, weights, etc., depending on the chosen reward function. To define them, we use the reward_kwargs parameter. Refer to the reward documentation for more details on reward classes and how to create a new one.

from sinergym.utils.rewards import ExpReward

env = gym.make('Eplus-5zone-hot-continuous-v1', reward=ExpReward, reward_kwargs={
    'temperature_variables': 'air_temperature',
    'energy_variables': 'HVAC_electricity_demand_rate',
    'range_comfort_winter': (20.0, 23.5),
    'range_comfort_summer': (23.0, 26.0),
    'energy_weight': 0.1})

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: 5zone-hot-continuous-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-v1-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

Updating the weather used by the environment

You can configure the weather dataset (and weather variability for stochastic environments) used by the environment with a new one.

The approach is similar as for the reward case:

env = gym.make('Eplus-5zone-cool-continuous-stochastic-v1',
               weather_files='PRT_Lisboa.085360_INETI.epw',
               weather_variability={
                   "drybulb": (1.0, 0.0, 0.001),
                   "windspd": (3.0, 0.0, 0.01)
               })

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: 5zone-cool-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-5zone-cool-continuous-stochastic-v1-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

You can also configure multiple weather files to be used. In this scenario, Sinergym will randomly select one of the available weather files and adjust the building model to that location for each episode:

env = gym.make('Eplus-5zone-cool-continuous-stochastic-v1',
               weather_files=['ESP_Granada.084190_SWEC.epw', 'FIN_Helsinki.029740_IWEC.epw', 'PRT_Lisboa.085360_INETI.epw'])

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: 5zone-cool-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-5zone-cool-continuous-stochastic-v1-res2
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

Changing observation and action spaces

By default, the default environments provided by Sinergym include pre-configured time variables, variables, meters, and actuators, along with observation (time variables, variables, and meters) and action (actuators) spaces for these components.

However, these can be overwritten with new definitions. You will need to define the names of the new time variables, variables, meters, or actuators, as well as the action space definition (and an action mapping for discrete environments).

Sinergym automatically calculates the observation space. It is also possible to modify a subset of these components. For more details on the structure definitions shown below, refer to the documentation.

import gymnasium as gym
import numpy as np

import sinergym

new_time_variables = ['month', 'day_of_month', 'hour']

new_variables = {
    'outdoor_temperature': ('Site Outdoor Air Drybulb Temperature', 'Environment'),
    'outdoor_humidity': ('Site Outdoor Air Relative Humidity', 'Environment'),
    'wind_speed': ('Site Wind Speed', 'Environment'),
    'wind_direction': ('Site Wind Direction', 'Environment'),
    'diffuse_solar_radiation': ('Site Diffuse Solar Radiation Rate per Area', 'Environment'),
    'direct_solar_radiation': ('Site Direct Solar Radiation Rate per Area', 'Environment'),
    'west_zone_htg_setpoint': ('Zone Thermostat Heating Setpoint Temperature', 'West Zone'),
    'east_zone_htg_setpoint': ('Zone Thermostat Heating Setpoint Temperature', 'East Zone'),
    'west_zone_clg_setpoint': ('Zone Thermostat Cooling Setpoint Temperature', 'West Zone'),
    'east_zone_clg_setpoint': ('Zone Thermostat Cooling Setpoint Temperature', 'East Zone'),
    'west_zone_air_temperature': ('Zone Air Temperature', 'West Zone'),
    'east_zone_air_temperature': ('Zone Air Temperature', 'East Zone'),
    'west_zone_air_humidity': ('Zone Air Relative Humidity', 'West Zone'),
    'east_zone_air_humidity': ('Zone Air Relative Humidity', 'East Zone'),
    'HVAC_electricity_demand_rate': ('Facility Total HVAC Electricity Demand Rate', 'Whole Building')
}

new_meters = {
    'east_zone_electricity': 'Electricity:Zone:EAST ZONE',
    'west_zone_electricity': 'Electricity:Zone:WEST ZONE',
}

new_actuators = {
    'Heating_Setpoint_RL': (
        'Schedule:Compact',
        'Schedule Value',
        'Heating Setpoints'),
    'Cooling_Setpoint_RL': (
        'Schedule:Compact',
        'Schedule Value',
        'Cooling Setpoints')
}

new_action_space = gym.spaces.Box(
    low=np.array([14.0, 22.0], dtype=np.float32),
    high=np.array([22.0, 30.5], dtype=np.float32),
    shape=(2,),
    dtype=np.float32)

env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1',
               time_variables=new_time_variables,
               variables=new_variables,
               meters=new_meters,
               actuators=new_actuators,
               action_space=new_action_space,
               )

print('New environment observation variables (time variables + variables + meters): {}'.format(
    env.get_wrapper_attr('observation_variables')))
print('New environment action variables (actuators): {}'.format(
    env.get_wrapper_attr('action_variables')))
for i in range(1):
    obs, info = env.reset()
    rewards = []
    truncated = terminated = False
    current_month = 0
    while not (terminated or truncated):
        a = env.action_space.sample()
        obs, reward, terminated, truncated, info = env.step(a)
        rewards.append(reward)
    print(
        'Episode ',
        i,
        'Mean reward: ',
        np.mean(rewards),
        'Cumulative reward: ',
        sum(rewards))
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: datacenter-cool-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-datacenter-cool-continuous-stochastic-v1-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.
New environment observation varibles (time variables + variables + meters): ['month', 'day_of_month', 'hour', 'outdoor_temperature', 'outdoor_humidity', 'wind_speed', 'wind_direction', 'diffuse_solar_radiation', 'direct_solar_radiation', 'west_zone_htg_setpoint', 'east_zone_htg_setpoint', 'west_zone_clg_setpoint', 'east_zone_clg_setpoint', 'west_zone_air_temperature', 'east_zone_air_temperature', 'west_zone_air_humidity', 'east_zone_air_humidity', 'HVAC_electricity_demand_rate', 'east_zone_electricity', 'west_zone_electricity']
New environment action varibles (actuators): ['Heating_Setpoint_RL', 'Cooling_Setpoint_RL']
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode.
[ENVIRONMENT] (INFO) : Episode 1: datacenter-cool-continuous-stochastic-v1
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created.
[MODELING] (INFO) : Weather file USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3.epw used.
[MODELING] (INFO) : Adapting weather to building model.
[ENVIRONMENT] (INFO) : Saving episode output path.
[ENVIRONMENT] (INFO) : Episode 1 started.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
Simulation Progress [Episode 1]: 100%|██████████| 100/100 [00:26<00:00,  4.07%/s, 100% completed]Episode  0 Mean reward:  -0.6645031249190511 Cumulative reward:  -23284.189497163552
Simulation Progress [Episode 1]: 100%|██████████| 100/100 [00:28<00:00,  3.47%/s, 100% completed]
[ENVIRONMENT] (INFO) : Environment closed. [datacenter-cool-continuous-stochastic-v1]

If the definition contains inconsistencies, such as mismatched spaces and variables, or non-existent observation variables, Sinergym will throw an error. The definitions for time variables, variables, meters, or actuators must be compatible with the EnergyPlus engine.

You can define the spaces using our environment register too. Refer to the documentation for additional details.

Adding extra configuration

In Sinergym, any argument of the environment constructor can be updated. This allows for modifications such as changing the environment name, adjusting the maximum number of episodes stored in its output, or altering the action normalization range.

env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1',
               env_name='new_env_name',
               max_ep_data_store_num=20
               )

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: new_env_name
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-new_env_name-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.

You can also provide a dictionary with additional parameters to update the building model directly with new simulation-related information.

extra_conf = {
    'timesteps_per_hour': 6,
    'runperiod': (1, 1, 1991, 2, 1, 1991),
}

env = gym.make('Eplus-datacenter-cool-continuous-stochastic-v1',
               config_params=extra_conf
               )

env.reset()
env.close()

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: datacenter-cool-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-datacenter-cool-continuous-stochastic-v1-res2
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Extra config: runperiod updated to {'apply_weekend_holiday_rule': 'No', 'begin_day_of_month': 1, 'begin_month': 1, 'begin_year': 1991, 'day_of_week_for_start_day': 'Tuesday', 'end_day_of_month': 2, 'end_month': 1, 'end_year': 1991, 'use_weather_file_daylight_saving_period': 'Yes', 'use_weather_file_holidays_and_special_days': 'Yes', 'use_weather_file_rain_indicators': 'Yes', 'use_weather_file_snow_indicators': 'Yes'}
[MODELING] (INFO) : Updated episode length (seconds): 172800.0
[MODELING] (INFO) : Updated timestep size (seconds): 600.0
[MODELING] (INFO) : Updated timesteps per episode: 288
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 172800.0
[MODELING] (INFO) : timestep size (seconds): 600.0
[MODELING] (INFO) : timesteps per episode: 288
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.
#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode.
[ENVIRONMENT] (INFO) : Episode 1: datacenter-cool-continuous-stochastic-v1
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created.
[MODELING] (INFO) : Weather file USA_WA_Port.Angeles-William.R.Fairchild.Intl.AP.727885_TMY3.epw used.
[MODELING] (INFO) : Adapting weather to building model.
[ENVIRONMENT] (INFO) : Saving episode output path.
[ENVIRONMENT] (INFO) : Episode 1 started.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[ENVIRONMENT] (INFO) : Environment closed. [datacenter-cool-continuous-stochastic-v1]
Simulation Progress [Episode 1]:  51%|█████     | 51/100 [00:00<00:00, 1534.34%/s, 51% completed]

For more information about extra configuration parameters, see the corresponding documentation.