Logging unused variables

Custom logger wrappers in Sinergym enable recording any metric based on interaction data with the environment (as shown here). However, there may be cases where you want to monitor data directly from the simulator.

Until version v3.3.2 of Sinergym, all variables monitored using the LoggerWrapper had to be present in the observation space. This represents a drawback when we want to monitor certain variables of the simulation that are not used for control (i.e., that are not present in the environment’s observation space).

Including extra variables that are not directly part of the observation space requires certain internal changes that break the minimalist structure of the classes and EnergyPlus API usage that make up the tool.

This notebook explains the correct way to do it, which is possible since Sinergym v3.3.3. It involves the use of the ReduceObservationWrapper in combination with the LoggerWrapper.

The idea is to define all the variables to be observed, whether they are part of the final observation space or not, and monitor everything with the LoggerWrapper, to later add a layer that removes the desired variables from the observation space (just when they are already being monitored, which is our goal).

[ ]:

import gymnasium as gym
import numpy as np

import sinergym
from sinergym.utils.wrappers import (
    LoggerWrapper,
    NormalizeAction,
    NormalizeObservation,
    ReduceObservationWrapper,
    CSVLogger,
    WandBLogger)

# Creating environment and applying wrappers for normalization and logging
env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')
env = NormalizeAction(env)
env = NormalizeObservation(env)
env = LoggerWrapper(env)
print('###########################################################################')
print('Old observation space shape: ', env.observation_space.shape[0])
print('Old observation variables: ',
      env.get_wrapper_attr('observation_variables'))
print('###########################################################################')

# Here we can add the CSVLogger or WandBLogger
env = CSVLogger(env)
env = WandBLogger(env,
                  entity='sail_ugr',
                  project_name='test-porject',
                  run_name='test-reduction')

env = ReduceObservationWrapper(env, obs_reduction=[
                               'outdoor_temperature', 'outdoor_humidity', 'air_temperature'])
print('###########################################################################')
print('Wrapped observation space shape: ', env.observation_space.shape[0])
print('Wrapped observation variables: ',
      env.get_wrapper_attr('observation_variables'))
print('###########################################################################')

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment.
[ENVIRONMENT] (INFO) : Name: 5zone-hot-continuous-stochastic-v1
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created.
[MODELING] (INFO) : Working directory: /workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Update building model Output:Variable with variable names.
[MODELING] (INFO) : Update building model Output:Meter with meter names.
[MODELING] (INFO) : Runperiod established.
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35040
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment created successfully.
[WRAPPER NormalizeAction] (INFO) : New normalized action Space: Box(-1.0, 1.0, (2,), float32)
[WRAPPER NormalizeAction] (INFO) : Wrapper initialized
[WRAPPER NormalizeObservation] (INFO) : Wrapper initialized.
[WRAPPER LoggerWrapper] (INFO) : Wrapper initialized.
###########################################################################
Old observation space shape:  17
Old observation variables:  ['month', 'day_of_month', 'hour', 'outdoor_temperature', 'outdoor_humidity', 'wind_speed', 'wind_direction', 'diffuse_solar_radiation', 'direct_solar_radiation', 'htg_setpoint', 'clg_setpoint', 'air_temperature', 'air_humidity', 'people_occupant', 'co2_emission', 'HVAC_electricity_demand_rate', 'total_electricity_HVAC']
###########################################################################
[WRAPPER CSVLogger] (INFO) : Wrapper initialized.

Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving.
wandb: Using wandb-core as the SDK backend. Please refer to https://wandb.me/wandb-core for more information.
wandb: Currently logged in as: alex_ugr (sail_ugr). Use `wandb login --relogin` to force relogin
Tracking run with wandb version 0.17.7
Run data is saved locally in /workspaces/sinergym/examples/wandb/run-20240910_074919-rl1s4e17
Syncing run test-reduction to Weights & Biases (docs)
View project at https://wandb.ai/sail_ugr/test-porject
View run at https://wandb.ai/sail_ugr/test-porject/runs/rl1s4e17
[WRAPPER WandBLogger] (INFO) : Wrapper initialized.
[WRAPPER ReduceObservationWrapper] (INFO) : Wrapper initialized.
###########################################################################
Wrapped observation space shape:  14
Wrapped observation variables:  ['month', 'day_of_month', 'hour', 'wind_speed', 'wind_direction', 'diffuse_solar_radiation', 'direct_solar_radiation', 'htg_setpoint', 'clg_setpoint', 'air_humidity', 'people_occupant', 'co2_emission', 'HVAC_electricity_demand_rate', 'total_electricity_HVAC']
###########################################################################

The order of the wrappers is important! By applying normalization first, for example, we ensure that this transformation is subsequently monitored.

As we apply the logger before reducing the observation space, we also record these variables before they are removed from the observations. If we use the logger wrapper at the end, these variables would not be monitored. This can be verified by reviewing the CSV files generated for the observation variables.

Let’s review the info dictionary to see the normalized variables that are no longer in obs:

[ ]:

obs, info = env.reset()
print('###########################################################################')
print('Reset observation length: ', len(obs))
print('Removed variables info: ', info['removed_observation'])
print('###########################################################################')

a = env.action_space.sample()
obs, _, _, _, info = env.step(a)
print('###########################################################################')
print('step observation length: ', len(obs))
print('Removed variables info: ', info['removed_observation'])
print('###########################################################################')

terminated = truncated = False
while not (terminated or truncated):
    a = env.action_space.sample()
    _, _, terminated, truncated, _ = env.step(a)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode.
[ENVIRONMENT] (INFO) : Episode 1: 5zone-hot-continuous-stochastic-v1
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created.
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Adapting weather to building model.
[MODELING] (INFO) : Weather noise applied in columns: ['drybulb']
[ENVIRONMENT] (INFO) : Saving episode output path.
[ENVIRONMENT] (INFO) : Episode 1 started.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data.
###########################################################################
Reset observation length:  14
Removed variables info:  {'outdoor_temperature': np.float64(0.009759825889935545), 'outdoor_humidity': np.float64(0.009998816541465044), 'air_temperature': np.float64(0.009987481748785842)}
###########################################################################
###########################################################################
step observation length:  14
Removed variables info:  {'outdoor_temperature': np.float64(0.9852854070847759), 'outdoor_humidity': np.float64(-0.9745626607331893), 'air_temperature': np.float64(-0.44056459754655924)}
###########################################################################
Simulation Progress [Episode 1]: 100%|██████████| 100/100 [00:14<00:00,  6.90%/s, 100% completed]

Note that even if we remove a variable that is used in the reward function, as this value is used in the core of the environment (before any wrapper), it still works as expected.

[3]:

env.close()

[WRAPPER WandBLogger] (INFO) : Environment closed, dumping summary metrics in WandB Platform.
Simulation Progress [Episode 1]: 100%|██████████| 100/100 [00:16<00:00,  6.90%/s, 100% completed]

wandb: Adding directory to artifact (/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1)... Done. 0.0s

Run history:


Agent_actions/Cooling_Setpoint_RL▃▆▇▇▄▁█▄▄▃▂▇▅█▇▃▄█▇▄▅▅█▅█▄█▂▃▂█▂▄▃▃
Agent_actions/Heating_Setpoint_RL▆█▁█▂▁▃▃█▆█▂▅▆▃▂▇▆▃▅▅▅▇▄▃▆▇▃▄▇▃▇▆▃▃
Info/abs_comfort_penalty▃▇█▃█▇██▃█▅█▁█▆█▇██▇▇█▃█▄██▇████▄█▆
Info/abs_energy_penalty██▁██▆██▇█▃█▆▇█▂█▇▇▆▇█▇█▇██▆██████▅
Info/comfort_term▃▇█▃█▇██▃█▅█▁█▆█▇██▇▇█▃█▄██▇████▄█▆
Info/energy_term██▁██▆██▇█▃█▆▇█▂█▇▇▆▇█▇█▇██▆██████▅
Info/total_power_demand▁▁█▁▁▃▁▁▂▁▆▁▃▂▁▇▁▂▂▃▂▁▂▁▂▁▁▃▁▁▁▁▁▁▄
Info/total_temperature_violation▆▂▁▆▁▂▁▁▆▁▄▁█▁▃▁▂▁▁▂▂▁▆▁▅▁▁▂▁▁▁▁▅▁▃
Normalized_observations/HVAC_electricity_demand_rate▁▁█▁▁▃▁▂▂▂▇▂▃▂▁█▂▂▂▃▂▁▂▁▂▁▁▃▁▁▁▁▁▁▅
Normalized_observations/air_humidity▆▆▄█▇▅▆▃▄▁▅▁▃▁▃▂▅▂▆▂▇▂▆▂▄▃▅▃▅▃▅▃▃▃▄
Normalized_observations/air_temperature▆▄▂▇▂▅▃▃▆▃▅▄▇▄▂▄▃▄▃▃▆▃█▃█▃▁▄▂▃▃▁▅▁▅
Normalized_observations/clg_setpoint▇▄█▆▄▂▄▇▇▂▃█▆▃▇▁▆▅▄▂██▇▃█▇▆▃▁▇▆▆▇▇▃
Normalized_observations/co2_emission▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
Normalized_observations/day_of_month██▁▄▆▂▄▇▂▅▇▂▅▇▂▅▇▂▅▇▂▅▇▃▅▇▃▅█▃▅█▃▆█
Normalized_observations/diffuse_solar_radiation▄▁▁▅▁▆▁▃▂▁█▁▄▁▂▃▁▇▁▃▂▁▄▁▄▁▁▃▁▂▁▂▁▁▃
Normalized_observations/direct_solar_radiation▆▁▁▂▁▇▁▆▄▁▅▁▇▁▄▆▁▅▁▆▂▁▁▁▆▁▁▄▁█▁▅▁▁▇
Normalized_observations/hour▄▇▃▆▂▅█▄▇▂▅▁▄▇▃▆▂▅█▄▇▂▅▁▄▇▃▆▂▅█▄▇▂▅
Normalized_observations/htg_setpoint▃▂█▄▁▂▇▂▅▆▂▅▂▄▇▆▄▇▄▅█▃▂▄▂▇▃▇▆▅▂▃▃▂▃
Normalized_observations/month▁▁█▂▂▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂
Normalized_observations/outdoor_humidity▆▆██▇▄▆▂▁▃▂▂▂▂▃▂▃▂█▆▄▅▃▆▂▃▆▂▆▂▄▄▃▅▂
Normalized_observations/outdoor_temperature▃▃▂▄▃▅▄▂█▅▇▆▇▆▅█▅▆▄▅▆▄▆▅▅▅▂▄▂▄▃▂▃▁▄
Normalized_observations/people_occupant█▂▁█▁▄▁█▄▁█▁█▁▁▁▁▁▁▁▄▁█▁█▁▁▁▁▁▁▁▅▁█
Normalized_observations/total_electricity_HVAC▁▁█▁▁▃▁▁▂▂▇▂▃▂▁█▁▂▂▃▂▁▂▁▂▁▁▃▁▁▁▁▁▁▅
Normalized_observations/wind_direction▄▆▄▃▄█▄▆▆▃▆▆▅▆▃▆▃▇▃▃▁▃▇▃▃▇▃▅▃▄▃▄█▃▆
Normalized_observations/wind_speed▂▂▄▃▃▅▃▂▅▃▄▄▂▅▃▆▃▃▄▄▃▃▇▃█▃▁▅▂▂▄▄▁▁▂
Observations/HVAC_electricity_demand_rate▁▁█▁▁▃▁▁▂▁▆▁▃▂▁▇▁▂▂▃▂▁▂▁▂▁▁▃▁▁▁▁▁▁▄
Observations/air_humidity▇▆▅██▆▇▄▅▁▅▁▃▁▃▁▅▁▇▂▇▂▆▂▄▂▅▃▅▃▅▃▃▃▄
Observations/air_temperature▆▄▂▆▁▄▃▂▆▂▅▃▇▄▂▄▃▄▃▃▆▃█▃█▃▁▄▂▃▃▁▅▁▅
Observations/clg_setpoint▇▄█▆▄▂▄▇▆▂▃█▆▃▇▁▆▅▄▂▇█▇▃█▇▆▃▁▇▆▆▇▇▃
Observations/co2_emission▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
Observations/day_of_month▃▆▁▃▆▂▄▇▂▄▇▂▅▇▂▅▇▂▅▇▂▅▇▃▅▇▃▅█▃▅█▃▆█
Observations/diffuse_solar_radiation▃▁▁▄▁▅▁▃▂▁█▁▄▁▂▃▁▇▁▃▂▁▄▁▄▁▁▃▁▂▁▂▁▁▃
Observations/direct_solar_radiation▆▁▁▁▁▆▁▆▃▁▄▁▇▁▄▆▁▆▁▆▂▁▁▁▇▁▁▄▁█▁▅▁▁▇
Observations/hour▄▇▃▆▂▅█▄▇▂▅▁▄▇▃▆▂▅█▄▇▂▅▁▄▇▃▆▂▅█▄▇▂▅
Observations/htg_setpoint▃▂█▄▁▂▇▂▅▆▂▅▂▄▇▆▄▇▄▅█▃▂▄▂▇▃▆▆▅▂▃▃▂▃
Observations/month▁▁▂▂▂▂▂▂▃▃▃▄▄▄▄▄▄▅▅▅▅▅▅▆▆▆▇▇▇▇▇▇███
Observations/outdoor_humidity▆▇███▅▇▃▁▄▂▂▁▂▃▁▂▁▇▅▃▄▃▅▂▂▅▁▅▁▃▃▂▄▂
Observations/outdoor_temperature▂▂▁▃▂▃▂▁▆▄▆▅▆▅▅█▅▇▄▆▇▅▇▅▆▆▂▅▂▅▃▂▃▁▄
Observations/people_occupant█▂▁█▁▄▁█▅▁█▁█▁▁▁▁▁▁▁▅▁█▁█▁▁▁▁▁▁▁▅▁█
Observations/total_electricity_HVAC▁▁█▁▁▃▁▁▂▁▇▁▃▂▁█▁▁▂▃▂▁▂▁▂▁▁▃▁▁▁▁▁▁▅
Observations/wind_direction▃▆▃▃▃█▄▆▆▃▆▆▅▆▃▆▃▇▃▃▁▃▇▄▃█▃▅▃▄▃▄█▄▆
Observations/wind_speed▂▂▄▃▃▄▂▂▄▂▄▄▂▅▃▆▃▃▄▄▃▃▇▃█▃▁▅▂▂▄▃▁▁▂
Reward/reward▄▇▆▃█▇██▃█▄█▁█▆▇▇██▇▇█▃█▄██▇████▅█▅
Simulation_actions/Cooling_Setpoint_RL▃▆▇▇▄▁█▄▄▃▂▇▅█▇▃▄█▇▄▅▅█▅█▄█▂▃▂█▂▄▃▃
Simulation_actions/Heating_Setpoint_RL▆█▁█▂▁▃▃█▆█▂▅▆▃▂▇▆▃▅▅▅▇▄▃▆▇▃▄▇▃▇▆▃▃
episode_summaries/comfort_violation_time(%)▁
episode_summaries/cumulative_power_demand▁
episode_summaries/episode_num▁
episode_summaries/length(timesteps)▁
episode_summaries/mean_abs_comfort_penalty▁
episode_summaries/mean_abs_energy_penalty▁
episode_summaries/mean_power_demand▁
episode_summaries/mean_reward▁
episode_summaries/mean_reward_comfort_term▁
episode_summaries/mean_reward_energy_term▁
episode_summaries/mean_temperature_violation▁
episode_summaries/std_abs_comfort_penalty▁
episode_summaries/std_abs_energy_penalty▁
episode_summaries/std_power_demand▁
episode_summaries/std_reward▁
episode_summaries/std_reward_comfort_term▁
episode_summaries/std_reward_energy_term▁
episode_summaries/std_temperature_violation▁
episode_summaries/time_elapsed(hours)▁

Run summary:


Agent_actions/Cooling_Setpoint_RL-0.4241
Agent_actions/Heating_Setpoint_RL-0.45542
Info/abs_comfort_penalty-1.32374
Info/abs_energy_penalty-4978.12566
Info/comfort_term-0.66187
Info/energy_term-0.24891
Info/total_power_demand4978.12566
Info/total_temperature_violation1.32374
Normalized_observations/HVAC_electricity_demand_rate1.27458
Normalized_observations/air_humidity-0.41403
Normalized_observations/air_temperature0.63074
Normalized_observations/clg_setpoint-0.92581
Normalized_observations/co2_emission0
Normalized_observations/day_of_month1.74094
Normalized_observations/diffuse_solar_radiation0.40586
Normalized_observations/direct_solar_radiation1.70599
Normalized_observations/hour0.36235
Normalized_observations/htg_setpoint-0.63437
Normalized_observations/month1.59077
Normalized_observations/outdoor_humidity-0.97885
Normalized_observations/outdoor_temperature-0.03853
Normalized_observations/people_occupant1.69558
Normalized_observations/total_electricity_HVAC1.36765
Normalized_observations/wind_direction0.71151
Normalized_observations/wind_speed-1.10769
Observations/HVAC_electricity_demand_rate4978.12549
Observations/air_humidity23.60334
Observations/air_temperature24.82374
Observations/clg_setpoint24.82435
Observations/co2_emission0
Observations/day_of_month31
Observations/diffuse_solar_radiation95.75
Observations/direct_solar_radiation879
Observations/hour14
Observations/htg_setpoint15.57557
Observations/month12
Observations/outdoor_humidity11.75
Observations/outdoor_temperature21.2873
Observations/people_occupant20
Observations/total_electricity_HVAC5090787.5
Observations/wind_direction247.5
Observations/wind_speed1.375
Reward/reward-0.91078
Simulation_actions/Cooling_Setpoint_RL25.19366
Simulation_actions/Heating_Setpoint_RL15.06324
episode_summaries/comfort_violation_time(%)47.27169
episode_summaries/cumulative_power_demand64460871.50538
episode_summaries/episode_num1
episode_summaries/length(timesteps)35040
episode_summaries/mean_abs_comfort_penalty-0.72661
episode_summaries/mean_abs_energy_penalty-1839.63674
episode_summaries/mean_power_demand1839.63674
episode_summaries/mean_reward-0.45529
episode_summaries/mean_reward_comfort_term-0.3633
episode_summaries/mean_reward_energy_term-0.09198
episode_summaries/mean_temperature_violation0.72661
episode_summaries/std_abs_comfort_penalty1.1996
episode_summaries/std_abs_energy_penalty2461.27112
episode_summaries/std_power_demand2461.27112
episode_summaries/std_reward0.60452
episode_summaries/std_reward_comfort_term0.5998
episode_summaries/std_reward_energy_term0.12306
episode_summaries/std_temperature_violation1.1996
episode_summaries/time_elapsed(hours)8760

View run test-reduction at: https://wandb.ai/sail_ugr/test-porject/runs/rl1s4e17
View project at: https://wandb.ai/sail_ugr/test-porject
Synced 5 W&B file(s), 0 media file(s), 12 artifact file(s) and 0 other file(s)
Find logs at: ./wandb/run-20240910_074919-rl1s4e17/logs
[WRAPPER CSVLogger] (INFO) : Environment closed, data updated in monitor and progress.csv.
Simulation Progress [Episode 1]: 100%|██████████| 100/100 [00:29<00:00,  3.37%/s, 100% completed]
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-continuous-stochastic-v1]
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data.

/usr/local/lib/python3.12/dist-packages/wandb/sdk/wandb_run.py:2372: UserWarning: Run (rl1s4e17) is finished. The call to `_console_raw_callback` will be ignored. Please make sure that you are using an active run.
  lambda data: self._console_raw_callback("stdout", data),