24. Logging unused variables

The system for creating a custom logger wrapper in Sinergym allows you to record custom metrics based on any interaction data with the environment (see example). However, there may be cases where you want to monitor data directly from the simulator.

Until version v3.3.2 of Sinergym, all variables monitored using the LoggerWrapper had to be present in the observation space. This presents a drawback when we want to monitor certain aspects of the simulation that are not used in the optimization process (in other words, that are not present in the environment’s observation space), it would be impossible.

Including extra variables that are not directly part of the observation space requires certain internal changes that break the minimalist structure of the classes and EnergyPlus API usage that make up the tool.

This notebook explains the correct way to do it, which is available from Sinergym v3.3.3. It involves the use of ReduceObservationWrapper in combination with LoggerWrapper.

The idea is to define all the variables we want, whether they are part of the final observation space or not, and monitor everything with the LoggerWrapper, to later add a layer that removes the desired variables from the observation space (when they would already be monitored, which is our goal).

[1]:

import gymnasium as gym
import numpy as np

import sinergym
from sinergym.utils.wrappers import (
    LoggerWrapper,
    NormalizeAction,
    NormalizeObservation,
    ReduceObservationWrapper,
    CSVLogger,
    WandBLogger)

# Creating environment and applying wrappers for normalization and logging
env = gym.make('Eplus-5zone-hot-continuous-stochastic-v1')
env = NormalizeAction(env)
env = NormalizeObservation(env)
env = LoggerWrapper(env)
print('###########################################################################')
print('Old observation space shape: ',env.observation_space.shape[0])
print('Old observation variables: ',env.get_wrapper_attr('observation_variables'))
print('###########################################################################')

# Here we could combine with wrappers logger outputs such as CSVLogger or WandBLogger
env = CSVLogger(env)
env = WandBLogger(env,
                  entity = 'sail_ugr',
                  project_name = 'test-porject',
                  run_name = 'test-reduction')

env = ReduceObservationWrapper(env, obs_reduction=['outdoor_temperature','outdoor_humidity','air_temperature'])
print('###########################################################################')
print('Wrapped observation space shape: ',env.observation_space.shape[0])
print('Wrapped observation variables: ',env.get_wrapper_attr('observation_variables'))
print('###########################################################################')

#==============================================================================================#
[ENVIRONMENT] (INFO) : Creating Gymnasium environment... [5zone-hot-continuous-stochastic-v1]
#==============================================================================================#
[MODELING] (INFO) : Experiment working directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1]
[MODELING] (INFO) : Model Config is correct.
[MODELING] (INFO) : Updated building model with whole Output:Variable available names
[MODELING] (INFO) : Updated building model with whole Output:Meter available names
[MODELING] (INFO) : runperiod established: {'start_day': 1, 'start_month': 1, 'start_year': 1991, 'end_day': 31, 'end_month': 12, 'end_year': 1991, 'start_weekday': 0, 'n_steps_per_hour': 4}
[MODELING] (INFO) : Episode length (seconds): 31536000.0
[MODELING] (INFO) : timestep size (seconds): 900.0
[MODELING] (INFO) : timesteps per episode: 35041
[REWARD] (INFO) : Reward function initialized.
[ENVIRONMENT] (INFO) : Environment 5zone-hot-continuous-stochastic-v1 created successfully.
[WRAPPER NormalizeAction] (INFO) : New normalized action Space: Box(-1.0, 1.0, (2,), float32)
[WRAPPER NormalizeAction] (INFO) : Wrapper initialized
[WRAPPER NormalizeObservation] (INFO) : Wrapper initialized.
[WRAPPER LoggerWrapper] (INFO) : Wrapper initialized.
###########################################################################
Old observation space shape:  17
Old observation variables:  ['month', 'day_of_month', 'hour', 'outdoor_temperature', 'outdoor_humidity', 'wind_speed', 'wind_direction', 'diffuse_solar_radiation', 'direct_solar_radiation', 'htg_setpoint', 'clg_setpoint', 'air_temperature', 'air_humidity', 'people_occupant', 'co2_emission', 'HVAC_electricity_demand_rate', 'total_electricity_HVAC']
###########################################################################
[WRAPPER CSVLogger] (INFO) : Wrapper initialized.

Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving.
wandb: Using wandb-core as the SDK backend. Please refer to https://wandb.me/wandb-core for more information.
wandb: Currently logged in as: alex_ugr (sail_ugr). Use `wandb login --relogin` to force relogin
Tracking run with wandb version 0.17.5
Run data is saved locally in /workspaces/sinergym/examples/wandb/run-20240821_113506-p2d9p07z
Syncing run test-reduction to Weights & Biases (docs)
View project at https://wandb.ai/sail_ugr/test-porject
View run at https://wandb.ai/sail_ugr/test-porject/runs/p2d9p07z
[WRAPPER WandBLogger] (INFO) : Wrapper initialized.
[WRAPPER ReduceObservationWrapper] (INFO) : Wrapper initialized.
###########################################################################
Wrapped observation space shape:  14
Wrapped observation variables:  ['month', 'day_of_month', 'hour', 'wind_speed', 'wind_direction', 'diffuse_solar_radiation', 'direct_solar_radiation', 'htg_setpoint', 'clg_setpoint', 'air_humidity', 'people_occupant', 'co2_emission', 'HVAC_electricity_demand_rate', 'total_electricity_HVAC']
###########################################################################

The order of the wrappers is important. By applying normalization first, for example, we ensure that this transformation is subsequently monitored. As we apply the Logger before reducing the observation space, we also record these variables before they are removed from the observations. If we left the logger for the end, these variables would not be recorded. This can be verified by reviewing the generated CSV files about observation values.

Let’s review the info dictionary to see these normalized variables that are no longer in obs:

[2]:

obs, info = env.reset()
print('###########################################################################')
print('Reset observation length: ',len(obs))
print('Removed variables info: ',info['removed_observation'])
print('###########################################################################')

a = env.action_space.sample()
obs, _, _, _, info = env.step(a)
print('###########################################################################')
print('step observation length: ',len(obs))
print('Removed variables info: ',info['removed_observation'])
print('###########################################################################')

terminated = truncated = False
while not (terminated or truncated):
    a = env.action_space.sample()
    _, _, terminated, truncated, _ = env.step(a)

#----------------------------------------------------------------------------------------------#
[ENVIRONMENT] (INFO) : Starting a new episode... [5zone-hot-continuous-stochastic-v1] [Episode 1]
#----------------------------------------------------------------------------------------------#
[MODELING] (INFO) : Episode directory created [/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1/Eplus-env-sub_run1]
[MODELING] (INFO) : Weather file USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw used.
[MODELING] (INFO) : Adapting weather to building model. [USA_AZ_Davis-Monthan.AFB.722745_TMY3.epw]
[ENVIRONMENT] (INFO) : Saving episode output path... [/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1/Eplus-env-sub_run1/output]
[SIMULATOR] (INFO) : Running EnergyPlus with args: ['-w', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1/Eplus-env-sub_run1/USA_AZ_Davis-Monthan.AFB.722745_TMY3_Random_1.0_0.0_0.001.epw', '-d', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1/Eplus-env-sub_run1/output', '/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1/Eplus-env-sub_run1/5ZoneAutoDXVAV.epJSON']
[ENVIRONMENT] (INFO) : Episode 1 started.
[SIMULATOR] (INFO) : handlers initialized.
[SIMULATOR] (INFO) : handlers are ready.
[SIMULATOR] (INFO) : System is ready.
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [5zone-hot-continuous-stochastic-v1]
###########################################################################
Reset observation length:  14
Removed variables info:  {'outdoor_temperature': np.float64(0.009694040064036152), 'outdoor_humidity': np.float64(0.009998816541465044), 'air_temperature': np.float64(0.00998743390464178)}
###########################################################################
###########################################################################
step observation length:  14
Removed variables info:  {'outdoor_temperature': np.float64(0.994618545162255), 'outdoor_humidity': np.float64(-0.9745626607331893), 'air_temperature': np.float64(-0.4008530311983138)}
###########################################################################
Progress: |***************************************************************************************************| 99%

Note that even if we remove a variable that is used in the reward function, as this value is used in the core of the environment (before any wrapper), it works perfectly.

[3]:

env.close()

[WRAPPER WandBLogger] (INFO) : Environment closed, dumping summary metrics in WandB Platform.

wandb: Adding directory to artifact (/workspaces/sinergym/examples/Eplus-env-5zone-hot-continuous-stochastic-v1-res1)... Done. 0.0s

Run history:


Agent_actions/Cooling_Setpoint_RL▂▄▁▇▅█▁▂▃▅▅▅█▄▇▅▇▆▇▁█▃▇▃▇▇▆█▄█▄▄█▇▃
Agent_actions/Heating_Setpoint_RL▃▃▄▃█▂▄▃▃▆▃▂▇▅▁█▁█▅▅▂▇▁█▇▅▅▇▇▃▇▇▄▅▃
Info/abs_comfort_penalty▅██▂█▃██▂█▁█▇█▆█▇██▇██▅█▇███████▂█▇
Info/abs_energy_penalty█████▆██▅█▆▇▁▆█▆▇▇▇▆▄▇▇▇▆▇█▆▆█████▃
Info/comfort_term▅██▂█▃██▂█▁█▇█▆█▇██▇██▅█▇███████▂█▇
Info/energy_term█████▆██▅█▆▇▁▆█▆▇▇▇▆▄▇▇▇▆▇█▆▆█████▃
Info/total_power_demand▁▁▁▁▁▃▁▁▄▁▃▂█▃▁▃▂▂▂▃▅▂▂▂▃▂▁▃▃▁▁▁▁▁▆
Info/total_temperature_violation▄▁▁▇▁▆▁▁▇▁█▁▂▁▃▁▂▁▁▂▁▁▄▁▂▁▁▁▁▁▁▁▇▁▂
Normalized_observations/HVAC_electricity_demand_rate▁▁▁▁▁▃▁▂▄▂▃▂█▃▂▃▂▂▂▃▅▂▂▂▃▂▁▃▃▂▁▁▁▁▆
Normalized_observations/air_humidity▆▆▅█▆▅▅▃▄▁▄▁▃▁▃▁▄▂▇▂▇▂▆▂▄▂▅▃▅▃▄▃▄▄▃
Normalized_observations/air_temperature▆▄▂█▂▇▃▂█▃█▄▅▄▃▅▃▅▄▃▆▃█▄▆▄▂▄▂▃▃▁▇▁▅
Normalized_observations/clg_setpoint▆▁█▆▂▅▅▅▆▇▆▃▂▁▄▅▆▇▅▄▄██▆▄▇▃▃▄▇▂▆█▆▂
Normalized_observations/co2_emission▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
Normalized_observations/day_of_month██▁▄▆▂▄▇▂▅▇▂▅▇▂▅▇▂▅▇▂▅▇▂▅▇▃▅█▃▅█▃▆█
Normalized_observations/diffuse_solar_radiation▄▁▁▄▁▅▁▃▂▁█▁▃▁▂▃▁▆▁▂▂▁▄▁▃▁▁▃▁▂▁▁▁▁▃
Normalized_observations/direct_solar_radiation▆▁▁▁▁▇▁▆▄▁▄▁▇▁▃▆▁▅▁▆▃▁▁▁▇▁▁▅▁█▁▅▁▁▇
Normalized_observations/hour▄▇▂▅▁▄▇▃▆▂▅█▄▇▂▅▁▄▇▃▆▂▅█▄▇▂▅▁▄▇▃▆▂▅
Normalized_observations/htg_setpoint▄▂▂▃▂▁▃▂▄▁▇▂▆▃▆▁▁▁█▇█▅▃▆▆█▂▄▆▄▇▅▃▂▇
Normalized_observations/month▁▁█▂▂▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂
Normalized_observations/outdoor_humidity▆▆██▆▄▅▂▁▃▂▂▂▂▄▂▃▂█▆▄▅▃▆▂▃▆▂▆▂▄▄▃▅▂
Normalized_observations/outdoor_temperature▄▃▁▄▃▆▃▃█▆▇▅▆▆▄█▅▆▄▄▆▄▆▄▅▅▂▅▂▄▃▁▃▁▃
Normalized_observations/people_occupant█▂▁█▁▇▁▁█▁▇▁█▁▁▁▁▁▁▁█▁▇▁█▁▁▁▁▁▁▁█▁▇
Normalized_observations/total_electricity_HVAC▁▁▁▁▁▃▁▂▄▂▃▂█▄▂▃▂▂▂▃▆▂▂▂▃▂▁▃▂▁▁▁▁▁▆
Normalized_observations/wind_direction▄▇▄▃▄▂▅▇▇▄▆▇▅▆▄▇▃█▃▃▁▃▇▃▃▇▃▅▄▄▃▄█▄▆
Normalized_observations/wind_speed▂▂▄▃▄▅▃▂▅▂▃▄▂▅▂▅▃▃▄▄▂▃█▃█▃▁▅▂▁▄▄▁▁▂
Observations/HVAC_electricity_demand_rate▁▁▁▁▁▃▁▁▄▁▃▂█▃▁▃▂▂▂▃▅▂▂▂▃▂▁▃▃▁▁▁▁▁▆
Observations/air_humidity▇▆▅██▆▇▄▅▁▅▁▃▁▃▁▄▁▇▂█▂▇▂▄▃▅▃▅▃▅▃▄▄▃
Observations/air_temperature▅▄▁▇▁▇▃▁▇▃█▃▅▄▂▅▃▄▃▃▆▃█▄▆▄▂▄▂▃▃▁▇▁▅
Observations/clg_setpoint▆▁█▆▂▅▅▅▆▇▆▃▂▁▄▅▆▇▅▄▄██▆▄▇▃▃▄▇▂▆█▆▂
Observations/co2_emission▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
Observations/day_of_month▃▆▁▃▆▂▄▇▂▄▇▂▅▇▂▅▇▂▅▇▂▅▇▂▅▇▃▅█▃▅█▃▆█
Observations/diffuse_solar_radiation▃▁▁▃▁▄▁▂▂▁█▁▃▁▁▃▁▆▁▂▂▁▄▁▄▁▁▃▁▂▁▁▁▁▃
Observations/direct_solar_radiation▆▁▁▁▁▆▁▅▄▁▄▁▇▁▃▆▁▆▁▆▃▁▁▁▇▁▁▅▁█▁▅▁▁▇
Observations/hour▄▇▂▅▁▄▇▃▆▂▅█▄▇▂▅▁▄▇▃▆▂▅█▄▇▂▅▁▄▇▃▆▂▅
Observations/htg_setpoint▄▂▂▃▂▁▃▂▄▁▇▂▆▃▆▁▁▁█▇█▅▃▆▆█▂▄▆▄▇▅▃▂▇
Observations/month▁▁▂▂▂▂▂▂▃▃▃▄▄▄▄▄▄▅▅▅▅▅▅▆▆▆▇▇▇▇▇▇███
Observations/outdoor_humidity▇▇███▅▇▃▁▄▂▂▁▂▃▁▂▁▇▅▃▄▃▅▂▂▅▂▅▂▃▄▂▅▂
Observations/outdoor_temperature▃▂▁▃▂▄▂▂▆▅▆▅▆▆▄█▅▇▅▅▇▅▇▅▆▆▃▅▂▅▃▁▃▁▄
Observations/people_occupant█▂▁█▁▇▁▁█▁▇▁█▁▁▁▁▁▁▁█▁▇▁█▁▁▁▁▁▁▁█▁▇
Observations/total_electricity_HVAC▁▁▁▁▁▂▁▁▃▁▃▁█▄▁▃▂▂▂▃▆▁▂▂▃▂▁▃▂▁▁▁▁▁▇
Observations/wind_direction▃▆▃▃▃▁▅▆▆▄▆▆▅▆▄▇▃█▃▃▁▃▇▄▃▇▄▅▄▄▃▄█▄▆
Observations/wind_speed▂▂▃▃▃▄▂▂▄▂▃▄▂▅▂▅▃▃▃▃▂▃█▃█▃▁▅▂▁▄▃▁▁▂
Reward/reward▅██▂█▃██▂█▁█▆█▆█▇██▇▇█▅█▇██▇████▂█▆
Simulation_actions/Cooling_Setpoint_RL▂▄▁▇▅█▁▂▃▅▅▅█▄▇▅▇▆▇▁█▃▇▃▇▇▆█▄█▄▄█▇▃
Simulation_actions/Heating_Setpoint_RL▃▃▄▃█▂▄▃▃▆▃▂▇▅▁█▁█▅▅▂▇▁█▇▅▅▇▇▃▇▇▄▅▃
episode_summaries/comfort_violation_time(%)▁
episode_summaries/cumulative_power_demand▁
episode_summaries/episode_num▁
episode_summaries/length(timesteps)▁
episode_summaries/mean_abs_comfort_penalty▁
episode_summaries/mean_abs_energy_penalty▁
episode_summaries/mean_power_demand▁
episode_summaries/mean_reward▁
episode_summaries/mean_reward_comfort_term▁
episode_summaries/mean_reward_energy_term▁
episode_summaries/mean_temperature_violation▁
episode_summaries/std_abs_comfort_penalty▁
episode_summaries/std_abs_energy_penalty▁
episode_summaries/std_power_demand▁
episode_summaries/std_reward▁
episode_summaries/std_reward_comfort_term▁
episode_summaries/std_reward_energy_term▁
episode_summaries/std_temperature_violation▁
episode_summaries/time_elapsed(hours)▁

Run summary:


Agent_actions/Cooling_Setpoint_RL-0.4778
Agent_actions/Heating_Setpoint_RL-0.25449
Info/abs_comfort_penalty-0.8399
Info/abs_energy_penalty-5115.03133
Info/comfort_term-0.41995
Info/energy_term-0.25575
Info/total_power_demand5115.03133
Info/total_temperature_violation0.8399
Normalized_observations/HVAC_electricity_demand_rate1.32263
Normalized_observations/air_humidity-0.97818
Normalized_observations/air_temperature0.40859
Normalized_observations/clg_setpoint-1.16473
Normalized_observations/co2_emission0
Normalized_observations/day_of_month1.74104
Normalized_observations/diffuse_solar_radiation0.29647
Normalized_observations/direct_solar_radiation1.75984
Normalized_observations/hour0.21787
Normalized_observations/htg_setpoint0.98495
Normalized_observations/month1.59085
Normalized_observations/outdoor_humidity-0.96832
Normalized_observations/outdoor_temperature-0.27763
Normalized_observations/people_occupant1.21883
Normalized_observations/total_electricity_HVAC1.4688
Normalized_observations/wind_direction0.61943
Normalized_observations/wind_speed-1.03508
Observations/HVAC_electricity_demand_rate5115.03125
Observations/air_humidity16.681
Observations/air_temperature24.3399
Observations/clg_setpoint24.3404
Observations/co2_emission0
Observations/day_of_month31
Observations/diffuse_solar_radiation86.5
Observations/direct_solar_radiation898
Observations/hour13
Observations/htg_setpoint20.82611
Observations/month12
Observations/outdoor_humidity12
Observations/outdoor_temperature19.19061
Observations/people_occupant16
Observations/total_electricity_HVAC5357560.5
Observations/wind_direction240
Observations/wind_speed1.5
Reward/reward-0.6757
Simulation_actions/Cooling_Setpoint_RL25.01243
Simulation_actions/Heating_Setpoint_RL16.19347
episode_summaries/comfort_violation_time(%)47.14041
episode_summaries/cumulative_power_demand64705432.84029
episode_summaries/episode_num1
episode_summaries/length(timesteps)35041
episode_summaries/mean_abs_comfort_penalty-0.73257
episode_summaries/mean_abs_energy_penalty-1846.61623
episode_summaries/mean_power_demand1846.61623
episode_summaries/mean_reward-0.45862
episode_summaries/mean_reward_comfort_term-0.36629
episode_summaries/mean_reward_energy_term-0.09233
episode_summaries/mean_temperature_violation0.73257
episode_summaries/std_abs_comfort_penalty1.20454
episode_summaries/std_abs_energy_penalty2470.25417
episode_summaries/std_power_demand2470.25417
episode_summaries/std_reward0.60726
episode_summaries/std_reward_comfort_term0.60227
episode_summaries/std_reward_energy_term0.12351
episode_summaries/std_temperature_violation1.20454
episode_summaries/time_elapsed(hours)8760.125

View run test-reduction at: https://wandb.ai/sail_ugr/test-porject/runs/p2d9p07z
View project at: https://wandb.ai/sail_ugr/test-porject
Synced 5 W&B file(s), 0 media file(s), 21 artifact file(s) and 0 other file(s)
Find logs at: ./wandb/run-20240821_113506-p2d9p07z/logs
[WRAPPER CSVLogger] (INFO) : Environment closed, data updated in monitor and progress.csv.
[ENVIRONMENT] (INFO) : Environment closed. [5zone-hot-continuous-stochastic-v1]
[WRAPPER NormalizeObservation] (INFO) : Saving normalization calibration data... [5zone-hot-continuous-stochastic-v1]