18. Wrappers example

We will see on this notebook what are the wrappers defined by Sinergym and how to use them. Currently, we have developed a normalization wrapper, multi-observation wrapper, multi-objective wrapper and Logger wrapper.

[1]:

import gymnasium as gym
import numpy as np
import sinergym
from sinergym.utils.wrappers import *
from sinergym.utils.constants import RANGES_5ZONE

18.1. Normalization Wrapper

[2]:

#Original env
env=gym.make('Eplus-demo-v1')
obs, info=env.reset()
print('BEFORE NORMILIZATION: ',obs)

#Normalize env
env = NormalizeObservation(env,ranges=RANGES_5ZONE)
obs, info=env.reset()
print('AFTER NORMILIZATION: ',obs)
env.close()

[2023-03-10 10:24:47,001] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:24:47,002] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:24:47,004] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:24:47,005] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:24:47,005] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:24:47,007] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:24:47,014] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res10/Eplus-env-sub_run1
BEFORE NORMILIZATION:  [1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04]
[2023-03-10 10:24:51,631] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2023-03-10 10:24:51,632] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:24:51,639] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res10/Eplus-env-sub_run2
AFTER NORMILIZATION:  [0.98466104 0.         0.         0.         0.33834645 0.9484536
 0.17748918 0.8111888  0.         0.         0.800001   0.44444445
 0.39061695 0.         0.43193626 0.         0.         0.
 0.         0.56101173]
[2023-03-10 10:24:56,120] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.

18.2. Multi-Objective Wrapper

MO-Gymnasium is an open source Python library for developing and comparing multi-objective reinforcement learning algorithms. These environments return a reward vector instead of a scalar value; one for each objective.

In order to be more general as possible, it could be interesting that Sinergym would give that reward vector too. In this way, Sinergym would have compatibility with both; multi-objective algorithms and algorithms that work with a traditional reward value.

We can transform reward returned in a vector using the next wrapper:

[3]:

env=gym.make('Eplus-demo-v1')
env=MultiObjectiveReward(env,reward_terms=['reward_energy','reward_comfort'])

[2023-03-10 10:25:04,189] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:04,189] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:04,191] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:04,191] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:04,193] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:04,193] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:04,194] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:04,194] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:04,194] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:04,194] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...

You have to ensure that reward_terms are available in info dict returned in step method of the environment. Otherwise, we will encounter an execution error. By default, Sinergym environments return in info dict all reward terms specified in reward class used, so if the objective exists in reward term you shouldn’t have any problem.

[4]:

env.reset()
action = env.action_space.sample()
obs, reward, terminated, truncated, info = env.step(action)
print(reward)

[2023-03-10 10:25:07,915] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:25:07,915] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:25:07,923] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res11/Eplus-env-sub_run1
[2023-03-10 10:25:07,923] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res11/Eplus-env-sub_run1
[-1.8054534463652903, -0.0]

18.3. Multi Observation Wrapper

[5]:

#Original environment
env=gym.make('Eplus-demo-v1')
obs, info=env.reset()
print('BEFORE MULTI OBSERVATION: ',obs)

#Multi Observation environment
env=MultiObsWrapper(env, n=5, flatten=True)
obs, info=env.reset()
print('AFTER MULTI OBSERVATION: ',obs)
env.close()

[2023-03-10 10:25:16,484] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:16,484] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:16,484] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:16,485] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:16,485] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:16,485] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:16,487] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:16,487] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:16,487] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:16,488] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:16,488] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:16,488] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:16,489] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:16,489] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:16,489] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:16,491] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:25:16,491] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:25:16,491] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:25:16,500] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res12/Eplus-env-sub_run1
[2023-03-10 10:25:16,500] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res12/Eplus-env-sub_run1
[2023-03-10 10:25:16,500] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res12/Eplus-env-sub_run1
BEFORE MULTI OBSERVATION:  [1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04]
[2023-03-10 10:25:21,146] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2023-03-10 10:25:21,146] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2023-03-10 10:25:21,146] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2023-03-10 10:25:21,147] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:25:21,147] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:25:21,147] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:25:21,156] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res12/Eplus-env-sub_run2
[2023-03-10 10:25:21,156] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res12/Eplus-env-sub_run2
[2023-03-10 10:25:21,156] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res12/Eplus-env-sub_run2
AFTER MULTI OBSERVATION:  [1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04
 1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04
 1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04
 1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04
 1.9910000e+03 1.0000000e+00 1.0000000e+00 0.0000000e+00 0.0000000e+00
 9.5000000e+01 4.0999999e+00 2.9000000e+02 0.0000000e+00 0.0000000e+00
 2.1000000e+01 2.5000000e+01 2.0999992e+01 0.0000000e+00 3.9707859e+01
 0.0000000e+00 0.0000000e+00 0.0000000e+00 0.0000000e+00 1.8362732e+04]
[2023-03-10 10:25:25,674] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2023-03-10 10:25:25,674] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2023-03-10 10:25:25,674] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.

18.4. Logger Wrapper

[6]:

env=gym.make('Eplus-demo-v1')
env=LoggerWrapper(env)

[2023-03-10 10:25:29,172] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:29,172] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:29,172] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:29,172] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:29,175] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:29,175] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:29,175] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:29,175] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:29,177] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:29,177] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:29,177] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:29,177] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:29,179] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:29,179] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:29,179] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:29,179] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:29,181] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:29,181] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:29,181] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:29,181] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...

Now, you can see in Sinergym output folder that you will have available progress.csv file and monitor.csv files in each episode.

18.5. All Wrappers in one

You can also combine wrappers in order to use all their functionality at the same time.

[7]:

env=gym.make('Eplus-demo-v1')
#Normalization
env=NormalizeObservation(env,ranges=RANGES_5ZONE)
#Logger
env=LoggerWrapper(env)
#Multi_objective
env=MultiObjectiveReward(env, reward_terms=['reward_energy','reward_comfort'])
#MultiObs
env=MultiObsWrapper(env,n=5,flatten=True)

[2023-03-10 10:25:34,260] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:34,260] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:34,260] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:34,260] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:34,260] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf ExternalInterface object if it is not present...
[2023-03-10 10:25:34,262] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:34,262] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:34,262] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:34,262] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:34,262] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf Site:Location and SizingPeriod:DesignDay(s) to weather and ddy file...
[2023-03-10 10:25:34,264] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:34,264] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:34,264] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:34,264] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:34,264] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Updating idf OutPut:Variable and variables XML tree model for BVCTB connection.
[2023-03-10 10:25:34,266] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:34,266] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:34,266] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:34,266] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:34,266] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up extra configuration in building model if exists...
[2023-03-10 10:25:34,267] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:34,267] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:34,267] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:34,267] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...
[2023-03-10 10:25:34,267] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Setting up action definition in building model if exists...

Warning

The order of wrappers if you are going to use several at the same time is really important. The correct order is Normalization - Logger - MultiObs and subsets (for example, Normalization - Multiobs is valid).

Note

For more information about Sinergym Logger, visit Logger documentation and Wrapper documentation

Now we just simply use the environment with the wrappers, for example:

[9]:

for i in range(1):
    obs, info = env.reset()
    terminated = False
    current_month = 0
    while not terminated:
        a = env.action_space.sample()
        obs, reward, terminated, truncated, info = env.step(a)
        if info['month'] != current_month:  # display results every month
            current_month = info['month']
            print('Reward: ', reward, info)
env.close()

/usr/local/lib/python3.10/dist-packages/numpy/core/fromnumeric.py:3464: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
/usr/local/lib/python3.10/dist-packages/numpy/core/_methods.py:192: RuntimeWarning: invalid value encountered in scalar divide
  ret = ret.dtype.type(ret / rcount)

[2023-03-10 10:26:40,178] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2023-03-10 10:26:40,178] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2023-03-10 10:26:40,178] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2023-03-10 10:26:40,178] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2023-03-10 10:26:40,178] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus episode completed successfully.
[2023-03-10 10:26:40,180] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:26:40,180] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:26:40,180] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:26:40,180] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:26:40,180] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:Creating new EnergyPlus simulation episode...
[2023-03-10 10:26:40,188] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res14/Eplus-env-sub_run2
[2023-03-10 10:26:40,188] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res14/Eplus-env-sub_run2
[2023-03-10 10:26:40,188] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res14/Eplus-env-sub_run2
[2023-03-10 10:26:40,188] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res14/Eplus-env-sub_run2
[2023-03-10 10:26:40,188] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus working directory is in /workspaces/sinergym/examples/Eplus-env-demo-v1-res14/Eplus-env-sub_run2
Reward:  [-0.3780170717786078, -0.7607145700276305] {'timestep': 1, 'time_elapsed': 900, 'year': 1991, 'month': 1, 'day': 1, 'hour': 0, 'action': [16, 29], 'reward': -0.5693658209031192, 'reward_energy': -0.3780170717786078, 'reward_comfort': -0.7607145700276305, 'total_energy': 3780.170717786078, 'abs_comfort': 0.7607145700276305, 'temperatures': [19.23928542997237]}
Reward:  [-2.170164060335223, -0.0] {'timestep': 2976, 'time_elapsed': 2678400, 'year': 1991, 'month': 2, 'day': 1, 'hour': 0, 'action': [21, 21], 'reward': -1.0850820301676114, 'reward_energy': -2.170164060335223, 'reward_comfort': -0.0, 'total_energy': 21701.64060335223, 'abs_comfort': 0.0, 'temperatures': [20.81967380155336]}
Reward:  [-0.8365440129567937, -0.03220371403475042] {'timestep': 5664, 'time_elapsed': 5097600, 'year': 1991, 'month': 3, 'day': 1, 'hour': 0, 'action': [20, 25], 'reward': -0.43437386349577206, 'reward_energy': -0.8365440129567937, 'reward_comfort': -0.03220371403475042, 'total_energy': 8365.440129567936, 'abs_comfort': 0.03220371403475042, 'temperatures': [19.96779628596525]}
Reward:  [-0.01524868953414246, -0.6854227056528615] {'timestep': 8640, 'time_elapsed': 7776000, 'year': 1991, 'month': 4, 'day': 1, 'hour': 0, 'action': [18, 27], 'reward': -0.35033569759350197, 'reward_energy': -0.01524868953414246, 'reward_comfort': -0.6854227056528615, 'total_energy': 152.4868953414246, 'abs_comfort': 0.6854227056528615, 'temperatures': [19.31457729434714]}
Reward:  [-0.24955729199812332, -0.0] {'timestep': 11520, 'time_elapsed': 10368000, 'year': 1991, 'month': 5, 'day': 1, 'hour': 0, 'action': [21, 21], 'reward': -0.12477864599906166, 'reward_energy': -0.24955729199812332, 'reward_comfort': -0.0, 'total_energy': 2495.572919981233, 'abs_comfort': 0.0, 'temperatures': [21.02192987873644]}
Reward:  [-0.7209530243624731, -1.00504857474834] {'timestep': 14496, 'time_elapsed': 13046400, 'year': 1991, 'month': 6, 'day': 1, 'hour': 0, 'action': [22, 23], 'reward': -0.8630007995554065, 'reward_energy': -0.7209530243624731, 'reward_comfort': -1.00504857474834, 'total_energy': 7209.530243624731, 'abs_comfort': 1.00504857474834, 'temperatures': [21.99495142525166]}
Reward:  [-0.017578665657165853, -1.7741709350940091] {'timestep': 17376, 'time_elapsed': 15638400, 'year': 1991, 'month': 7, 'day': 1, 'hour': 0, 'action': [17, 28], 'reward': -0.8958748003755875, 'reward_energy': -0.017578665657165853, 'reward_comfort': -1.7741709350940091, 'total_energy': 175.7866565716585, 'abs_comfort': 1.7741709350940091, 'temperatures': [21.22582906490599]}
Reward:  [-1.636839603914276, -2.0794240943146285] {'timestep': 20352, 'time_elapsed': 18316800, 'year': 1991, 'month': 8, 'day': 1, 'hour': 0, 'action': [21, 24], 'reward': -1.8581318491144523, 'reward_energy': -1.636839603914276, 'reward_comfort': -2.0794240943146285, 'total_energy': 16368.39603914276, 'abs_comfort': 2.0794240943146285, 'temperatures': [20.92057590568537]}
Reward:  [-0.7885407667844642, -1.0543995195674611] {'timestep': 23328, 'time_elapsed': 20995200, 'year': 1991, 'month': 9, 'day': 1, 'hour': 0, 'action': [22, 23], 'reward': -0.9214701431759627, 'reward_energy': -0.7885407667844642, 'reward_comfort': -1.0543995195674611, 'total_energy': 7885.407667844641, 'abs_comfort': 1.0543995195674611, 'temperatures': [21.94560048043254]}
Reward:  [-0.9931231984191132, -0.0] {'timestep': 26208, 'time_elapsed': 23587200, 'year': 1991, 'month': 10, 'day': 1, 'hour': 0, 'action': [22, 23], 'reward': -0.4965615992095566, 'reward_energy': -0.9931231984191132, 'reward_comfort': -0.0, 'total_energy': 9931.231984191132, 'abs_comfort': 0.0, 'temperatures': [21.96863496011508]}
Reward:  [-0.9805738638282033, -0.0] {'timestep': 29184, 'time_elapsed': 26265600, 'year': 1991, 'month': 11, 'day': 1, 'hour': 0, 'action': [22, 22], 'reward': -0.49028693191410166, 'reward_energy': -0.9805738638282033, 'reward_comfort': -0.0, 'total_energy': 9805.738638282033, 'abs_comfort': 0.0, 'temperatures': [21.98911477390746]}
Reward:  [-1.343724380043439, -0.0] {'timestep': 32064, 'time_elapsed': 28857600, 'year': 1991, 'month': 12, 'day': 1, 'hour': 0, 'action': [21, 21], 'reward': -0.6718621900217195, 'reward_energy': -1.343724380043439, 'reward_comfort': -0.0, 'total_energy': 13437.24380043439, 'abs_comfort': 0.0, 'temperatures': [20.70296337586111]}
Reward:  [-0.5065791281596288, -1.2305126337192291] {'timestep': 35040, 'time_elapsed': 31536000, 'year': 1992, 'month': 1, 'day': 1, 'hour': 0, 'action': [17, 28], 'reward': -0.868545880939429, 'reward_energy': -0.5065791281596288, 'reward_comfort': -1.2305126337192291, 'total_energy': 5065.791281596288, 'abs_comfort': 1.2305126337192291, 'temperatures': [18.76948736628077]}
[2023-03-10 10:26:56,301] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2023-03-10 10:26:56,301] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2023-03-10 10:26:56,301] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2023-03-10 10:26:56,301] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.
[2023-03-10 10:26:56,301] EPLUS_ENV_demo-v1_MainThread_ROOT INFO:EnergyPlus simulation closed successfully.