9. Wrappers

Sinergym has several wrappers in order to add some functionality in the environment that it doesn’t have by default. The code can be found in sinergym/sinergym/utils/wrappers.py. You can implement your own wrappers inheriting from gym.Wrapper or some of its variants, see Gymnasium documentation.

9.1. MultiObjectiveReward

Environment step will return a vector reward (selected elements in wrapper constructor, one for each objective) instead of a traditional scalar value. See #301.

9.2. PreviousObservationWrapper

Wrapper to add observation values from previous timestep to current environment observation. It is possible to select the variables you want to track its previous observation values.

9.3. DatetimeWrapper

Wrapper to substitute day_of_month value by is_weekend flag, and hour and month by sin and cos values. Observation space is updated automatically.

9.4. NormalizeAction

Wrapper to apply normalization in action space. It is very useful in DRL algorithms such as some of them only works with normalized values correctly, making environments more generic in DRL solutions.

By default, the normalization is applied in the range [-1,1]. However, other range can be specified when wrapper is instantiated.

Sinergym parse these values to real action space defined in original environment internally before to send it to EnergyPlus Simulator by the API middleware.

Important

The method in charge of parse this values from [-1,1] to real action space if it is required is called reverting_action(action) in the wrapper class. We always recommend to use the normalization in action space for DRL solutions, since this space is compatible with all algorithms. However, if you are implementing your own rule-based controller and working with real action values, for example, you don’t must to apply this wrapper.

Normalize action wrapper graph.

9.5. DiscretizeEnv

Wrapper to discretize the action space. Discrete space must be defined following the Gymnasium standard. This space should be gym.spaces.Discrete, gym.spaces.MultiDiscrete or gym.spaces.MultiBinary. An action mapping function is also given, in order to map this/these value(s) into ones that fit the underlying continuous environment (before to send it to the simulator).

Important

The discrete space must discretize the original continuous space. Therefore, discrete space only should reach values which are considered in the original environment action space.

This action mapping function can be defined by users to specify how to jump from discrete to continuous values. If action mapping function output doesn’t match with original environment action space, an error will be raised. See Environment Discretization Wrapper for an example of use.

Discretize wrapper graph.

9.6. DiscreteIncrementalWrapper

A wrapper for an incremental setpoint action space environment. This wrapper will update an environment, converting it in a discrete environment with an action mapping function and action space depending on the step and delta specified. The action will be sum with current setpoint values instead of overwrite the latest action. Then, the action is the current setpoint values with the increase instead of the discrete value action whose purpose is to define the increment/decrement itself.

Incremental wrapper graph.

9.7. NormalizeObservation

It is used to transform observation received from simulator in values between -1 and 1. It is based in the dynamic normalization wrapper of Gymnasium. At the beginning, it is not precise and the values may be out of range usually, so use this wrapper carefully.

9.8. LoggerWrapper

Wrapper for logging all interactions between agent and environment. Logger class can be selected in the constructor if other type of logging is required. For more information about Sinergym Logger visit Logger.

9.9. MultiObsWrapper

Stack observation received in a history queue (size can be customized).

Note

For examples about how to use these wrappers, visit Wrappers example.

Important

You have to be careful if you are going to use several nested wrappers. A wrapper works on top of the previous one. The order is flexible since Sinergym v3.0.5.