Environment recorder #709
Open
Environment recorder #709
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
# Conflicts: # grid2op/Environment/baseEnv.py # setup.py
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
Signed-off-by: Geoffroy Jamgotchian <[email protected]>
|
BDonnot
reviewed
May 26, 2025
| This class serves as a wrapper for a given environment and records its | ||
| observations into Parquet files for later analysis. It ensures that environment | ||
| data such as observations are properly stored in a structured format. | ||
There was a problem hiding this comment.
Could you write in the docstring the directory layout for the current implementation ?
There was a problem hiding this comment.
And also explicitly tell this does not save simulation (obs.simulate) or forecast (obs.get_forecast_env)
| json.dump(env_data, f, indent=4) | ||
|
|
||
| # one table for each kind of element | ||
| self.write_element_table([env.name_gen, env.gen_type, env.gen_to_subid], ['name', 'type', 'gen_to_subid'], directory, 'gen') |
There was a problem hiding this comment.
can you also save the "gen_pos_topo_vect", "load_pos_topo_vect" etc. vector ?
|
|
||
| # env general data | ||
| env_data = { | ||
| "grid2op_version": GRID2OP_CURRENT_VERSION_STR, |
There was a problem hiding this comment.
Environments can be used in "compatibility mode", can you also add the:
"env_grid2op_version": type(env).glop_version
Comment on lines
+27
to
+298
| """ | ||
| Reset the environment to a clean state. | ||
| It will reload the next chronics if any. And reset the grid to a clean state. | ||
| This triggers a full reloading of both the chronics (if they are stored as files) and of the powergrid, | ||
| to ensure the episode is fully over. | ||
| This method should be called only at the end of an episode. | ||
| Parameters | ||
| ---------- | ||
| seed: int | ||
| The seed to used (new in version 1.9.8), see examples for more details. Ignored if not set (meaning no seeds will | ||
| be used, experiments might not be reproducible) | ||
| options: dict | ||
| Some options to "customize" the reset call. For example (see detailed example bellow) : | ||
| - "time serie id" (grid2op >= 1.9.8) to use a given time serie from the input data | ||
| - "init state" that allows you to apply a given "action" when generating the | ||
| initial observation (grid2op >= 1.10.2) | ||
| - "init ts" (grid2op >= 1.10.3) to specify to which "steps" of the time series | ||
| the episode will start | ||
| - "max step" (grid2op >= 1.10.3) : maximum number of steps allowed for the episode | ||
| - "thermal limit" (grid2op >= 1.11.0): which thermal limit to use for this episode | ||
| (and the next ones, until they are changed) | ||
| - "init datetime": which time stamp is used in the first observation of the episode. | ||
| See examples for more information about this. Ignored if | ||
| not set. | ||
| Examples | ||
| -------- | ||
| The standard "gym loop" can be done with the following code: | ||
| .. code-block:: python | ||
| import grid2op | ||
| # create the environment | ||
| env_name = "l2rpn_case14_sandbox" | ||
| env = grid2op.make(env_name) | ||
| # start a new episode | ||
| obs = env.reset() | ||
| done = False | ||
| reward = env.reward_range[0] | ||
| while not done: | ||
| action = agent.act(obs, reward, done) | ||
| obs, reward, done, info = env.step(action) | ||
| .. versionadded:: 1.9.8 | ||
| It is now possible to set the seed and the time series you want to use at the new | ||
| episode by calling `env.reset(seed=..., options={"time serie id": ...})` | ||
| Before version 1.9.8, if you wanted to use a fixed seed, you would need to (see | ||
| doc of :func:`grid2op.Environment.BaseEnv.seed` ): | ||
| .. code-block:: python | ||
| seed = ... | ||
| env.seed(seed) | ||
| obs = env.reset() | ||
| ... | ||
| Starting from version 1.9.8 you can do this in one call: | ||
| .. code-block:: python | ||
| seed = ... | ||
| obs = env.reset(seed=seed) | ||
| For the "time series id" it is the same concept. Before you would need to do (see | ||
| doc of :func:`Environment.set_id` for more information ): | ||
| .. code-block:: python | ||
| time_serie_id = ... | ||
| env.set_id(time_serie_id) | ||
| obs = env.reset() | ||
| ... | ||
| And now (from version 1.9.8) you can more simply do: | ||
| .. code-block:: python | ||
| time_serie_id = ... | ||
| obs = env.reset(options={"time serie id": time_serie_id}) | ||
| ... | ||
| .. versionadded:: 1.10.2 | ||
| Another feature has been added in version 1.10.2, which is the possibility to set the | ||
| grid to a given "topological" state at the first observation (before this version, | ||
| you could only retrieve an observation with everything connected together). | ||
| In grid2op 1.10.2, you can do that by using the keys `"init state"` in the "options" kwargs of | ||
| the reset function. The value associated to this key should be dictionnary that can be | ||
| converted to a non ambiguous grid2op action using an "action space". | ||
| .. note:: | ||
| The "action space" used here is not the action space of the agent. It's an "action | ||
| space" that uses a :func:`grid2op.Action.Action.BaseAction` class meaning you can do any | ||
| type of action, on shunts, on topology, on line status etc. even if the agent is not | ||
| allowed to. | ||
| Likewise, nothing check if this action is legal or not. | ||
| You can use it like this: | ||
| .. code-block:: python | ||
| # to start an episode with a line disconnected, you can do: | ||
| init_state_dict = {"set_line_status": [(0, -1)]} | ||
| obs = env.reset(options={"init state": init_state_dict}) | ||
| obs.line_status[0] is False | ||
| # to start an episode with a different topolovy | ||
| init_state_dict = {"set_bus": {"lines_or_id": [(0, 2)], "lines_ex_id": [(3, 2)]}} | ||
| obs = env.reset(options={"init state": init_state_dict}) | ||
| .. note:: | ||
| Since grid2op version 1.10.2, there is also the possibility to set the "initial state" | ||
| of the grid directly in the time series. The priority is always given to the | ||
| argument passed in the "options" value. | ||
| Concretely if, in the "time series" (formelly called "chronics") provides an action would change | ||
| the topology of substation 1 and 2 (for example) and you provide an action that disable the | ||
| line 6, then the initial state will see substation 1 and 2 changed (as in the time series) | ||
| and line 6 disconnected. | ||
| Another example in this case: if the action you provide would change topology of substation 2 and 4 | ||
| then the initial state (after `env.reset`) will give: | ||
| - substation 1 as in the time serie | ||
| - substation 2 as in "options" | ||
| - substation 4 as in "options" | ||
| .. note:: | ||
| Concerning the previously described behaviour, if you want to ignore the data in the | ||
| time series, you can add : `"method": "ignore"` in the dictionary describing the action. | ||
| In this case the action in the time series will be totally ignored and the initial | ||
| state will be fully set by the action passed in the "options" dict. | ||
| An example is: | ||
| .. code-block:: python | ||
| init_state_dict = {"set_line_status": [(0, -1)], "method": "force"} | ||
| obs = env.reset(options={"init state": init_state_dict}) | ||
| obs.line_status[0] is False | ||
| .. versionadded:: 1.10.3 | ||
| Another feature has been added in version 1.10.3, the possibility to skip the | ||
| some steps of the time series and starts at some given steps. | ||
| The time series often always start at a given day of the week (*eg* Monday) | ||
| and at a given time (*eg* midnight). But for some reason you notice that your | ||
| agent performs poorly on other day of the week or time of the day. This might be | ||
| because it has seen much more data from Monday at midnight that from any other | ||
| day and hour of the day. | ||
| To alleviate this issue, you can now easily reset an episode and ask grid2op | ||
| to start this episode after xxx steps have "passed". | ||
| Concretely, you can do it with: | ||
| .. code-block:: python | ||
| import grid2op | ||
| env_name = "l2rpn_case14_sandbox" | ||
| env = grid2op.make(env_name) | ||
| obs = env.reset(options={"init ts": 1}) | ||
| Doing that your agent will start its episode not at midnight (which | ||
| is the case for this environment), but at 00:05 | ||
| If you do: | ||
| .. code-block:: python | ||
| obs = env.reset(options={"init ts": 12}) | ||
| In this case, you start the episode at 01:00 and not at midnight (you | ||
| start at what would have been the 12th steps) | ||
| If you want to start the "next day", you can do: | ||
| .. code-block:: python | ||
| obs = env.reset(options={"init ts": 288}) | ||
| etc. | ||
| .. note:: | ||
| On this feature, if a powerline is on soft overflow (meaning its flow is above | ||
| the limit but below the :attr:`grid2op.Parameters.Parameters.HARD_OVERFLOW_THRESHOLD` * `the limit`) | ||
| then it is still connected (of course) and the counter | ||
| :attr:`grid2op.Observation.BaseObservation.timestep_overflow` is at 0. | ||
| If a powerline is on "hard overflow" (meaning its flow would be above | ||
| :attr:`grid2op.Parameters.Parameters.HARD_OVERFLOW_THRESHOLD` * `the limit`), then, as it is | ||
| the case for a "normal" (without options) reset, this line is disconnected, but can be reconnected | ||
| directly (:attr:`grid2op.Observation.BaseObservation.time_before_cooldown_line` == 0) | ||
| .. seealso:: | ||
| The function :func:`Environment.fast_forward_chronics` for an alternative usage (that will be | ||
| deprecated at some point) | ||
| Yet another feature has been added in grid2op version 1.10.3 in this `env.reset` function. It is | ||
| the capacity to limit the duration of an episode. | ||
| .. code-block:: python | ||
| import grid2op | ||
| env_name = "l2rpn_case14_sandbox" | ||
| env = grid2op.make(env_name) | ||
| obs = env.reset(options={"max step": 288}) | ||
| This will limit the duration to 288 steps (1 day), meaning your agent | ||
| will have successfully managed the entire episode if it manages to keep | ||
| the grid in a safe state for a whole day (depending on the environment you are | ||
| using the default duration is either one week - roughly 2016 steps or 4 weeks) | ||
| .. note:: | ||
| This option only affect the current episode. It will have no impact on the | ||
| next episode (after reset) | ||
| For example: | ||
| .. code-block:: python | ||
| obs = env.reset() | ||
| obs.max_step == 8064 # default for this environment | ||
| obs = env.reset(options={"max step": 288}) | ||
| obs.max_step == 288 # specified by the option | ||
| obs = env.reset() | ||
| obs.max_step == 8064 # retrieve the default behaviour | ||
| .. seealso:: | ||
| The function :func:`Environment.set_max_iter` for an alternative usage with the different | ||
| that `set_max_iter` is permenanent: it impacts all the future episodes and not only | ||
| the next one. | ||
| If you want your environment to start at a given time stamp you can do: | ||
| .. code-block:: python | ||
| import grid2op | ||
| env_name = "l2rpn_case14_sandbox" | ||
| env = grid2op.make(env_name) | ||
| obs = env.reset(options={"init datetime": "2024-12-06 00:00"}) | ||
| obs.year == 2024 | ||
| obs.month == 12 | ||
| obs.day == 6 | ||
| .. seealso:: | ||
| If you specify "init datetime" then the observation resulting to the | ||
| `env.reset` call will have this datetime. If you specify also `"skip ts"` | ||
| option the behaviour does not change: the first observation will | ||
| have the date time attributes you specified. | ||
| In other words, the "init datetime" refers to the initial observation of the | ||
| episode and NOT the initial time present in the time series. | ||
| """ |
There was a problem hiding this comment.
I think this docstring is better in the Environment class because it's an environment feature.
FOrecasted env, or others might behave differently
| """ | ||
|
|
||
| class BaseEnv(GridObjects, RandomObject, ABC): | ||
| class BaseEnv(EnvInterface, GridObjects, RandomObject, ABC): |
There was a problem hiding this comment.
From the python MRO point of view, I think it's better to move it Before RandomObject and ABC.
Or even making EnvInterface inherit from GriObjects and RandomObjects directly ?
Comment on lines
-1069
to
-1090
| """ | ||
| Reset the environment to a clean state. | ||
| It will reload the next chronics if any. And reset the grid to a clean state. | ||
| This triggers a full reloading of both the chronics (if they are stored as files) and of the powergrid, | ||
| to ensure the episode is fully over. | ||
| This method should be called only at the end of an episode. | ||
| Parameters | ||
| ---------- | ||
| seed: int | ||
| The seed to used (new in version 1.9.8), see examples for more details. Ignored if not set (meaning no seeds will | ||
| be used, experiments might not be reproducible) | ||
| options: dict | ||
| Some options to "customize" the reset call. For example (see detailed example bellow) : | ||
| - "time serie id" (grid2op >= 1.9.8) to use a given time serie from the input data | ||
| - "init state" that allows you to apply a given "action" when generating the | ||
| initial observation (grid2op >= 1.10.2) | ||
| - "init ts" (grid2op >= 1.10.3) to specify to which "steps" of the time series |
There was a problem hiding this comment.
I think it's better to have the entire docstring here instead of in EnvInterface, as "reset" does not do exactly the same things for all types of environment (_ObsEnv, ForecastEnv notably), though the method has the same signature.
| "typing_extensions", | ||
| "orderly_set<5.4.0; python_version<='3.8'" | ||
| "orderly_set<5.4.0; python_version<='3.8'", | ||
| "pyarrow>=17.0.0" |
There was a problem hiding this comment.
Maybe make it optional and add the according try/catch when importing EnvRecorder to keep the base install as small as possible ?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.