Skip to content

Commit cc899e6

Browse files
BDonnotgeofjamg
andauthored
Dev switches (#723)
* initial commit with possible ideas [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * a first possible solution, will see if it works or not [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * improving on the detailed topology representation [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * first working POC on ieee grid and for backend actions, no regular action, no observation yet Signed-off-by: DONNOT Benjamin <[email protected]> * fixing 'legacy' import when no switches are defined Signed-off-by: DONNOT Benjamin <[email protected]> * fixing a broken test Signed-off-by: DONNOT Benjamin <[email protected]> * trying to fix readthedocs [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * trying to fix readthedocs [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * WIP, fixing read the docs Signed-off-by: DONNOT Benjamin <[email protected]> * start to add the switches action side, will require more time Signed-off-by: DONNOT Benjamin <[email protected]> * fixing an error in the action Signed-off-by: DONNOT Benjamin <[email protected]> * before solving conflicts Signed-off-by: DONNOT Benjamin <[email protected]> * fix a bug after merging Signed-off-by: DONNOT Benjamin <[email protected]> * in the refacto for the connectivity node + synch with master [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * model with the connectivity nodes seems consistent, need to fix tests and implement two main methods [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * tests for detailed topo now pass again Signed-off-by: DONNOT Benjamin <[email protected]> * bump to upload / download artifact v4 Signed-off-by: DONNOT Benjamin <[email protected]> * fight with github ci and node.js again Signed-off-by: DONNOT Benjamin <[email protected]> * adding function to convert switches state to topo_vect, need more tests [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * ready for detailed_topo class itself, now let's add action and later observation [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * start to implement the action module for switches Signed-off-by: DONNOT Benjamin <[email protected]> * attempt to fix CI and nodejs complaining about kernel version Signed-off-by: DONNOT Benjamin <[email protected]> * fix broken tests after the change from IllegalAction to AmbiguousAction Signed-off-by: DONNOT Benjamin <[email protected]> * CI should be fixed, possible to do switch actions, need backend support and observation too Signed-off-by: DONNOT Benjamin <[email protected]> * start implementation -required...- for converting nodal to switch topology Signed-off-by: DONNOT Benjamin <[email protected]> * improving the representation of detailed topo [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * first version of detailed topo with generic topology Signed-off-by: DONNOT Benjamin <[email protected]> * fix a bug due to shunt when all shunts are disconnected Signed-off-by: DONNOT Benjamin <[email protected]> * add more tests, fix issues when setting topology of multiple substations Signed-off-by: DONNOT Benjamin <[email protected]> * some work to do for topologies similar to real ones [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * fix a bug, seems to work now [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * more test pass [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * improving detailed top [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * improving detailed top [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * improve the algo [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * add lots of test for detailed topo description Signed-off-by: DONNOT Benjamin <[email protected]> * fixing broken test (topo_vect is the full vector and not only the substation) Signed-off-by: DONNOT Benjamin <[email protected]> * setting up independant tests for detailed topology - can take quite some time Signed-off-by: DONNOT Benjamin <[email protected]> * fix broken test due to imports Signed-off-by: DONNOT Benjamin <[email protected]> * working on the detailed topology description, for backend [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * improving the switch support in backend action Signed-off-by: DONNOT Benjamin <[email protected]> * add the 'get_XXX_info' method and the possibility to reset with new thermal limtis Signed-off-by: DONNOT Benjamin <[email protected]> * improve handling of detailed topology Signed-off-by: DONNOT Benjamin <[email protected]> * Fix merge Signed-off-by: Geoffroy Jamgotchian <[email protected]> * synch with dev_1.12.0 * fix a bug in merge of BaseAction Signed-off-by: DONNOT Benjamin <[email protected]> * fix some broken tests after integration Signed-off-by: DONNOT Benjamin <[email protected]> * fix some other tests Signed-off-by: DONNOT Benjamin <[email protected]> * ready to make proper integration tests for the feature, with backend not supporting switches first Signed-off-by: DONNOT Benjamin <[email protected]> * tests seem to pass Signed-off-by: DONNOT Benjamin <[email protected]> * optimize grid2Op action building [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * start to optimize action [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * fixing errors in CI Signed-off-by: DONNOT Benjamin <[email protected]> * increase version number: 1.12.2.dev0 Signed-off-by: DONNOT Benjamin <[email protected]> * fix some errors Signed-off-by: DONNOT Benjamin <[email protected]> * fix github ci for extended test suite about the switches Signed-off-by: DONNOT Benjamin <[email protected]> * improve parralelism in grid2op circleci test suite Signed-off-by: DONNOT Benjamin <[email protected]> * fix some broken tests after merge Signed-off-by: DONNOT Benjamin <[email protected]> * (undo) improve parralelism in grid2op circleci test suite Signed-off-by: DONNOT Benjamin <[email protected]> * improve building of the reference attribute Signed-off-by: DONNOT Benjamin <[email protected]> * fix issues Signed-off-by: DONNOT Benjamin <[email protected]> * some addition for definition of classes Signed-off-by: DONNOT Benjamin <[email protected]> * some more fixes, but some other bugs Signed-off-by: DONNOT Benjamin <[email protected]> * fix some issues in CI Signed-off-by: DONNOT Benjamin <[email protected]> * fix some issues on sonarqube [skip ci] Signed-off-by: DONNOT Benjamin <[email protected]> * fix some issues on sonarqube Signed-off-by: DONNOT Benjamin <[email protected]> * removing commented out code Signed-off-by: DONNOT Benjamin <[email protected]> * fix unused import in unused script Signed-off-by: DONNOT Benjamin <[email protected]> --------- Signed-off-by: DONNOT Benjamin <[email protected]> Signed-off-by: Geoffroy Jamgotchian <[email protected]> Co-authored-by: Geoffroy Jamgotchian <[email protected]>
1 parent 777868b commit cc899e6

File tree

708 files changed

+30525
-782
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

708 files changed

+30525
-782
lines changed

.circleci/config.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ executors:
3131
jobs:
3232
test:
3333
executor: grid2op-executor
34-
resource_class: medium+
34+
resource_class: large
3535
parallelism: 4
3636
steps:
3737
- checkout

.github/workflows/main.yml

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -266,11 +266,68 @@ jobs:
266266
- name: Test the automatic generation of classes in the env folder
267267
run: |
268268
python -m unittest grid2op/tests/automatic_classes.py -v -f
269+
270+
detailed_topology:
271+
name: Test detailed topology on ${{ matrix.config.name }}
272+
runs-on: ${{ matrix.config.os }}
273+
strategy:
274+
matrix:
275+
config:
276+
# - {
277+
# name: darwin,
278+
# os: macos-latest,
279+
# }
280+
# - {
281+
# name: windows,
282+
# os: windows-2019,
283+
# }
284+
- {
285+
name: ubuntu,
286+
os: ubuntu-latest,
287+
}
288+
python:
289+
- {
290+
name: cp39,
291+
version: '3.9',
292+
}
293+
- {
294+
name: cp313,
295+
version: '3.13',
296+
}
297+
298+
steps:
299+
300+
- name: Checkout sources
301+
uses: actions/checkout@v3
302+
with:
303+
submodules: true
304+
305+
- name: Setup Python
306+
uses: actions/setup-python@v2
307+
with:
308+
python-version: ${{ matrix.python.version }}
309+
310+
- name: Install Python dependencies
311+
run: |
312+
python -m pip install --upgrade pip wheel setuptools gymnasium build
313+
314+
- name: Build wheel
315+
run: python -m build --wheel .
316+
317+
- name: Install wheel
318+
shell: bash
319+
run: |
320+
python -m pip install dist/*.whl --user
321+
pip freeze
322+
323+
- name: Test the working of the detailed topo representation
324+
run: |
325+
python -m unittest grid2op/tests/indep_test_detailed_topo_extended.py -f
269326
270327
package:
271328
name: Test install
272329
runs-on: ubuntu-latest
273-
needs: [manylinux_build, macos_windows_build, auto_class_in_file]
330+
needs: [manylinux_build, macos_windows_build, auto_class_in_file, detailed_topology]
274331

275332
steps:
276333
- name: Download wheels

.gitignore

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -416,7 +416,17 @@ getting_started/venv_310_ray/
416416
grid2op/tests/venv_test_autoclass/
417417
test_eduardo.py
418418
grid2op/tests/failed_test*
419+
grid2op/tests/ampl_from_iidm.py
420+
grid2op/tests/do_I_repeat.txt
421+
grid2op/tests/parse_res35_working.txt
422+
grid2op/tests/parse_res_35 (copie).txt
423+
grid2op/tests/parse_res_35.txt
424+
grid2op/tests/res_algo_topo_35.txt
425+
grid2op/tests/test_topo_ampl/
426+
grid2op/tests/test_topo_ampl2/
419427
venv_312
428+
venv_*
429+
grid2op/tests/internal_*.pickle
420430
grid2op_test_issue_discord.ipynb
421431
grid2op_test_issue_discord_orig.ipynb
422432
grid_bug_pp3.json

CHANGELOG.rst

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -100,6 +100,19 @@ Native multi agents support:
100100
- add detachment
101101
- add change_bus / set_bus
102102

103+
[1.13.0] - DETAILED TOPO
104+
---------------------------
105+
- [FIXED] some issues with gym classes
106+
- [IMPROVED] the serialization of class definition
107+
when using "automatic_classes" or "experimental_read_from_local_dir"
108+
- [IMPROVED] when using gym MultiDiscrete as an action space,
109+
if the dimension is 0 (for one of the MultiDiscrete) then it is now
110+
removed from the action space (for example, if your environment does not
111+
contain storage unit, the corresponding MultiDiscrete will have one less
112+
dimension, because you cannot act on storage unit.)
113+
- [IMPROVED] classes for action and observation classes are now easier to
114+
initialize (added the `finalize_class_definition` classmethod)
115+
103116
[1.12.1] - 2025-08-28
104117
----------------------
105118
- [BREAKING] (small impact) action "property" `shunt_p`, `shunt_q` and `shunt_bus`
@@ -288,9 +301,13 @@ Native multi agents support:
288301

289302
[1.10.4] - 2024-10-14
290303
-------------------------
291-
- [FIXED] an issue in the backend: if the backend failed to be
304+
- [FIXED] an issue in the backend: if the backend failed to be copied
292305
created the `_grid` attribute was set to `None` and not set back to
306+
its original value in the copied backend.
293307
- [FIXED] the `self.skip_if_needed()` was missing for one of the test suite.
308+
- [FIXED] the correct `AmbiguousAction` is now raised when grid2op does not understand
309+
what an action should be doing (an incorrect `IllegalAction` used to be sent)
310+
- [FIXED] a test in `test_ActionProperties` did not test the correct property
294311
- [FIXED] an error in the descirption of the `educ_case14_storage` environment
295312
(wrong sign for the slack generator)
296313
- [FIXED] the environment would not load in case of an incorrect "layout.json"
@@ -303,6 +320,7 @@ Native multi agents support:
303320
in the readme)
304321
- [ADDED] numpy 2 support (now that pandapower allows it)
305322
- [IMPROVED] error message when forecasts are not correctly set-up
323+
- [IMPROVED] an error message when loading a grid with forecasts
306324

307325

308326
[1.10.3] - 2024-07-12
@@ -373,7 +391,6 @@ Native multi agents support:
373391
processes (using `fork` at least), see `ep_data.make_serializable`
374392
- [IMPROVED] chronix2grid tests are now done independantly on the CI
375393

376-
377394
[1.10.2] - 2024-05-27
378395
-------------------------
379396
- [BREAKING] the `runner.run_one_episode` now returns an extra argument (first position):
@@ -1160,7 +1177,7 @@ Native multi agents support:
11601177
- [ADDED]: function to retrieve the maximum duration of the current episode.
11611178
- [ADDED]: a new kind of opponent that is able to attack at "more random" times with "more random" duration.
11621179
See the `GeometricOpponent`.
1163-
- [IMPROVED]: on windows at least, grid2op does not work with gym < 0.17.2 Checks are performed in order to make sure
1180+
- [IMPROVED]: on windows at least, grid2op does not work with `gym < 0.17.2` Checks are performed in order to make sure
11641181
the installed open ai gym package meets this requirement (see issue
11651182
`Issue#185 <https://github.com/Grid2Op/grid2op/issues/185>`_ )
11661183
- [IMPROVED] the seed of openAI gym for composed action space (see issue `https://github.com/openai/gym/issues/2166`):

_profiling/profiler_switch.py

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Copyright (c) 2019-2020, RTE (https://www.rte-france.com)
2+
# See AUTHORS.txt
3+
# This Source Code Form is subject to the terms of the Mozilla Public License, version 2.0.
4+
# If a copy of the Mozilla Public License, version 2.0 was not distributed with this file,
5+
# you can obtain one at http://mozilla.org/MPL/2.0/.
6+
# SPDX-License-Identifier: MPL-2.0
7+
# This file is part of Grid2Op, Grid2Op a testbed platform to model sequential decision making in power systems.
8+
9+
"""
10+
This file should be used to assess the performance of grid2op in "runner" mode: the loading time are not studied,
11+
neither are the import times.
12+
13+
Data are loaded only once, when the environment is "done" the programm stops.
14+
15+
This corresponds to the situation: you have a trained agent, you want to assess its performance using the runner
16+
"""
17+
18+
import os
19+
20+
from grid2op import make
21+
from grid2op.Action import CompleteAction
22+
from grid2op.Space import AddDetailedTopoIEEE
23+
from grid2op.Parameters import Parameters
24+
from grid2op.Agent import BaseAgent
25+
from grid2op.Rules import AlwaysLegal
26+
from grid2op.Backend import PandaPowerBackend
27+
import cProfile
28+
29+
from utils_benchmark import run_env, str2bool
30+
31+
try:
32+
from lightsim2grid import LightSimBackend
33+
light_sim_avail = True
34+
except ImportError:
35+
light_sim_avail = False
36+
37+
ENV_NAME = "rte_case5_example"
38+
ENV_NAME = "rte_case14_realistic"
39+
MAX_TS = 1000
40+
RANDOM_SWITCH_AGENT = False # do not change this
41+
# for profiling it has no impact.
42+
43+
44+
class RandomBusSwitchAgent(BaseAgent):
45+
def act(self, observation, reward, done=False):
46+
sub_id = self.space_prng.randint(type(self.action_space).n_sub)
47+
new_position = self.space_prng.choice([-1, 1], size=1)
48+
return self.action_space({"set_switch": [(sub_id, new_position)]})
49+
50+
51+
class RandomElSwitchAgent(BaseAgent):
52+
def act(self, observation, reward, done = False):
53+
if observation.current_step <= 1:
54+
# deactivate the connector between busbars
55+
res = self.action_space({"set_switch": [(sub_id, -1)
56+
for sub_id in range(type(self.action_space).n_sub)]})
57+
else:
58+
res = self.action_space()
59+
60+
do_act = self.space_prng.choice([0, 1], size=1, p = [0.95, 0.05])
61+
if not do_act:
62+
return res
63+
64+
n_switch = type(self.action_space).detailed_topo_desc.switches.shape[0]
65+
switch_id = self.space_prng.randint(type(self.action_space).n_sub,
66+
n_switch,
67+
size=1)
68+
new_position = self.space_prng.choice([-1, 1], size=1)
69+
res.update({"set_switch": [(switch_id, new_position)]})
70+
return res
71+
72+
def main(max_ts, name, use_lightsim=False, test_env=True):
73+
param = Parameters()
74+
if use_lightsim:
75+
if light_sim_avail:
76+
class ThisBackendClass(AddDetailedTopoIEEE, LightSimBackend):
77+
pass
78+
else:
79+
raise RuntimeError("LightSimBackend not available")
80+
else:
81+
class ThisBackendClass(AddDetailedTopoIEEE, PandaPowerBackend):
82+
pass
83+
84+
backend = ThisBackendClass()
85+
86+
param.init_from_dict(
87+
{"NO_OVERFLOW_DISCONNECTION": True,
88+
"STOP_EP_IF_GEN_BREAK_CONSTRAINTS": False,
89+
"ENV_DOES_REDISPATCHING": False})
90+
91+
env_klu = make(name,
92+
backend=backend,
93+
param=param,
94+
gamerules_class=AlwaysLegal,
95+
test=test_env,
96+
action_class=CompleteAction)
97+
assert type(env_klu).detailed_topo_desc is not None
98+
agent = RandomBusSwitchAgent(action_space=env_klu.action_space)
99+
if RANDOM_SWITCH_AGENT:
100+
agent = RandomElSwitchAgent(action_space=env_klu.action_space)
101+
agent.seed(4)
102+
103+
cp = cProfile.Profile()
104+
cp.enable()
105+
nb_ts_klu, *_, time_step = run_env(env_klu, max_ts, agent)
106+
cp.disable()
107+
print(f'Time for {nb_ts_klu} steps: {time_step} => {time_step / nb_ts_klu} s/step or {nb_ts_klu / time_step:.3e} step / s')
108+
nm_f, ext = os.path.splitext(__file__)
109+
nm_out = "{}_{}_{}.prof".format(nm_f, "lightsim" if use_ls else "pp", name)
110+
cp.dump_stats(nm_out)
111+
print("You can view profiling results with:\n\tsnakeviz {}".format(nm_out))
112+
113+
114+
if __name__ == "__main__":
115+
import argparse
116+
parser = argparse.ArgumentParser(description='Benchmark pyKLU and Pandapower Backend for an agent that takes every '
117+
'topological action possible')
118+
parser.add_argument('--name', default=ENV_NAME, type=str,
119+
help='Environment name to be used for the benchmark.')
120+
parser.add_argument('--number', type=int, default=MAX_TS,
121+
help='Maximum number of time steps for which the benchamark will be run.')
122+
parser.add_argument("--use_ls", type=str2bool, nargs='?',
123+
const=True, default=False,
124+
help="Use the LightSim2Grid Backend.")
125+
parser.add_argument("--no_test", type=str2bool, nargs='?',
126+
const=True, default=False,
127+
help="Do not use a test environment for the profiling (default to False: meaning you use a test env)")
128+
129+
args = parser.parse_args()
130+
131+
max_ts = int(args.number)
132+
name = str(args.name)
133+
use_ls = args.use_ls
134+
test_env = not args.no_test
135+
main(max_ts, name, use_lightsim=use_ls, test_env=test_env)

_profiling/utils_benchmark.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
import numpy as np
1111
from tqdm import tqdm
1212
import argparse
13+
from grid2op.Environment import Environment
1314
from grid2op.Agent import AgentWithConverter
1415
from grid2op.Converter import IdToAct
1516

@@ -179,12 +180,12 @@ def print_res(env_klu, env_pp,
179180
print("Absolute value of the difference for gen_q: {}".format(np.max(np.abs(gen_q_klu - gen_q_pp))))
180181

181182

182-
def run_env(env, max_ts, agent):
183+
def run_env(env: Environment, max_ts, agent):
183184
nb_rows = min(env.chronics_handler.max_timestep(), max_ts)
184185
aor = np.zeros((nb_rows, env.n_line))
185186
gen_p = np.zeros((nb_rows, env.n_gen))
186187
gen_q = np.zeros((nb_rows, env.n_gen))
187-
obs = env.get_obs()
188+
obs = env.get_obs(_do_copy=False)
188189
done = False
189190
reward = env.reward_range[0]
190191
nb_ts = 0
@@ -196,6 +197,7 @@ def run_env(env, max_ts, agent):
196197
act = agent.act(obs, reward, done)
197198
beg_step = time.perf_counter()
198199
obs, reward, done, info = env.step(act)
200+
assert not info["is_ambiguous"], info["exception"]
199201
time_step += time.perf_counter() - beg_step
200202
aor[nb_ts, :] = obs.a_or
201203
gen_p[nb_ts, :] = obs.prod_p

docs/conf.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@
2222
author = 'Benjamin Donnot'
2323

2424
# The full version, including alpha/beta/rc tags
25-
release = '1.12.1'
25+
release = '1.12.2.dev0'
2626
version = '1.12'
2727

2828

docs/detailed_topology.rst

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
.. _detailed-topology-modeling-module:
2+
3+
Dive into the detailed topology "modeling" in grid2op
4+
===================================================================
5+
6+
.. warning::
7+
Work in progress
8+
9+
What is a "detailed" topology in grid2op
10+
-----------------------------------------
11+
12+
.. warning::
13+
Work in progress
14+
15+
16+
- Concept of connectivity nodes
17+
- Switches
18+
- Processing of the switches to "original topoolgy"
19+
-
20+
21+
Impact in grid2op
22+
------------------
23+
24+
.. warning::
25+
Work in progress
26+
27+
- new action
28+
- new observation
29+
30+
.. danger::
31+
Be carefull with convertion fo / from switches !!!
32+
33+
Why did we add it ?
34+
--------------------
35+
36+
.. warning::
37+
Work in progress
38+
39+
What features are actually implemented ?
40+
-----------------------------------------
41+
42+
.. warning::
43+
Work in progress
44+
45+
46+
Pros and cons of using it ?
47+
---------------------------------
48+
.. warning::
49+
Work in progress
50+

docs/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,7 @@ graph.
110110
modeled_elements
111111
grid_graph
112112
topology
113+
detailed_topology
113114

114115
Environments
115116
---------------

0 commit comments

Comments
 (0)