Proposed changes to make EasyBuild plugin-able through entrypoints #4918

Crivella · 2025-06-10T16:03:15Z

The proposed changes leverage python EntryPoints to allow injecting new functionalities in easybuild without touching the main repo

For now this is a demo of what could be possible implementing the following:

easybuild.toolchain entrypoint to inject a toolchain into EasyBuild from a separate python package
easybuild.easyblock entrypoint to inject a new easyblock into EasyBuild from a separate python package
easybuild.hooks entrypoint to inject new hooks into EasyBuild from a separate python package

For easier error detection in the plugin package, the new easybuild.tools.entrypoints defines several registed_XXX decorators that needs to be used in order to pass the entrypoint validation.

An example of a python package that makes use of the proposed functionalities can be found in https://github.com/Crivella/test_eb_ep

Using entrypoints is not enabled by default and requires the --use-entrypoints flag to be passed to eb command.

Possible upsides

Simplifies extending EasyBuild functionalities without touching the main repo
Could simplify working with/maintaining a customized local stack
- In particular allowing running multiple hooks on the same step with ordering enforced by the priority can be useful
Could allow to have a minimal version of easybuild that is extensible via modules

Possible downsides

Could promote fragmentation, with different packages providing different functionalities and less upstream contributions

TODO

Add documentation on how to create a package that uses the entrypoints
- Added documentation for entrypoints easybuild-docs#331
Add tests

MAYBE

Add new entrypoints for other functionalities
Possibly rework current (demo) functionalities for better integration (for now i took what i felt was the path of least resistance to implement the demo)

Notes

importlib.metadata has been part of python only from 3.8 before the 3rd party importlib-metadata package would be required
Currently for docs.py CLI options like --list-easyblocks --list-toolchains and --show-config that are run before the build options are initialized, the default is to assume that use_entrypoints is True. At runtime when the hooks/easyblocks/toolchains need to be injected, this will only be performed if the options is actually set to True

boegel · 2025-06-12T06:42:40Z

I only took a (very) quick look at this, but couple of questions:

This is basically a (more modern) alternative to --include-easyblocks, --hooks, --include-toolchains, correct?
What are the main advantages of this approach compared to the mechanisms we already have?
One downside I see is that this is more "hidden", having entrypoints available won't show up in the output of eb --show-config, for example. I think it's important to somehow expose the fact that entrypoints are available, perhaps by making --show-config print something extra in case one or more entrypoints are found?

Other random thoughts:

The fact that this requires Python >= 3.8 (or the installation of an extra Python package) is fine, we're somewhat close to recommending Python >= 3.9 anyway, definitely not a blocker (especially since this is opt-in), as long as there's good error reporting when importlib.metadata is not available.
Since this is opt-in, I have no objections on including this alongside the existing mechanisms we have (--include-*, --hooks, etc.).
Can the entrypoints mechanisms and the existing mechanisms (--include-*, --hooks) be combined? If not, we should clearly define/document which one "wins" in case of clashes, and print a warning to make it crystal clear which easyblock/hooks/etc is being ignored (or even just error out and not allow going that route).

Crivella · 2025-06-12T10:07:01Z

This is basically a (more modern) alternative to --include-easyblocks, --hooks, --include-toolchains, correct?

This is what I've implemented as a demo as it was probably the easiest features, but i think several other features could also be done through plugins. A few ideas:

extending the command line options
adding custom module_tools / naming schemes
adding custom schedulers for jobs

I think several features could be implemented, requiring varying degree of effort

What are the main advantages of this approach compared to the mechanisms we already have

Hooks: Right now only one hook can be ran per step, which requires the logic to run multiple sub-hooks to be implemented in the hook itself like we do for EESSI.
With this, multiple hooks can be registered and ran in the order specified. Also the hooks can come from several independent sources, without needing to manually tying them together.
Toolchains/EasyBlocks: While it is possible to specify tcs,ebs coming from multiple files, automating their loading eg by switching modules/virtualenvs is tricky.
EG, for venvs, right now one would require manually modifying the activation script to add the environment variables, every time one wants to add a new included by default toolchain or easyblock.

Using entrypoints, this would be made easier, eg one only has to install/uninstall the respective python package

One downside I see is that this is more "hidden", having entrypoints available won't show up in the output of eb --show-config, for example. I think it's important to somehow expose the fact that entrypoints are available, perhaps by making --show-config print something extra in case one or more entrypoints are found?

Will work on it

The fact that this requires Python >= 3.8 (or the installation of an extra Python package) is fine, we're somewhat close to recommending Python >= 3.9 anyway, definitely not a blocker (especially since this is opt-in), as long as there's good error reporting when importlib.metadata is not available.

Added a guard for it. It was also needed to pass the CI (still need a more robust way as the entry_points() itself is implemented differently for 3.8 <= Python < 3.10 vs later versions)

Can the entrypoints mechanisms and the existing mechanisms (--include-*, --hooks) be combined? If not, we should clearly define/document which one "wins" in case of clashes, and print a warning to make it crystal clear which easyblock/hooks/etc is being ignored (or even just error out and not allow going that route).

Yes they coexist, bot for easyblocks and toolchains I've modified the functions that get them to include the one coming from entrypoints after the ones already picked up by easybuild.

Hooks: The normal easybuild hook will be ran first, then entrypoint hooks will be ran after (if enabled and found) in the order specified by the priority (descending) and hook name in case of same prio (ascending) (could probably also log a warning if hooks with same priority are found).
Toolchains: The entrypoint toolchains are appended or prependend (defaults to appended) to the list of toolchains found by easybuild.tools.toolchains.utilities.search_toolchains(), allowing an entrypoint toolchain to either just add to the list of toolchains or override an existing one (also here we could add warnings/errors for name clashes between entrypoints and possible another flag to say if a registered toolchain is allowed to override an existing one).
EasyBlocks: For now entrypoints easyblocks are picked up before existing ones (by changing the results of easybuild.framework.easyconfig.get_module_path()) but similar logic could also be implemented here to change the priority

…tlib.metadata` in old python

- Added listing of entrypoints available through `--show-config` - Enhanced test cases

Crivella · 2025-06-19T14:08:54Z

Added tests
--show-config will now also report all the hooks that will be available at runtime

The errors in the 3.7 CI seems to be of the kind

2025-06-19T13:09:24.6497370Z ======================================================================
2025-06-19T13:09:24.6497515Z FAIL: test_check_log_for_errors (test.framework.run.RunTest)
2025-06-19T13:09:24.6497602Z Test for check_log_for_errors
2025-06-19T13:09:24.6497710Z ----------------------------------------------------------------------
2025-06-19T13:09:24.6497791Z Traceback (most recent call last):
2025-06-19T13:09:24.6498306Z   File "/tmp/runner/825309bae7e9b24d2d2f9173c0a8b4ccbcbaef5f/lib/python3.7/site-packages/easybuild/base/testing.py", line 170, in assertErrorRegex
2025-06-19T13:09:24.6498390Z     call(*args, **kwargs)
2025-06-19T13:09:24.6498859Z   File "/tmp/runner/825309bae7e9b24d2d2f9173c0a8b4ccbcbaef5f/lib/python3.7/site-packages/easybuild/_deprecated.py", line 850, in check_log_for_errors
2025-06-19T13:09:24.6499387Z     _log.deprecated("check_log_for_errors is deprecated, you should stop using it", '6.0')
2025-06-19T13:09:24.6500043Z easybuild.tools.build_log.EasyBuildError: 'DEPRECATED (since v6.0) functionality used: check_log_for_errors is deprecated, you should stop using it; see https://docs.easybuild.io/deprecated-functionality/ for more information'
2025-06-19T13:09:24.6500056Z 
2025-06-19T13:09:24.6500222Z During handling of the above exception, another exception occurred:
2025-06-19T13:09:24.6500233Z 
2025-06-19T13:09:24.6500315Z Traceback (most recent call last):
2025-06-19T13:09:24.6500815Z   File "/tmp/runner/825309bae7e9b24d2d2f9173c0a8b4ccbcbaef5f/lib/python3.7/site-packages/test/framework/run.py", line 1928, in test_check_log_for_errors
2025-06-19T13:09:24.6501197Z     self.assertErrorRegex(EasyBuildError, "Invalid input:", check_log_for_errors, "", [42])
2025-06-19T13:09:24.6501666Z   File "/tmp/runner/825309bae7e9b24d2d2f9173c0a8b4ccbcbaef5f/lib/python3.7/site-packages/easybuild/base/testing.py", line 178, in assertErrorRegex
2025-06-19T13:09:24.6501891Z     self.assertTrue(regex.search(msg), "Pattern '%s' is found in '%s'" % (regex.pattern, msg))
2025-06-19T13:09:24.6502607Z AssertionError: None is not true : Pattern 'Invalid input:' is found in 'DEPRECATED (since v6.0) functionality used: check_log_for_errors is deprecated, you should stop using it; see https://docs.easybuild.io/deprecated-functionality/ for more information'
2025-06-19T13:09:24.6502718Z

or

2025-06-19T13:09:25.4353716Z == 2025-06-19 13:08:40,845 build_log.py:226 ERROR EasyBuild encountered an error (at easybuild/base/exceptions.py:126 in __init__): Failed to create directory /tmp/eb-p64t0__o/eb-lh9axsjv/eb-6qub6uza/eb-eos8avwi/eb-wgvaxs_l/eb-lux7c5ua/eb-5v8m9itz/eb-p4mv0hf9/eb-64hom17r/eb-i_bpvb7s/eb-3h8jw78o/eb-7pinubbt/eb-j0ilmdjx/eb-lcwyda3q/eb-2053q4bl/eb-o5wtfasa/eb-o429qvxs/eb-5ms13swi/eb-483btj_h/eb-58sj851i/eb-k4t9u0sq/eb-nbxa6_c5/eb-x5nsuog7/eb-lii8ypgm/eb-_vx6_y7k/eb-noovmfd7/eb-mik1_ihm/tmpbv17huit/software/.locks/

do not think it is related to this PR

Crivella · 2025-06-19T15:00:02Z

What it looks like

Using the test repo from https://github.com/Crivella/test_eb_ep
Other examples can also be found in the test file

Running an EC

Easyconfig

crivella@crivella-desktop:~$ workon easybuild-dev
(easybuild-dev) crivella@crivella-desktop:~$ cd test/easyconfigs/
(easybuild-dev) crivella@crivella-desktop:~/test/easyconfigs$ cat test_test.eb 
easyblock = 'TestEasyBlock'

name = 'test_eps'
version = '1.0.0'

homepage = 'https://localhost:8000'
description = """TEST"""

toolchain = {'name': 'gompi', 'version': '2024a'}

dependencies = [
]

osdependencies = []

sources = []
checksums = []

# skipsteps = ['configure', 'build']
# install_cmd = ':'

# files_to_copy = [(['tmpi'], 'bin')]

# sanity_check_paths = {
#     'files': ['],
#     'dirs': [],
# }

moduleclass = 'debugger'

Running without entrypoints

(easybuild-dev) crivella@crivella-desktop:~/test/easyconfigs$ eb test_test.eb --installpath=`mktemp -d`
== Temporary log file in case of crash /home/crivella/.local/easybuild/logs/easybuild-b0o3evp8.log

WARNING: Index for /home/crivella/Documents/GIT/easybuild-easyconfigs/easybuild/easyconfigs is no longer valid (too old), so ignoring it...

ERROR: Failed to process easyconfig /home/crivella/test/easyconfigs/test_test.eb: Failed to obtain class for TestEasyBlock 
easyblock (not available?): No module named 'easybuild.easyblocks.generic.testeasyblock'

####################################################################
# Show that the package is installed
(easybuild-dev) crivella@crivella-desktop:~/test/easyconfigs$ pip list | grep test
pytest                8.2.1
test_eb_entrypoints   0.1.0      /home/crivella/test/test_eb_ep

Running with entrypoints

####################################################################
# Rerun with entrypoints
(easybuild-dev) crivella@crivella-desktop:~/test/easyconfigs$ eb test_test.eb --installpath=`mktemp -d` --use-entrypoints
== Temporary log file in case of crash /home/crivella/.local/easybuild/logs/easybuild-u3l0_o3j.log
== Running entry point start hook...
Hello, World! ----------------------------------------
Hello, World! ----------------------------------------
Hello, World! ----------------------------------------
Hello, World! ----------------------------------------
Hello, World! ----------------------------------------

WARNING: Index for /home/crivella/Documents/GIT/easybuild-easyconfigs/easybuild/easyconfigs is no longer valid (too old), so ignoring it...


WARNING: Index for /home/crivella/Documents/GIT/easybuild-easyconfigs/easybuild/easyconfigs is no longer valid (too old), so ignoring it...

== processing EasyBuild easyconfig /home/crivella/test/easyconfigs/test_test.eb
== building and installing test_eps/1.0.0-gompi-2024a...
  >> installation prefix: /tmp/tmp.vfsQsDBt6t/software/test_eps/1.0.0-gompi-2024a
== fetching files and verifying checksums...
== ... (took < 1 sec)
== creating build dir, resetting environment...
  >> build dir: /home/crivella/.local/easybuild/build/test_eps/1.0.0/gompi-2024a
== ... (took < 1 sec)
== unpacking...
== ... (took < 1 sec)
== patching...
== ... (took < 1 sec)
== preparing...
  >> loading toolchain module: gompi/2024a
  >> defining build environment for gompi/2024a toolchain
== ... (took < 1 sec)
== configuring...
== Running entry point configure hook...
test_pre_configure called with args: (<test_eb_entrypoints.easyblock.TestEasyBlock object at 0x7de8329c31d0>,) and kwargs: {}
TestEasyBlock: configure_step called.
== Running entry point configure hook...
test_post_configure called with args: (<test_eb_entrypoints.easyblock.TestEasyBlock object at 0x7de8329c31d0>,) and kwargs: {}
== ... (took < 1 sec)
== building...
TestEasyBlock: build_step called.
== ... (took < 1 sec)
== testing...
== ... (took < 1 sec)
== installing...
TestEasyBlock: install_step called.
== ... (took < 1 sec)
== taking care of extensions...
== ... (took < 1 sec)
== restore after iterating...
== ... (took < 1 sec)
== postprocessing...
== ... (took < 1 sec)
== sanity checking...
TestEasyBlock: sanity_check_step called.
== ... (took < 1 sec)
== cleaning up...
== ... (took < 1 sec)
== creating module...
  >> generating module file @ /tmp/tmp.vfsQsDBt6t/modules/all/test_eps/1.0.0-gompi-2024a.lua
== ... (took < 1 sec)
== permissions...
== ... (took < 1 sec)
== packaging...
== ... (took < 1 sec)
== COMPLETED: Installation ended successfully (took 1 secs)
== Results of the build can be found in the log file(s) /tmp/tmp.vfsQsDBt6t/software/test_eps/1.0.0-gompi-2024a/easybuild/easybuild-test_eps-1.0.0-20250619.164447.log
== Build succeeded for 1 out of 1
== Summary:
   * [SUCCESS] test_eps/1.0.0-gompi-2024a
== Temporary log file(s) /home/crivella/.local/easybuild/logs/easybuild-u3l0_o3j.log* have been removed.
== Temporary directory /tmp/eb-26gjnn8r has been removed.

Docs CLI options

Runs with:

--list-easyblocks
--list-toolchains
--show-config

docs options

#####################################################################
# Commands with package installed
(easybuild-dev) crivella@crivella-desktop:~/test/easyconfigs$ eb --show-config
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: configuration file)
#
buildpath              (D) = /home/crivella/.local/easybuild/build
containerpath          (D) = /home/crivella/.local/easybuild/containers
debug                  (E) = True
github-user            (E) = Crivella
installpath            (D) = /home/crivella/.local/easybuild
output-style           (E) = no_color
repositorypath         (D) = /home/crivella/.local/easybuild/ebfiles_repo
robot-paths            (E) = /home/crivella/Documents/GIT/easybuild-easyconfigs/easybuild/easyconfigs, /home/crivella/test/easyconfigs, /home/crivella/test/easyconfigs/LLVMtc
rpath                  (D) = True
sourcepath             (D) = /home/crivella/.local/easybuild/sources
test-report-env-filter (E) = re.compile('^SSH|USER|HOSTNAME|UID|.*COOKIE.*|.*LICENSE.*|.*LICENCE.*|TMUX|NVM|GNOME_|DBUS_')
tmp-logdir             (E) = /home/crivella/.local/easybuild/logs

Hooks from entrypoints (3):
- EntrypointHook <test_eb_entrypoints.hooks:test_pre_configure>
- EntrypointHook <test_eb_entrypoints.hooks:hello_world>
- EntrypointHook <test_eb_entrypoints.hooks:test_post_configure>

Easyblocks from entrypoints (1):
- EntrypointEasyblock <test_eb_entrypoints.easyblock:TestEasyBlock>

Toolchains from entrypoints (5):
- EntrypointToolchain <test_eb_entrypoints.unified:Lompi>
- EntrypointToolchain <test_eb_entrypoints.unified:Lolf>
- EntrypointToolchain <test_eb_entrypoints.unified:LLVMtc>
- EntrypointToolchain <test_eb_entrypoints.unified:Lfbf>
- EntrypointToolchain <test_eb_entrypoints.unified:LFoss>
(easybuild-dev) crivella@crivella-desktop:~/test/easyconfigs$ eb --list-easyblocks | grep Test
|-- TestEasyBlock
(easybuild-dev) crivella@crivella-desktop:~/test/easyconfigs$ eb --list-toolchains | grep lfos
	lfoss: BLACS, FFTW, FlexiBLAS, LLVMtc, OpenMPI, ScaLAPACK
(easybuild-dev) crivella@crivella-desktop:~/test/easyconfigs$ cd ../test_eb_ep/
(easybuild-dev) crivella@crivella-desktop:~/test/test_eb_ep [main ≡]

#####################################################################
# Uninstall the package
(easybuild-dev) crivella@crivella-desktop:~/test/test_eb_ep [main ≡]
$ pip uninstall test_eb_entrypoints
Found existing installation: test_eb_entrypoints 0.1.0
Uninstalling test_eb_entrypoints-0.1.0:
  Would remove:
    /home/crivella/.virtualenvs/easybuild-dev/lib/python3.11/site-packages/test_eb_entrypoints-0.1.0.dist-info/*
    /home/crivella/.virtualenvs/easybuild-dev/lib/python3.11/site-packages/test_eb_entrypoints.pth
Proceed (Y/n)? 
  Successfully uninstalled test_eb_entrypoints-0.1.0

#####################################################################
# Commands without package installed
(easybuild-dev) crivella@crivella-desktop:~/test/test_eb_ep [main ≡]
$ eb --show-config
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable, F: configuration file)
#
buildpath              (D) = /home/crivella/.local/easybuild/build
containerpath          (D) = /home/crivella/.local/easybuild/containers
debug                  (E) = True
github-user            (E) = Crivella
installpath            (D) = /home/crivella/.local/easybuild
output-style           (E) = no_color
repositorypath         (D) = /home/crivella/.local/easybuild/ebfiles_repo
robot-paths            (E) = /home/crivella/Documents/GIT/easybuild-easyconfigs/easybuild/easyconfigs, /home/crivella/test/easyconfigs, /home/crivella/test/easyconfigs/LLVMtc
rpath                  (D) = True
sourcepath             (D) = /home/crivella/.local/easybuild/sources
test-report-env-filter (E) = re.compile('^SSH|USER|HOSTNAME|UID|.*COOKIE.*|.*LICENSE.*|.*LICENCE.*|TMUX|NVM|GNOME_|DBUS_')
tmp-logdir             (E) = /home/crivella/.local/easybuild/logs
(easybuild-dev) crivella@crivella-desktop:~/test/test_eb_ep [main ≡]
$ eb --list-easyblocks | grep Test
(easybuild-dev) crivella@crivella-desktop:~/test/test_eb_ep [main ≡]
$ eb --list-toolchains | grep lfos

verdurin · 2025-06-20T08:04:06Z

I like the idea - anything that makes it potentially easier to customise EB is a good thing.

- Code cleanup - Better function names - Improved tests/comments/docstrints

Crivella force-pushed the feature-entrypoints branch from 9d3321c to c21684c Compare June 10, 2025 16:08

boegel added the enhancement label Jun 12, 2025

boegel added this to the 5.x milestone Jun 12, 2025

Crivella force-pushed the feature-entrypoints branch 2 times, most recently from 72d2804 to 35e1919 Compare June 12, 2025 09:08

Crivella force-pushed the feature-entrypoints branch from 1fabe4c to f8f63d9 Compare June 17, 2025 10:07

boegel changed the title ~~Proposed changes to make EasyBuildd plugin-able through entrypoints~~ Proposed changes to make EasyBuild plugin-able through entrypoints Jun 18, 2025

Crivella and others added 10 commits June 18, 2025 12:44

New files + modifications for entrypoint features

773dd67

Lint and cleanup

641c8f7

Replace _log.error wiht warning

45ef374

Olden typehints to make CI happy and add guard against missing `impor…

dff3af7

…tlib.metadata` in old python

Fix for Python<3.10

8b4d29f

Removed EntryPoint typehints

575cfdb

More fixes for python versions

165ef96

Improved version check logic

0edd625

WIP - adding tests

1a5ea64

lint

b0aeb73

Crivella force-pushed the feature-entrypoints branch from 5e3953c to b0aeb73 Compare June 18, 2025 10:45

Fixes undefined Distribution

46af0e9

Crivella force-pushed the feature-entrypoints branch from 0e0cc87 to 46af0e9 Compare June 18, 2025 10:52

Crivella added 5 commits June 18, 2025 14:17

Better tearDown

6a78c9b

- Reworked the way entrypoints are registered and recalled

05ec7ce

- Added listing of entrypoints available through `--show-config` - Enhanced test cases

lint

1a8e270

Lint and fixes

6a8346f

Cleanup and enhanced test-cases to run correct hooks in correct order

6f017f2

Crivella mentioned this pull request Jun 19, 2025

Added documentation for entrypoints easybuilders/easybuild-docs#331

Draft

lint

5224113

Crivella marked this pull request as ready for review June 19, 2025 14:08

ocaisa mentioned this pull request Jun 20, 2025

We should somehow have CI for our eb_hooks.py EESSI/software-layer-scripts#19

Open

Crivella added 2 commits June 20, 2025 14:02

- Readded typehints

08f096a

- Code cleanup - Better function names - Improved tests/comments/docstrints

Lint and improved validation

2d33f2b

boegel modified the milestones: 5.x, release after 5.1.1 Jul 1, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Proposed changes to make EasyBuild plugin-able through entrypoints #4918

Proposed changes to make EasyBuild plugin-able through entrypoints #4918

Uh oh!

Crivella commented Jun 10, 2025 •

edited

Loading

Uh oh!

boegel commented Jun 12, 2025

Uh oh!

Crivella commented Jun 12, 2025

Uh oh!

Crivella commented Jun 19, 2025

Uh oh!

Crivella commented Jun 19, 2025

Uh oh!

verdurin commented Jun 20, 2025

Uh oh!

Uh oh!

Proposed changes to make EasyBuild plugin-able through entrypoints #4918

Are you sure you want to change the base?

Proposed changes to make EasyBuild plugin-able through entrypoints #4918

Uh oh!

Conversation

Crivella commented Jun 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Possible upsides

Possible downsides

TODO

MAYBE

Notes

Uh oh!

boegel commented Jun 12, 2025

Uh oh!

Crivella commented Jun 12, 2025

Uh oh!

Crivella commented Jun 19, 2025

Uh oh!

Crivella commented Jun 19, 2025

What it looks like

Running an EC

Docs CLI options

Uh oh!

verdurin commented Jun 20, 2025

Uh oh!

Uh oh!

Crivella commented Jun 10, 2025 •

edited

Loading