first draft for instructions how to run planemo and submit jobs to a slurm cluster #1512
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
I have problems running planemo on bridges2. Every things behaves really slow and I get the following error: File "/ocean/projects/mcb140028p/psmeds/test_cwl/planemo_venv/lib64/python3.11/site-packages/planemo/galaxy/serve.py", line 66, in _serve
config.install_workflows()
File "/ocean/projects/mcb140028p/psmeds/test_cwl/planemo_venv/lib64/python3.11/site-packages/planemo/galaxy/config.py", line 848, in install_workflows
self._install_workflow(runnable)
File "/ocean/projects/mcb140028p/psmeds/test_cwl/planemo_venv/lib64/python3.11/site-packages/planemo/galaxy/config.py", line 854, in _install_workflow
workflow_repos = install_shed_repos(
^^^^^^^^^^^^^^^^^^^
File "/ocean/projects/mcb140028p/psmeds/test_cwl/planemo_venv/lib64/python3.11/site-packages/planemo/galaxy/workflows.py", line 118, in install_shed_repos
install_results = install_tool_manager.install_repositories(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ocean/projects/mcb140028p/psmeds/test_cwl/planemo_venv/lib64/python3.11/site-packages/ephemeris/shed_tools.py", line 236, in install_repositories
result = self.install_repository_revision(repository, log)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ocean/projects/mcb140028p/psmeds/test_cwl/planemo_venv/lib64/python3.11/site-packages/ephemeris/shed_tools.py", line 518, in install_repository_revision
success = self.wait_for_install(repository=repository, log=log, timeout=3600)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ocean/projects/mcb140028p/psmeds/test_cwl/planemo_venv/lib64/python3.11/site-packages/ephemeris/shed_tools.py", line 554, in wait_for_install
installed_repos = self.tool_shed_client.get_repositories()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ocean/projects/mcb140028p/psmeds/test_cwl/planemo_venv/lib64/python3.11/site-packages/bioblend/galaxy/toolshed/__init__.py", line 46, in get_repositories
return self._get()
^^^^^^^^^^^
File "/ocean/projects/mcb140028p/psmeds/test_cwl/planemo_venv/lib64/python3.11/site-packages/bioblend/galaxy/client.py", line 134, in _get
raise ConnectionError(
bioblend.ConnectionError: HTTPConnectionPool(host='localhost', port=9090): Max retries exceeded with url: /api/tool_shed_repositories (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x15008bf47510>: Failed to establish a new connection: [Errno 111] Connection refused')), 0 attempts left: |
|
That does sound like a bridges problem ... annoying, but not sure we can do anything about this. |
mvdbeek
reviewed
Jun 19, 2025
| Slurm | ||
| ~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| To run the workflow on a SLURM cluster, you’ll need to configure the ``job_config.yaml`` file to set up a job |
There was a problem hiding this comment.
Can you use https://github.com/galaxyproject/galaxy-job-config-init to set up the job config ?
There was a problem hiding this comment.
Well we added a Planemo command to wrap that with Planemo-style arguments and such - I would use planemo job_config_init to template this? xref https://github.com/galaxyproject/planemo/blob/master/planemo/commands/cmd_job_config_init.py. There is a profile equivalent also.
mvdbeek
reviewed
Jun 19, 2025
| Troubleshooting | ||
| ------------------ | ||
|
|
||
| **Temp direcory not shared between nodes** |
There was a problem hiding this comment.
This shouldn't be necessary with tmp_dir: true, the default
This was referenced Jun 19, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These instructions have currently been tested on:
by smeds