|
| 1 | +--- |
| 2 | +created_at: 2025-11-10 |
| 3 | +tags: |
| 4 | + - workflow |
| 5 | +--- |
| 6 | + |
| 7 | +!!! note "Alternatives" |
| 8 | + See [Nextflow](Nextflow.md) and [Cylc](Cylc.md) for other possible choices of workflow engine. |
| 9 | + |
| 10 | +[//]:snakemake.md> (APPS PAGE BOILERPLATE START) |
| 11 | +{% set app_name = "snakemake" %} |
| 12 | +{% set app = applications[app_name] %} |
| 13 | +{% include "partials/app_header.html" %} |
| 14 | +[//]:snakemake.md> (APPS PAGE BOILERPLATE END) |
| 15 | + |
| 16 | +## Job groups |
| 17 | + |
| 18 | +Please use [Snakemake's job grouping](https://snakemake.readthedocs.io/en/stable/executing/grouping.html) |
| 19 | +functionality to avoid Snakemake submitting many short jobs to the queue. |
| 20 | +Each job submitted to Slurm should take at least 30 minutes. |
| 21 | + |
| 22 | +## Running in an interactive Slurm session |
| 23 | + |
| 24 | +Request an interactive Slurm session with the resources required to run all sub-jobs labeled as `localrules`: |
| 25 | + |
| 26 | +```sl |
| 27 | +srun --account nesi12345 |
| 28 | +``` |
| 29 | + |
| 30 | +Once the interactive session has launched, load the Snakemake module (see above) and you should be able to launch your workflow. |
| 31 | + |
| 32 | +## Submitting a workflow as a Slurm job |
| 33 | + |
| 34 | +As with an interactive session, the main batch job should request the resources needed to run any sub-jobs marked as `localrules`. |
| 35 | + |
| 36 | +```sl |
| 37 | +#!/bin/bash -e |
| 38 | +
|
| 39 | +#SBATCH --job-name my job |
| 40 | +#SBATCH --account nesi12345 |
| 41 | +#SBATCH --time 01:00:00 |
| 42 | +#SBATCH --mem 2G |
| 43 | +
|
| 44 | +# load snakemake |
| 45 | +module load snakemake/{{app.default}} |
| 46 | +
|
| 47 | +# launch snakemake workflow |
| 48 | +snakemake -pr --keep-going -j $SLURM_CPUS_PER_TASK all |
| 49 | +``` |
| 50 | + |
| 51 | +## Slurm plugin |
| 52 | + |
| 53 | +Snakemake maintains a [plugin](https://snakemake.github.io/snakemake-plugin-catalog/plugins/executor/slurm.html) to assist with running Snakemake with Slurm as the executor. |
| 54 | +This plugin supports the use of MPI jobs. |
| 55 | + |
| 56 | +## Community resources |
| 57 | + |
| 58 | +There are two main repositories for reusable Snakemake rules and workflows. |
| 59 | +[Snakemake Wrappers](https://snakemake-wrappers.readthedocs.io/en/stable/) |
| 60 | +provide version controlled rules to conduct common steps in workflows. |
| 61 | +[Snakemake Workflows](https://snakemake.github.io/snakemake-workflow-catalog/index.html) |
| 62 | +are entire Snakemake workflows for common pipelines. |
| 63 | + |
| 64 | +!!! note "Other Resources" |
| 65 | + - [Snakemake Docs](https://snakemake.readthedocs.io/en/stable/) |
| 66 | + - [Snakemake Wrappers Repository](https://snakemake-wrappers.readthedocs.io/en/stable/) |
| 67 | + - [Snakemake Workflows Catalog](https://snakemake.github.io/snakemake-workflow-catalog/index.html) |
0 commit comments