10X_5v3 support

[nick-youngblut]

Description of feature

According to the docs, 10X_5v2 is supported but not 10X_5v3. Are there plans to support 5' v3?

[lianov]

@nick-youngblut yes! @atrull314 is about to push the upgrades needed for this support into the dev branch and we hope to have it officially released in the near future. However, he can update you here once it is in dev if you would like to give it a try.

One caveat from us is that we actually don't have any 5' v3 data for a full test of this new feature. So if you do already have this data, it would be great to give it a try while it is still in case any unexpected issues arise.

[nick-youngblut]

I see that

scnanoseq/nextflow_schema.json

Line 163 in 500954e

"enum": ["10X_3v3", "10X_5v2"],

has been changed to

scnanoseq/nextflow_schema.json

Line 147 in d420686

"enum": ["cellranger_3_prime", "cellranger_5_prime"],

in https://github.com/nf-core/scnanoseq/tree/feature_5_prime. So, the user will no longer specific a particular version?

[lianov]

@nick-youngblut : no that second link you posted (with ["cellranger_3_prime", "cellranger_5_prime"] is from an older version of the pipeline / pre-release). The new addition will retain ["10X_3v3", "10X_5v2"] but provide additional options for 3' V4 and 5' V3 (e.g.: ["10X_3v3", "10X_3v4", ,"10X_5v2", "10X_5v3"]). It should be in GitHub soon and we can link it here.

[atrull314]

Hi @nick-youngblut,

We’ve got the BLAZE upgrade in our dev branch. If you want to give it a try you should just need to do the following:

1. If you’ve previously downloaded/ran scnanoseq, run the below command. Otherwise, go to step 2.

nextflow pull nf-core/scnanoseq

2. I’ve provided an example command below, update as needed for your dataset and files. The important thing is to include the -r dev as this will use the dev version of scnanoseq

nextflow run nf-core/scnanoseq \
    -r dev \
    --outdir ./results \
    -params-file ./params.yml \
    -c ./custom.conf

The full list of kits supported by the barcode_format parameter in dev are:

• 10X_3v4
• 10X_3v3
• 10X_5v3
• 10X_5v2

Like @lianov mentioned, we currently don't have any test data for the new barcode kits, so haven't been able to do any meaningful tests. So if you do test them out, let us know if you encounter any issues while using them, and we’ll be happy to resolve them the best we can!

[nick-youngblut]

Thanks @atrull314 for letting me know! I gave the dev branch a try with a 10X_5v3 dataset, and it's dying on the FASTQC_NANOPLOT_PRE_TRIM:NANOPLOT process:

Caused by:
  Process `NFCORE_SCNANOSEQ:SCNANOSEQ:FASTQC_NANOPLOT_PRE_TRIM:NANOPLOT (Sph12092024_dTom-HPC)` terminated with an error exit status (2)


Command executed:

  NanoPlot \
       \
      -t 36 \
  
  
  for nanoplot_file in *.html *.png *.txt *.log
  do
      if [[ -s $nanoplot_file ]]
      then
          mv $nanoplot_file Sph12092024_dTom-HPC.raw_$nanoplot_file
      fi
  done
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_SCNANOSEQ:SCNANOSEQ:FASTQC_NANOPLOT_PRE_TRIM:NANOPLOT":
      nanoplot: $(echo $(NanoPlot --version 2>&1) | sed 's/^.*NanoPlot //; s/ .*$//')
  END_VERSIONS

Command exit status:
  2

Command output:
  (empty)

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  usage: NanoPlot [-h] [-v] [-t THREADS] [--verbose] [--store] [--raw] [--huge]
                  [-o OUTDIR] [--no_static] [-p PREFIX] [--tsv_stats]
                  [--info_in_report] [--maxlength N] [--minlength N]
                  [--drop_outliers] [--downsample N] [--loglength]
                  [--percentqual] [--alength] [--minqual N] [--runtime_until N]
                  [--readtype {1D,2D,1D2}] [--barcoded] [--no_supplementary]
                  [-c COLOR] [-cm COLORMAP]
                  [-f [{png,jpg,jpeg,webp,svg,pdf,eps,json} ...]]
                  [--plots [{kde,hex,dot} ...]] [--legacy [{kde,dot,hex} ...]]
                  [--listcolors] [--listcolormaps] [--no-N50] [--N50]
                  [--title TITLE] [--font_scale FONT_SCALE] [--dpi DPI]
                  [--hide_stats]
                  (--fastq file [file ...] | --fasta file [file ...] | --fastq_rich file [file ...] | --fastq_minimal file [file ...] | --summary file [file ...] | --bam file [file ...] | --ubam file [file ...] | --cram file [file ...] | --pickle pickle | --feather file [file ...])
  NanoPlot: error: one of the arguments --fastq --fasta --fastq_rich --fastq_minimal --summary --bam --ubam --cram --pickle --feather/--arrow is required

As you can see, there's no input provided to nanoplot.

The command that I used:

nextflow -c ~/nextflow/slurm.config run main.nf \
   -profile singularity,slurm,chimera \
   -ansi-log false \
   --barcode_format 10X_5v3 \
   --quantifier isoquant \
   --genome_fasta ${REF_DIR}/fasta/genome.fa \
   --gtf ${REF_DIR}/genes/genes.gtf \
   --input ${OUT_DIR}/samples.csv \
   --outdir ${OUT_DIR}

In case you want to see my custom config:

profiles {
    slurm {
        process.executor      = "slurm"
        process.queue         = "cpu_batch"
        process.errorStrategy = "retry"
        process.maxRetries    = 2
        executor.queueSize    = 200
    }
    chimera {
        workDir               = getWorkDir()
        conda.cacheDir        = getCondaCacheDir()
        cache                 = "lenient"
    }
}

def getWorkDir() {
    def userGroup = "id -gn".execute().text.trim()
    def userName = "whoami".execute().text.trim()
    def workDir = "/scratch/$userGroup/$userName/nextflow-work/default"
    return workDir
}

def getCondaCacheDir() {
    def userName = "whoami".execute().text.trim()
    def cacheDir = "/home/$userName/nextflow/conda-cache/default"
    return cacheDir
}

[atrull314]

So I think I may be able to see the issue. What extension do your input files have? Is it like .fq or .fq.gz by chance?

[nick-youngblut]

Yeah, the input is .fq.gz

[atrull314]

Ok yep, I'll push a fix for this and will let you know when its in dev

[atrull314]

@nick-youngblut The fix should be in dev, make sure to run the below command to update your local copy
nextflow pull -r dev nf-core/scnanoseq

And then you should be able to run it like normal

[atrull314]

Hi @nick-youngblut,

Just wanted to follow up on if you've had a chance to fully test/encountered any other issues during your tests?

We’re preparing to go ahead and release the new version as we’ve got other changes and fixes that we want to go ahead and put out there. We’ll of course fix any bugs after release, but wanted to check before we moved forward. And if you haven’t that’s okay as well!

[nick-youngblut]

@atrull314 I've been traveling, but I'm going to run scnanoseq today. I'll let you know how it goes

[nick-youngblut]

It appears that NFCORE_SCNANOSEQ:SCNANOSEQ:SPLIT_FILE is not set to scale to large datasets:

Caused by:
  Process `NFCORE_SCNANOSEQ:SCNANOSEQ:SPLIT_FILE (5prime_cDNA_lane2)` terminated with an error exit status (140)


Command executed:

  split -a10 -l 10 -d --additional-suffix .fastq reads_sup.fq 5prime_cDNA_lane2.
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_SCNANOSEQ:SCNANOSEQ:SPLIT_FILE":
      split: $(echo $(split --version 2>&1 | head -n1 | sed 's#split (GNU coreutils) ##g'))
  END_VERSIONS

Command exit status:
  140

12 GB starting mem is likely not enough:

    withLabel:process_low {
        cpus   = { 2     * task.attempt }
        memory = { 12.GB * task.attempt }
        time   = { 4.h   * task.attempt }
    }

[nick-youngblut]

I'm curious why you are not using seqkit for splitting the fastq files, given that it is specifically designed for such tasks versus split.