CONVERTER_uncompressed_to_gz writes to input dir

[natefoo]

Describe the bug
CONVERTER_uncompressed_to_gz uses bgzip to create a compressed version of the input, and writes that to the correct output path, but it also generates an index that is written next to the input because of the use of the -i flag, which fails when the input dir is not writeable (such as when running in a container). Per the bgzip documentation:

-c, --stdout Write to standard output, keep original files unchanged.

-i, --index Create a BGZF index while compressing. Unless the -I option is used, this will have the name of the compressed file with .gzi appended to it.

-I, --index-name FILE Index file name.

We are using the -c option to write to stdout, and bgzip has no knowledge of what the compressed filename will be in this case, and is using what it would be if the file were being compressed in-place (without -c). We need to add the -I option to set the filename if we actually want the index file. But - do we? If we do, it should probably be a MetadataFile.

Galaxy Version and/or server at which you observed the bug
22.01.1.dev0 (da5894f)

To Reproduce
Steps to reproduce the behavior:

1. Go to usegalaxy.org
2. Run MiModD VCF Filter on an uncompressed vcf
3. See error

Expected behavior
Dataset is correctly converted to compressed vcf

Screenshots
Command line:

cp '/corral4/main/jobs/042/665/42665018/configs/tmpy0nm2mle' 'galaxy.json' && bgzip -@ ${GALAXY_SLOTS:-1} -ci '/corral4/main/files/043/259/dataset_43259150.dat' > '/corral4/main/jobs/042/665/42665018/outputs/galaxy_dataset_1465c575-7505-4eb5-9bda-fdbb371c80dd.dat'

Error:

[E::bgzf_index_dump] Error opening /corral4/main/files/043/259/dataset_43259150.dat.gz.gzi : Read-only file system
Could not write index to '/corral4/main/files/043/259/dataset_43259150.dat.gz.gzi'
[E::bgzf_index_dump] Error opening /corral4/main/files/043/259/dataset_43259150.dat.gz.gzi : Read-only file system
Could not write index to '/corral4/main/files/043/259/dataset_43259150.dat.gz.gzi'

Additional context
N/A

[wm75]

Thanks for digging into this @natefoo !

Just a minor correction to the Steps to reproduce:
2. Got to Edit attributes for any VCF dataset (MiModD VCF Filter was just a tool that had produced a VCF in one of my histories)
3. Go to Convert and select "vcf_bgzip"

[wm75]

The same would happen with implicit conversion for tools that accept only input format "vcf_bgzip", just that I don't know of any such tool on usegalaxy.org.

[wm75]

For the specific vcf_bgzip branch of the converter I'm pretty sure we do not want the index anyway because

galaxy/lib/galaxy/datatypes/tabular.py

Line 1043 in 9143dd7

def set_meta(self, dataset, **kwd):

builds a tabix .tbi metadata index for the datatype already.

I'm not sure what other datatypes would use the rather general "uncompressed file to compressed" converter though and whether the index would make sense for some of them.
Could be a question for @bernt-matthias ?

[bernt-matthias]

My intuition tells me that you are right @wm75 .. I also guess that creating the index would be meaningless if Galaxy is configured to output to the working dir, or?

Lets also ask @gallardoalba, who contributed the converter...

[gallardoalba]

Yes, I agree with @wm75 too.

[natefoo]

So then I believe all that is needed for the fix will be to drop -i from the command line?

[bernt-matthias]

Would be nice to have galaxyproject/planemo#1199 (and/or galaxyproject/planemo#1190 .. but I'm not that convinced of this one).