Associate file extensions supported by default in napari (no plugins needed)

[jaimergp]

Closes #215

References used to find predefined mime types and UTIs:

• Use this for Linux: https://mimetype.io/all-types
• Use this for macOS: https://developer.apple.com/library/archive/documentation/Miscellaneous/Reference/UTIRef/Articles/System-DeclaredUniformTypeIdentifiers.html
• The custom ones (e.g. npy, npz, zarr) must be defined in with glob_patterns and UTExportedTypeDeclarations, respectively.

[jaimergp]

Demo this locally:

1. Get the artifacts from https://github.com/napari/packaging/actions/runs/17676123221/artifacts/3996501940 and unzip it
2. Create a new conda environment: conda create -n napari-menu-filetypes napari pyqt
3. Install this PR's napari-menu: conda install -n napari-menu-filetypes path/to/unzipped/artifacts/napari-menu-*.conda
4. Check your system desktop and see if it worked! It works for me on my Mac, arguably the trickiest platform, so hopefully that's the case elsewhere too.

[jaimergp]

napari-menu-0.6.5dev16+ga5632f588-pyh803a8fc_0.conda.zip

Also attached here so no auth is required, remove the .zip extension before using.

[jaimergp]

On Linux, I can get our XPRA image to do this just by installing these shortcuts 🚀

$ xdg-mime query default image/png
napari-065dev16ga5632f588_napari-065dev16ga5632f588.desktop

Now let's see if this flies on Windows. Edit: yes, but I can't figure out how to get a better icon there (turns out to not be possible)

[jaimergp]

Picked a few to get started, but I'd like to know which ones we should include @napari/core-devs

[brisvag]

We generate the full list by running the gen_extension() function from https://github.com/napari/napari/blob/5d6ab46f836a3a4ac97cb3c4324671209b99d6ae/tools/check_extensions.py.

The list is pasted in https://github.com/brisvag/napari/blob/5d6ab46f836a3a4ac97cb3c4324671209b99d6ae/src/napari_builtins/builtins.yaml#L146.

[jni]

I'm a little sad that we can't include bioformats in the extension list.

Since we can't modify this list based on plugins, here's a proposed path forward:

• include in the list the union of extensions declared by readers of all plugins compatible with that version of napari.
• if we can't open a file with what's installed, use npe2 to check at open time which plugins declare compatibility with the current extension, and show the relevant message to the user: "napari can't open this file with the currently installed plugins. However, these plugins may provide readers for the given file: [list of plugins]"
• bonus points if the plugins can directly be installed from that dialog
• super bonus points if readers don't require a restart and we can just open the file directly after the plugin install

What do you think @jaimergp @psobolewskiPhD @Czaki? Does this plan sound feasible?

Overall though — awesome work @jaimergp!!! 😍🤩

[jaimergp]

Since we can't modify this list based on plugins,

Technically we can, but we need to do that from napari itself and it's going to be a lot of code with a lot of hacks for platform-specific configurations that is going to be a pain to maintain. For now we have this static configuration in menuinst.

include in the list the union of extensions declared by readers of all plugins compatible with that version of napari.

That might become a HUGE list, but maybe not? Let's see, I used the attached script to generate this list:

*
*.1sc
*.2fl
*.3ds
*.3fr
*.CUB
*.FITS
*.IMG
*.LBL
*.PTU
*.SDT
*.TCF
*.TIF
*.TIFF
*.XLS
*.acff
*.acqp
*.afi
*.afm
*.aim
*.al3d
*.ali
*.am
*.amiramesh
*.ano
*.apl
*.apts
*.arf
*.array-like
*.arw
*.asc
*.asd
*.avi
*.avs
*.b2nd
*.bay
*.bdf
*.bif
*.bin
*.bip
*.bmp
*.bmq
*.box
*.bre
*.brim.zarr
*.brim.zip
*.bsdf
*.btf
*.bufr
*.bw
*.bwp
*.bz2
*.c01
*.cap
*.cat
*.cbox
*.cells*
*.cfg
*.cgns
*.ch5
*.cif
*.cine
*.coords
*.cr2
*.crw
*.cs
*.cs1
*.csv
*.ct
*.ct.img
*.ctm
*.cub
*.cur
*.cut
*.cxd
*.czi
*.dae
*.dat
*.dato
*.dato.gz
*.db
*.dc2
*.dcm
*.dcr
*.dcx
*.dds
*.df3
*.dib
*.dicom
*.dm2
*.dm3
*.dm4
*.dng
*.drf
*.dsc
*.dti
*.dv
*.e
*.e57
*.ecw
*.eer
*.ele
*.em
*.emd
*.emf
*.eps
*.epsi
*.erf
*.es
*.exo
*.exp
*.exr
*.f3grid
*.fake
*.fbd
*.fbx
*.fdf
*.fem
*.fff
*.ffr
*.fid
*.fit
*.fits
*.flc
*.flex
*.fli
*.fpx
*.frm
*.ftc
*.fts
*.ftu
*.fz
*.fzy
*.fzzy
*.g3
*.gbr
*.gdcm
*.gel
*.geojson
*.gif
*.gii
*.gipl
*.gipl.gz
*.glb
*.gltf
*.grey
*.grib
*.griottes
*.gwy
*.gz
*.h5
*.h5m
*.hdf
*.hdf5
*.hdp
*.hdr
*.hed
*.his
*.htd
*.htm
*.html
*.hx
*.i2i
*.ia
*.ibw
*.icns
*.ico
*.ics
*.ids
*.iff
*.iim
*.iiq
*.im
*.im3
*.img
*.imggz
*.ims
*.imzML
*.inf
*.inp
*.inr
*.ipl
*.ipm
*.ipw
*.j2c
*.j2k
*.jfif
*.jif
*.jng
*.jp2
*.jpc
*.jpe
*.jpeg
*.jpf
*.jpg
*.jpk
*.jpx
*.json
*.jxr
*.k25
*.kc2
*.kdc
*.klb
*.koa
*.l2d
*.labels
*.lbl
*.lbl.json
*.lbm
*.lei
*.lfp
*.lfr
*.lif
*.liff
*.lim
*.lms
*.lof
*.lsm
*.map
*.mat
*.mcd
*.mdb
*.mdc
*.mdpa
*.mea
*.med
*.mef
*.mesh
*.meshb
*.mgh
*.mgh.gz
*.mgz
*.mha
*.mhd
*.mic
*.mkv
*.mnc
*.mnc.gz
*.mnc2
*.mng
*.mod
*.mos
*.mov
*.mp4
*.mpeg
*.mpg
*.mpo
*.mrc
*.mrci
*.mrcs
*.mri
*.mrw
*.msh
*.msp
*.msr
*.mtb
*.mvd2
*.mzarr
*.mzz
*.naf
*.nas
*.nd
*.nd2
*.ndjson
*.ndpi
*.ndpis
*.nef
*.nhdr
*.nia
*.nii
*.nii.gz
*.niigz
*.node
*.npy
*.npz
*.nrrd
*.nrw
*.obf
*.obj
*.obsep
*.ods
*.off
*.oib
*.oif
*.oir
*.ome
*.ome.tif
*.ome.tiff
*.ome.zarr
*.omp2info
*.orf
*.par
*.parquet
*.pbm
*.pcd
*.pcoraw
*.pct
*.pcx
*.pdb
*.pds
*.pef
*.pfm
*.pgm
*.pic
*.picks
*.pict
*.ply
*.png
*.pnl
*.post
*.post.gz
*.ppm
*.pr3
*.preali
*.ps
*.psd
*.ptif
*.ptiff
*.ptu
*.ptx
*.pxn
*.pxr
*.qobj
*.qpi
*.qptiff
*.qtk
*.r3d
*.raf
*.ras
*.raw
*.rcpnl
*.rdc
*.rec
*.res
*.rgb
*.rgba
*.rings*
*.rw2
*.rwl
*.rwz
*.scan
*.scan*
*.scn
*.sdt
*.seg
*.seq
*.sif
*.sld
*.sldy
*.sm2
*.sm3
*.spc
*.spe
*.spi
*.splineit
*.spm
*.sr2
*.srf
*.srw
*.st
*.star
*.sti
*.stk
*.stl
*.stp
*.su2
*.surf
*.svs
*.swc
*.swf
*.sxm
*.targa
*.tbl
*.tbz2
*.tdb
*.temb
*.tf2
*.tf8
*.tfr
*.tga
*.tgz
*.thm
*.tif
*.tiff
*.tiledb
*.tim
*.tloc
*.tmap
*.tnb
*.top
*.topostats
*.tsv
*.txt
*.ugrid
*.v
*.vff
*.vmi
*.vms
*.vol
*.vol.gz
*.vrt
*.vsi
*.vtk
*.vtp
*.vtu
*.vws
*.wap
*.wat
*.wav
*.wbm
*.wbmp
*.wdp
*.webp
*.wkt
*.wlz
*.wmf
*.wmv
*.wpi
*.wrl
*.x3d
*.xbm
*.xdce
*.xdmf
*.xlef
*.xls
*.xlsx
*.xmf
*.xml
*.xpm
*.xqd
*.xqf
*.xv
*.xys
*.xyz
*.yaml
*.ysc
*.zarr
*.zarr*
*.zarr.zip
*.zfp
*.zfr
*.zif
*.zip
*.zpo
*.zvi
*P=*.tif
*RES
*ome.tif
*omero*
.jpeg
.jpg
.png
.tif
.tiff
.x3dv
<EDIT_ME>
TRA
http://*
https://*
s3://*
tiledb://*

Some of them need to be cleaned up, but you get the point.

The other suggestions may be doable with some effort. I guess most of that would be through the plugin manager. Maybe we can have a second plugin that acts as a meta-reader and auto-install+dispatches. Maybe that second plugin is the plugin manager itself. Maybe that's too meta.

[jni]

<EDIT_ME>

😂

[jni]

To me this list seems manageable. We need to get rid of placeholders, possibly URL prefixes but maybe not?

Since the plugin manager infrastructure needs to be updated, I suggest we start with builtins, merge that for 0.6.5, then aim to have the full list with the helpful error messages for 0.7.0. Thoughts?

[jaimergp]

I suggest we start with builtins, merge that for 0.6.5

I think this makes sense even if we don't proceed with the plugin support, so let's make sure we have a list we like here and then I'll open the corresponding PR in napari/napari. Do we want to add ALL of them in #215 (comment), @jni?

[jni]

Do we want to add ALL of them in #215 (comment), @jni?

I don't know. I defer to @psobolewskiPhD @TimMonko. For me, .tiff and .zarr would already be huge. I think it's probably safe to add all of them but that means like 90% of them are effectively untested by us so we just have to trust that things will work? 😬

[TimMonko]

I'm a little sad that we can't include bioformats in the extension list.

Since we can't modify this list based on plugins, here's a proposed path forward:

* include in the list the _union_ of extensions declared by readers of _all_ plugins compatible with that version of napari.

* if we can't open a file with what's installed, use npe2 to check at open time which plugins declare compatibility with the current extension, and show the relevant message to the user: "napari can't open this file with the currently installed plugins. However, these plugins may provide readers for the given file: [list of plugins]"

* bonus points if the plugins can directly be installed from that dialog

* super bonus points if readers don't require a restart and we can just open the file directly after the plugin install

This is awesome, but is it worth the lift? I know some users would think so in the end, but it's quite big it seems.

For me, .tiff and .zarr would already be huge. I think it's probably safe to add all of them but that means like 90% of them are effectively untested by us so we just have to trust that things will work? 😬

I think we should add all supported by builtins. AFAIK, it just means that operating systems recognizes that napari is an option to load the file, it's not like it forces anything. I think it's also a plus that certain file types might not load correctly, maybe users will open an issue to let us know. But in all the years I've survived on open source software I've found many times where extension support doesn't work (often version mismatches and stuff) -- it has seemed to be expected behavior, never fussed me, and I just moved forward knowing the nature of the beast. Maybe I'm an odd duck for having mostly relied on FOSS my whole life.

[jaimergp]

I added all builtin-supported extensions now.

[jni]

but it's quite big it seems.

I actually don't think it's that huge a lift, particularly because it sits "outside" the napari architecture. I think it's a very nice usability enhancement.

[willingc]

Love this! Code looks good but don't have time to pull it locally.

[jaimergp]

I have enabled PR artifacts so it's easy to test locally @willingc. I'll paste the link once available.

[brisvag]

Is this close to ready and 0.6.5 material?

[TimMonko]

tested artifact on Windows. :)
Opened jpg, png, tiff, some ome tiff, npz, and all worked great. I think I was surprised that it opened a new napari each time, but I suppose thats consistent with the behavior of other programs, like its not like word will open a new doc inside the current word.

A csv of rat hematology data (so not like shapes), mp4, mov all were recognized as openable, opened napari, but the image/movie did not load. There was warning given to the user, etc.

I did try to open a tiff with absolutely horrible metadata and napari opened and then quietly crashed -- that seems to be the part thats most problematic (these come from the most pain in the ass proprietary microscope ever)

I don't think this should be a blocker by any means, as I mentioned previously. It's a huge QOL improvement that should help in most expected cases. We should just open issues if we merge before these are resolved. (Perhaps the most confusing "it's broken, I guess, but awesome anyways" approval I've ever done)

[jaimergp]

In the case of macOS, the multiple instances of napari has to do with this issue: napari/napari#8312. Not sure what the equivalent would be on Windows. Maybe is a simple launcher that finds opened napari processes and then dispatches some sort of event to one of them.

[jaimergp]

I did try to open a tiff with absolutely horrible metadata and napari opened and then quietly crashed -- that seems to be the part thats most problematic (these come from the most pain in the ass proprietary microscope ever)

Does that tiff open if invoked from the CLI?

[jni]

Oops, we missed this for 0.6.5. 😅 My proposal is that we merge this and then see whether we can make some progress on #296 over the coming couple of months. This would be an amazing headline feature for napari 0.7.0. 🚀

[jni]

I did try to open a tiff with absolutely horrible metadata and napari opened and then quietly crashed -- that seems to be the part thats most problematic (these come from the most pain in the ass proprietary microscope ever)

Does that tiff open if invoked from the CLI?

And does it open with napari-tiff installed?

[TimMonko]

I'm a bit confused and concerned. I re-downloaded the artifcat (the same one) and I'm struggling to open anything this time. I've only been able to open one tiff. Not even pngs or anything. Everything just quietly exits. Maybe a computer restart will help, but thats a tomorrow problem.

[jaimergp]

Any updates @TimMonko? How did you uninstall the previous one? Did you use the uninstaller? Or did you just install things on top?

[TimMonko]

I have like 2 minutes to reply @jaimergp, apologies
I've been using this over the past week or so because it pops up as the open option on my computer now.
I would say for the most part it seems to work. The surprising thing is sometimes it takes minutes -- i.e., I've went and done many other things on my computer and then napari opens with the image. There are still some silent fails, but for the most part if it opens with the base reader it seems to work.
Also, don't worry about my tiff with bad metadata, the thing is so chaotic it works on like every other tifffile version (it's partly why I keep the dratted thing around) -- haven't tested it again

[jaimergp]

I say we give it a try and merge so we can gather more feedback as people begin to use it, what do you think @TimMonko?

[TimMonko]

I say we give it a try and merge so we can gather more feedback as people begin to use it, what do you think @TimMonko?

Yes, agreed.
@brisvag this should be highlighted in release notes. I think it should also make mention that we need feedback if things aren't working. Maybe we make an issue in advance to link to?

[jaimergp]

I'll let @brisvag merge then!