2025.9.0 Issue - Cannot save dask array with compute=False followed by dask.compute #10725
Description
What happened?
I cannot save a netcdf with to_netcdf using either the default engine (netcdf4) nor h5netcdf, when calling with compute=False. The same code is capable of running without issue in 2025.8.0 (using the default engine. I did not test with h5netcdf.). It is also capable of running with compute=True, however this is a regression.
What did you expect to happen?
Output is saved as 'test.nc' without issue.
Minimal Complete Verifiable Example
import xarray as xr
import dask
from dask.distributed import Client
if __name__ == "__main__":
dask.config.set({"num_workers":1})
client = Client(n_workers=1, threads_per_worker=1, memory_limit="1GB", dashboard_address=8785)
ds = xr.tutorial.open_dataset("air_temperature",
chunks={
"lat": "auto",
"lon": 25,
"time": -1,
},
)
d = ds.to_netcdf(f"test.nc", engine='h5netcdf', compute=False)
dask.compute((d,))MVCE confirmation
- Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- Complete example — the example is self-contained, including all data and the text of any traceback.
- Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
- New issue — a search of GitHub Issues suggests this is not a duplicate.
- Recent environment — the issue occurs with the latest version of xarray and its dependencies.
Relevant log output
<sys>:0: SerializationWarning: saving variable air with floating point data as an integer dtype without any _FillValue to use for NaNs
2025-09-09 15:31:40,859 - distributed.worker - ERROR - Compute Failed
Key: ('store-map-87d977c39eb5847280d90f1d66399722-b08e7c87ed6d40fb8ed27c05698a7512', 0, 0, 2)
State: executing
Task: <Task ('store-map-87d977c39eb5847280d90f1d66399722-b08e7c87ed6d40fb8ed27c05698a7512', 0, 0, 2) _identity(...)>
Exception: 'BlockingIOError(11, "Unable to synchronously open file (unable to lock file, errno = 11, error message = \'Resource temporarily unavailable\')")'
Traceback: ' File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/dask/array/core.py", line 4622, in load_store_chunk\n out[index] = x\n ~~~^^^^^^^\n File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/netCDF4_.py", line 86, in __setitem__\n data = self.get_array(needs_lock=False)\n File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/h5netcdf_.py", line 60, in get_array\n ds = self.datastore._acquire(needs_lock)\n File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/h5netcdf_.py", line 227, in _acquire\n with self._manager.acquire_context(needs_lock) as root:\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^\n File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/contextlib.py", line 141, in __enter__\n return next(self.gen)\n File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/file_manager.py", line 199, in acquire_context\n file, cached = self._acquire_with_cache_info(needs_lock)\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^\n File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info\n file = self._opener(*self._args, **kwargs)\n File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/h5netcdf/core.py", line 1552, in __init__\n self.__h5file = self._h5py.File(\n ~~~~~~~~~~~~~~~^\n path, mode, track_order=track_order, **kwargs\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/h5py/_hl/files.py", line 564, in __init__\n fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)\n File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/h5py/_hl/files.py", line 250, in make_fid\n fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)\n File "h5py/_objects.pyx", line 56, in h5py._objects.with_phil.wrapper\n File "h5py/_objects.pyx", line 57, in h5py._objects.with_phil.wrapper\n File "h5py/h5f.pyx", line 102, in h5py.h5f.open\n'
Traceback (most recent call last):
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/file_manager.py", line 211, in _acquire_with_cache_info
file = self._cache[self._key]
^^^^^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/lru_cache.py", line 56, in __getitem__
value = self._cache[key]
^^^^^^^^^^^
KeyError: [<class 'h5netcdf.core.File'>, ('/home/sgammon/projects/portraits/preparation/test.nc',), 'a', (('decode_vlen_strings', True), ('driver', None), ('invalid_netcdf', None)), '1c7be0e3-82bc-4ce1-ae98-82ec7d2f9a29']
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/sgammon/projects/portraits/preparation/minimal_working_example.py", line 18, in <module>
dask.compute((d,))
~~~~~~~~~~~~^^^^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/dask/base.py", line 681, in compute
results = schedule(expr, keys, **kwargs)
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/netCDF4_.py", line 86, in __setitem__
data = self.get_array(needs_lock=False)
^^^^^^^^^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/h5netcdf_.py", line 60, in get_array
ds = self.datastore._acquire(needs_lock)
^^^^^^^^^^^^^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/h5netcdf_.py", line 227, in _acquire
with self._manager.acquire_context(needs_lock) as root:
^^^^^^^^^^^^^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/contextlib.py", line 141, in __enter__
return next(self.gen)
^^^^^^^^^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/file_manager.py", line 199, in acquire_context
file, cached = self._acquire_with_cache_info(needs_lock)
^^^^^^^^^^^^^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info
file = self._opener(*self._args, **kwargs)
^^^^^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/h5netcdf/core.py", line 1552, in __init__
self.__h5file = self._h5py.File(
^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/h5py/_hl/files.py", line 564, in __init__
fid = make_fid(name, mode, userblock_size, fapl, fcpl, swmr=swmr)
^^^^^^^
File "/exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/h5py/_hl/files.py", line 250, in make_fid
fid = h5f.open(name, h5f.ACC_RDWR, fapl=fapl)
^^^^^^^^^^^
File "h5py/_objects.pyx", line 56, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 57, in h5py._objects.with_phil.wrapper
File "h5py/h5f.pyx", line 102, in h5py.h5f.open
BlockingIOError: [Errno 11] Unable to synchronously open file (unable to lock file, errno = 11, error message = 'Resource temporarily unavailable')
2025-09-09 15:31:40,867 - distributed.worker.state_machine - WARNING - Async instruction for <Task cancelled name="execute(('store-map-87d977c39eb5847280d90f1d66399722-b08e7c87ed6d40fb8ed27c05698a7512', 0, 0, 1))" coro=<Worker.execute() done, defined at /exec/sgammon/.conda/envs/portclim/lib/python3.13/site-packages/distributed/worker_state_machine.py:3607>> ended with CancelledErrorAnything else we need to know?
Environment
INSTALLED VERSIONS
commit: None
python: 3.13.7 | packaged by conda-forge | (main, Sep 3 2025, 14:30:35) [GCC 14.3.0]
python-bits: 64
OS: Linux
OS-release: 4.18.0-553.40.1.el8_10.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.6
libnetcdf: 4.9.3
xarray: 2025.9.0
pandas: 2.3.2
numpy: 2.2.6
scipy: 1.16.1
netCDF4: 1.7.2
pydap: 3.5.6
h5netcdf: 1.6.4
h5py: 3.14.0
zarr: 2.18.7
cftime: 1.6.4
nc_time_axis: 1.4.1
iris: None
bottleneck: 1.6.0
dask: 2025.7.0
distributed: 2025.7.0
matplotlib: 3.10.6
cartopy: 0.25.0
seaborn: None
numbagg: None
fsspec: 2025.9.0
cupy: None
pint: 0.25
sparse: 0.17.0
flox: 0.10.6
numpy_groupies: 0.11.3
setuptools: 80.9.0
pip: 25.2
conda: None
pytest: None
mypy: None
IPython: 9.5.0
sphinx: None