Skip to content

Bugfix/1827 invalid cross device link #1829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

siemdejong
Copy link
Contributor

Fixes #1827

@siemdejong siemdejong marked this pull request as ready for review April 26, 2025 08:08
@siemdejong siemdejong marked this pull request as draft April 27, 2025 05:32
@siemdejong
Copy link
Contributor Author

siemdejong commented Apr 28, 2025

Creating the temporary file next to the target path (making sure it's on the same device) solves this issue. Took inspiration from

fd, fn = tempfile.mkstemp(
dir=os.path.dirname(path), prefix=os.path.basename(path) + "-"
)

This seems to work as expected with shutil.move.

Then os.chmod may raise PermissionError too, so that error can be catched.

It is difficult to mock the cross-device issue as it requires a temporary file on another device than the target device, so I mock this as an EXDEV at os.rename in shutil.move and then a PermissionError at shutil.move -> shutil.copystat -> lookup('chmod').

@siemdejong
Copy link
Contributor Author

Why do the tests fail? These tests sporadically fail locally, but not always. And I get these errors also sporadically on master.

@siemdejong siemdejong marked this pull request as ready for review April 28, 2025 06:44
@siemdejong
Copy link
Contributor Author

I'm not sure if this is the cleanest way to do it. I'm also not sure if os.path.exists() creates too much overhead, but I think not doing this check may result in catching permission errors that were not from chmod, but from creating files in shutil.move.

@siemdejong siemdejong requested a review from martindurant April 28, 2025 20:34
@martindurant martindurant merged commit 372ef69 into fsspec:master May 1, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Invalid cross-device link
2 participants