Skip to content

Conversation

Keno
Copy link
Member

@Keno Keno commented Aug 30, 2025

As I proposed in #561 (comment). Unless I'm missing something, this should fix #561. Written by Claude.

This commit removes the lock file mechanism and replaces it with atomic file operations using temporary files for safer concurrent access.

Changes:

  • Remove cluFlock dependency from Cargo.toml
  • Remove lock file creation and management code
  • Implement atomic save using tempfile's NamedTempFile
  • Write to temporary file then atomically rename to target
  • Simplify load_config_db by removing lock parameter
  • Update all callers to use new API

The new approach writes configuration changes to a temporary file in the same directory, then atomically renames it to replace the original file. This ensures readers always see either the complete old version or complete new version, never partial writes.

🤖 Generated with Claude Code

@IanButterworth
Copy link
Member

This seems like an improvement for launching Julia, but if two juliaup add/up processes are running at the same time only one of those actions could end up installed? Though that seems far less common. For that we could add an install lock that the launcher doesn't take?

@Keno
Copy link
Member Author

Keno commented Sep 6, 2025

I'm ok with taking a lock file for write operations to ensure sequential ordering. Although I also think it's probably rare in practice to be an issue. Might as well serialize though.

@Keno
Copy link
Member Author

Keno commented Sep 6, 2025

That said, I don't see a good reason to use a separate lock file in conjuction with posix advisory locks here. We should either be using pid files like julia precompilation does, or just use fcntl advisory locks (regular advisory locks on windows) on the file itself.

This commit combines atomic file operations with exclusive file locking
to ensure safe concurrent access to configuration files.

Changes:
- Keep atomic file operations (write to temp file, then rename)
- Add exclusive locking on config file during write operations
- Lock is acquired in load_mut_config_db before reading
- Lock is held throughout the read-modify-write cycle
- Lock is automatically released when JuliaupConfigFile is dropped
- Removes separate lock file mechanism

The approach provides two layers of safety:
1. Exclusive locking ensures write operations are sequenced
2. Atomic rename ensures readers never see partial writes

This eliminates race conditions while keeping the code simple.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@Keno Keno force-pushed the atomic-file-operations branch from f56adc9 to e88083c Compare September 6, 2025 13:27
@Keno
Copy link
Member Author

Keno commented Sep 6, 2025

I've asked Claude to update this to lock on the config file itself.

@Keno
Copy link
Member Author

Keno commented Sep 6, 2025

Alright, that doesn't quite work that way on windows. I thought it would be quick, but it's not, so I'll need to revisit later when I have more time.

@davidanthoff
Copy link
Collaborator

I'm fairly positive that things will break if we move to a last-write-wins strategy. The config file is not just updated by explicit commands like juliaup add but also with things like last update time etc and those happen frequently.

Having said that, it would still be beneficial to move to an atomic file update strategy (and retain the locking story) simply to deal with things like unexpected process termination in the middle of the write etc.

@davidanthoff
Copy link
Collaborator

That said, I don't see a good reason to use a separate lock file in conjuction with posix advisory locks here. We should either be using pid files like julia precompilation does, or just use fcntl advisory locks (regular advisory locks on windows) on the file itself.

I'm trying to remember why I used this separate file there... I think there was some cross-platform twist where the code would have had to be different on Windows vs the rest if we took the lock on the config file itself, and then it was easier to just go with this separate file story? But I don't really remember, it might also have been a different reason. I did have an implementation originally that locked the config file itself and then ran into some kind of problem with that...

@MilesCranmer
Copy link
Member

@Keno Could you add tests to this? For file locking you can have a few threads run juliaup operations in parallel, and verify that without locking (or atomics) it gets corrupted, but with locking/atomics, it is not.

I do something like this to test rip2's file lock, in case you want to borrow stuff: https://github.com/MilesCranmer/rip2/blob/8bf228f97e069c17b5f24cd4181b36060f637549/tests/integration_tests.rs#L1094-L1181

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Don't lock the juliaup config file when only reading
4 participants