Clean up MemoryCache and fix thread-safety issues #12440
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Mark as sealed and change protected members to private
- Change "_dict" from "IDictionary<TKey, TValue>" to "ConcurrentDictionary<TKey, TValue>" and rename to "_map".
- Rename "CacheEntry" to "Entry"
- Change "Entry" to protect its data
- Change "Entry.LastAccess" to get and set its data using Volatile.Read/Write to ensure that other threads don't get stale values due to instruction reordering.
- Change "Compact" method to "CompactIfNeeded" and make it acquire the lock.
- Use double-checked locking for compaction
- Use the "SelectAndOrderByAsArray" to avoid the extra allocation of LINQ "OrderBy" followed by "ToArray"
- Update compaction to be thread-safe by...
1. Capture the value of LastAccess when acquiring an ordered array of all items. This is used when removing items to ensure that items touched by other threads during compaction aren't thrown away.
2. Switch from IDictionary<TKey, TValue>.Remove(...) to ConcurrentDictionary<TKey, TValue>.TryRemove(...) for removing items.
3. If we remove an item that was touched during compaction, call TryAdd to restore it.
4. Count the number of items we remove to ensure that we remove *at least* the expected number of items.
- Add comments throughout.
davidwengier
approved these changes
Nov 3, 2025
Member
davidwengier
left a comment
davidwengier
left a comment
There was a problem hiding this comment.
LGTM (but I didn't really look at the new tests. Sorry. Not supposed to be working today anyway 😛)
src/Razor/src/Microsoft.CodeAnalysis.Razor.Workspaces/Utilities/MemoryCache`2.cs
Show resolved
Hide resolved
src/Razor/src/Microsoft.CodeAnalysis.Razor.Workspaces/Utilities/MemoryCache`2.cs
Show resolved
Hide resolved
DustinCampbell
force-pushed
the
clean-up-memorycache
branch
4 times, most recently
from
013e64e to
2b92d41
Compare
Cover more concurrency and edge cases.
DustinCampbell
force-pushed
the
clean-up-memorycache
branch
from
2b92d41 to
00b0d4f
Compare
ToddGrun
reviewed
Nov 5, 2025
src/Razor/test/Microsoft.CodeAnalysis.Razor.Workspaces.Test/Utilities/MemoryCacheTest.cs
Outdated
Show resolved
Hide resolved
ToddGrun
reviewed
Nov 5, 2025
src/Razor/test/Microsoft.CodeAnalysis.Razor.Workspaces.Test/Utilities/MemoryCachePerfTest.cs
Show resolved
Hide resolved
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While reviewing #12434, I noticed that the fix applied doesn't actually fix the race condition in
MemoryCache<TKey, TValue>.Compact(). The reason is subtle! The fix was to avoid enumeratingConcurrentDictionarywhile items could be added and removed by first callingConcurrentDictionary<TKey, TValue>.ToArray(), which is thread-safe. However, the receiver of the call isn't aConcurrentDictionary<TKey, TValue>; it's anIDictionary<TKey, TValue>, which doesn't offer aToArray()method. So, the new call actually goes toEnumerable.ToArray(...), and it's still enumerating the dictionary while items can be added and removed -- it's just doing it earlier than before.To address this, I went ahead and implemented a comprehensive fix for the
MemoryCache<TKey, TValue>thread-safety issues. I also updated the tests to use a test accessor instead of subclassing, which better encapsulates theMemoryCacheimplementation. Many thanks to copilot for helping with comments, documentation, and tests!Here is a summary of the fixes and improvements made to
MemoryCache<TKey, TValue>.