Skip to content

Fix eng/common cashing #4829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 14, 2025
Merged

Fix eng/common cashing #4829

merged 8 commits into from
May 14, 2025

Conversation

dkurepa
Copy link
Member

@dkurepa dkurepa commented May 9, 2025

@Copilot Copilot AI review requested due to automatic review settings May 9, 2025 15:08
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR addresses caching improvements by updating methods to include an additional repository flag (repoIsVmr), ensuring that caching keys are correctly differentiated for multiple repositories.

  • Updated method signatures and calls to include the repoIsVmr parameter.
  • Adjusted caching key composition in GitHubClient.
  • Introduced an extraneous variable which may need removal.

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/Microsoft.DotNet.Darc/DarcLib/RemoteRepoBase.cs Unused variable introduced in CommitFilesAsync
src/Microsoft.DotNet.Darc/DarcLib/Remote.cs Updated GetFilesAtCommitAsync signature to pass repoIsVmr
src/Microsoft.DotNet.Darc/DarcLib/IRemoteGitRepo.cs Updated interface signature for GetFilesAtCommitAsync
src/Microsoft.DotNet.Darc/DarcLib/GitHubClient.cs Updated method signatures, parameter propagation, and caching key formation
src/Microsoft.DotNet.Darc/DarcLib/AzureDevOpsClient.cs Updated GetFilesAtCommitAsync signature to pass repoIsVmr

@mmitche
Copy link
Member

mmitche commented May 12, 2025

Why would we need to specify isVMR or not? This implies that treeItem.Path is the same between an VMR-sourced arcade update and an arcade-sourced arcade update. Could we instead of treeItem.Path specify a full relative repo path? That way there is never any overlap.

@dkurepa
Copy link
Member Author

dkurepa commented May 12, 2025

Why would we need to specify isVMR or not? This implies that treeItem.Path is the same between an VMR-sourced arcade update and an arcade-sourced arcade update. Could we instead of treeItem.Path specify a full relative repo path? That way there is never any overlap.

we currently don't have it, but I think we could add it to some wrapper while we're traversing the tree, if you think that'd be a better solution

@mmitche
Copy link
Member

mmitche commented May 12, 2025

we're traversing the tree, if you think that'd be a better solution

I think a more general solution would be passing a more complete path down, if it's feasible.

@dkurepa dkurepa changed the title Cache fix 2 Fix eng/common cashing May 13, 2025
@dkurepa
Copy link
Member Author

dkurepa commented May 13, 2025

we're traversing the tree, if you think that'd be a better solution

I think a more general solution would be passing a more complete path down, if it's feasible.

I debugged the code, I got it a bit wrong on my first try. The blob type trees have their full path as the Path property from the root of the tree. So for example the blob tree containing src\arcade\eng\common\core-templates\steps\cleanup-microbuild.yml will have core-templates\steps\cleanup-microbuild.yml as the Path, and the path variable we already have in the method is the true path to eng/common, so it's src/arcade/eng/common for the VMR and eng/common for non VMR, so just adding that to the key makes things work too

// We're adding the full path here because the eng/common files have the same relative path in the VMR
// and in product repos relative to the eng/common folder, and we don't want to get bad cache hits.
// Their full paths are different so this mitigates the problem
return await Cache.GetOrCreateAsync((path, treeItem.Path, treeItem.Sha), async (entry) =>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return await Cache.GetOrCreateAsync((path, treeItem.Path, treeItem.Sha), async (entry) =>
return await Cache.GetOrCreateAsync(($"{path}/{treeItem.Path}", treeItem.Sha), async (entry) =>

we could also do this, but I don't think we need the extra string interpolation, think the above is more efficient.

@michalpavelka michalpavelka enabled auto-merge (squash) May 14, 2025 09:10
@michalpavelka michalpavelka merged commit 16fa965 into dotnet:main May 14, 2025
7 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants