Regular subscriptions to VMR end up pulling eng/common under /src/arcade in regular repos

[premun]

Pull request

dotnet/cecil#310

Type of problem

Unexpected behavior in the PR (e.g. conflicts, EOLs, mangled files, ..)

Description of the issue

The files should go to /eng/common instead of /src/arcade/eng/common

[premun]

Originally, I thought this happened here:

arcade-services/src/Microsoft.DotNet.Darc/DarcLib/Remote.cs

Lines 189 to 199 in c979cdf

	IDependencyFileManager arcadeFileManager = await remoteFactory.CreateDependencyFileManagerAsync(arcadeItem.RepoUri);
	try
	{
	targetDotNetVersion = await arcadeFileManager.ReadToolsDotnetVersionAsync(arcadeItem.RepoUri, arcadeItem.Commit, sourceRepoIsVmr);
	}
	catch (DependencyFileNotFoundException)
	{
	// global.json not found in src/arcade meaning that repo is not the VMR
	sourceRepoIsVmr = false;
	targetDotNetVersion = await arcadeFileManager.ReadToolsDotnetVersionAsync(arcadeItem.RepoUri, arcadeItem.Commit, sourceRepoIsVmr);
	}

but we do not throw DependencyFileNotFoundException when rate limiting happens:

arcade-services/src/Microsoft.DotNet.Darc/DarcLib/GitHubClient.cs

Lines 133 to 136 in c979cdf

	catch (HttpRequestException reqEx) when (reqEx.Message.Contains(((int)HttpStatusCode.NotFound).ToString()))
	{
	throw new DependencyFileNotFoundException(filePath, $"{owner}/{repo}", branch, reqEx);
	}

[premun]

This one does not reproduce anymore and I was not able to figure out how it happened in the first place even. Let's close it and re-open if we see it again

[akoeplinger]

For the record, it happened in dotnet/emsdk#1192 as well. Closing the PR and retriggering the subscription didn't reproduce it though.

[premun]

Thanks for letting us know. I suspect that it might be that when we're verifying existence of some files to tell a repo from a VMR, the request fails for other reasons (e.g. rate limiting) and we cannot distinguish that and then wrongly consider the source repo not a VMR. Reopening

[wtgodbe]

Happened in dotnet/aspnetcore#61878 too

[akoeplinger]

looks like a bunch of repos accidentally merged the wrong src/arcade changes: https://github.com/search?q=org%3Adotnet%20path%3Asrc%2Farcade%2Feng%2Fcommon%2FREADME.md&type=code

[dkurepa]

ouch, looks like we're going to have to remove all of those, don't think they'll be gone after we fix this. Thanks for bringing it up

[dkurepa]

we rolled out the cashing fix last night, this shouldn't happen anymore. We'll now need to cleanup the repos that got the src/arcade/eng/common PRs merged

Tracking issue for the cleanup: #4837