[SDK] Migrate to MSBuild SDK

[jviau]

Follow up work to #3119

Will resolve #1252

To improve our SDK experience, we will be migrating to an MSBuild SDK. Our SDK will still be delivered as a nuget package, but how customers reference us will change.

Motivation

The inner-build we perform today is becoming an increasingly challenging pain point for various customers. The problem stems from us performing the generation & restore of that csproj in the build (and not restore) phase. Our SDK as designed today is inherently unable to address this problem. This is due to us being a PackageReference ourselves, and thus our targets do not exist until after restore, and thus we can never reliably influence restore.

Being an MSBuild SDK offers a chance to address this problem. While still a nuget package, we are referenced via the special Sdk element in the csproj. Instead of being resolved during restore, we are resolved before evaluation and our targets will always be available during restore.

Limitations & Challenges

There are several limitations with this approach, some will be on us to solve, and others require work from MSBuild to address.

1. Post-restore hook is inconsistent

Hooking into post-restore is inconsistent depending on how the function csproj is restored. We are seeing if dotnet/nuget/msbuild team is open to making all CLI scenarios work. Additionally, we will fall back to performing inner project generation and restore during build if it did not happen at restore time.

Restore Method	Result
dotnet restore MyFunctionApp.csproj	post-restore hook works
dotnet restore MySolution.sln	Post-restore hook does not run. Customer will need a Directory.Solution.targets which calls our post restore hook.
dotnet restore dirs.proj	Post-restore hook does not run. Customer will need to add a target to dirs.proj file to call our post restore hook.
dotnet restore SomeAppReferencingTheFunctionApp.csproj	Post-restore hook does not run. Customer will need to manually call our post restore hook.
Restore in Visual Studio	Post restore hook does not run. VS restore is very different from CLI restore, we may not be able to support this. We will fallback to restoring during build.

2. Trimming of unused extension packages will be challenging

Today we generated and restore the inner csproj after compilation, so we know exactly what extensions the app does and does not use. We then exclude unused extensions from the inner project to slim down the payload and extension loads at runtime. With the msbuild SDK approach we generate the inner csproj before compilation, so we have no way of knowing what is and isn't used.

Trimming may still be possible by evaluating unused packages after compilation and trying to exclude assemblies directly. However, this approach will have a side effect: unused extensions may still affect the package versions of used extensions and/or their transitive references. Due to these challenges, we will forgo trimming at this time. Possibly including it as an opt-in behavior in the future.

3. Multi-TFM support

Since restore is TFM-agnostic, but package references can be TFM-dependent, we will need to manually perform multi-TFM support. The challenge will be in keeping this performant: the generated project's TFM is static. It does not change. This will mean for the time being we will need to generate 1 extension project per TFM to support distinct inner-restores. This issue: NuGet/Home#12124 will eventually allow us to consolidate to a single generated project, with synthetic TFMs per outer TFM.

Project File Changes

Before

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <OutputType>Exe</OutputType>
    <AzureFunctionsVersion>v4</AzureFunctionsVersion>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.Azure.Functions.Worker" Version="2.0.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="2.0.4" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.ServiceBus" Version="5.22.0" />
  </ItemGroup>

</Project>

After

<Project Sdk="Azure.Functions.Sdk/1.0.0">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <!-- Our core packages will be implicitly included -->
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.ServiceBus" Version="5.22.0" />
  </ItemGroup>

</Project>

In the above chances, many default values are moved into the SDK itself, reducing how much the customer needs to define to get started. Customers are always able to redefine these when necessary.

• Microsoft.Azure.Functions.Worker is implicitly included. Customers can still add a PackageReference to override it.
• All sensible defaults are moved into the SDK
  • Default AzureFunctionsVersion
  • <OutputType>Exe</OutputType>

Refactoring Wins

This refactor has also had some decent wins with improving the entire inner build loop:

1. We no longer use Microsoft.NET.Sdk.Functions at all. Instead, the inner project also uses Azure.Functions.Sdk. During evaluation of the inner project, the SDK will detect it is the generated project (recognized by name "azure_functions.g.csproj") and shift its import graph.
2. We no longer even build the inner project. Instead, we call a target ResolveFunctionsExtensionFiles on it which will return the exact set of files to include in the .azurefunctions folder (and the function.deps.json).
   • This means we have less file copying. We copy directly from the nuget package location on disk instead of the inner projects build output.
3. Generating of extensions.json is now done in the outer project, scanning the resolved extension files directly.
4. Including codes for our logs as appropriate so they work with NoWarn and similar existing MSBuild behaviors.