Azure VM Tasks (non-ACI)

[n1hility]

Not sure if this should be a separate issue than #778. I have a use case that requires nested virt on Windows. GCP's nested virt only supports Linux. EC2 only supports it with cost-prohibitive bare metal nodes. Azure on the other hand supports nested virt on the Dv3 and Ev3 nodes. It would be great to have a direct vm option, similar to ec2-instance that works with azure to make usage of these nodes.

[fkorotkov]

Seems similar to #778. @n1hility, I'm curious what is your use case for nested virtualization on Windows?

[n1hility]

@fkorotkov Sure glad to provide details. I am contributing a patch to the podman project that enables windows support (containers/podman#12503). The implementation is based off of WSL, which internally uses hyper-v to run the linux kernel in a special shared VM. We need the ability to test this implementation as part of pull request checking/CI. AFAICT the only approach that works with Cirrus CI atm is a persistent worker. Alternatively, I suppose it could be possible to doubly nest inside of a google compute L1 linux node with windows running under KVM. Although that introduces some challenges with upkeep on the windows instance.

[fkorotkov]

Hey @n1hility, we are a little overwhelmed with other things like bringing all GCP specific feature for AWS as well to have a feature parity between the clouds. On that note I was wondering if AWS' Metal option will work for you. It seems you are already getting some AWS Graviton instances to your fleet in https://github.com/containers/aardvark-dns.

[n1hility]

@fkorotkov Ok thanks for the update. Unfortunately, metal instances are exceptionally cost-prohibitive. The cheapest Windows metal instance is around 60 times the cost of a standard azure instance (6-7$ an hour vs 10cents for a Dv3 node typical for CI), so not really practical for pull request verification. EC2 Arm metal nodes aren't licensed for/support Windows, Although even if so, the metal nodes are all Graviton v2 / ARMv8.2, so missing FEAT_NV* needed for nested virt.

[fkorotkov]

Got it. Azure instances that is. Once #979 and #900 are closed I promise Azure stuff will be next.

[n1hility]

Awesome! Thanks @fkorotkov!

[n1hility]

hey @fkorotkov, just wondering if this one is still on your roadmap, and if so when you think it might land. Thanks so much!

[evertonlperes]

Hi there,
Just jumping in this thread.
Also, the kvm enabled on Windows runners will be very appreciated. I do have the almost the same scenario as @n1hility. If you need further details, I'll be happy to share @fkorotkov. Thanks!