[CI] Use containers on Linux

[osteffenrh]

I looked into using containers to run the Ubuntu based CI Jobs on Azure DevOps.

Using containers makes the CI independent from the images Azure DevOps provide.
It also gives the opportunity to include build dependencies and tools, so that
these don't need to be fetched for each job.

Here is a PR demonstrating how it could work:

• [TEST] CI: Use containers for Ubuntu jobs #2628

I built a custom Ubuntu 20.04 image. Using the official plain Ubuntu image does
not work, because it lacks sudo, which is needed to install dependencies at job runtime.
The Dockerfiles are here: https://github.com/osteffenrh/edk2-build-images,
but they should, of course, then move to a repo in the Tianocore group, or something
like that. The images are built automatically via GitHub Actions and then pushed to
ghcr.io.

What do you think? Any comments welcome!

-- Oliver

[spbrogan]

I really like this idea and given the issues we have had pulling the arm/arch64 compiler tools in CI I think this would be more reliable. It would be great if we could add support or new container for RiscV as well. I also wonder if we really need all those packages or more importantly can we trim down the container for CI to improve the bandwidth and build times.

I also wonder if there is an opportunity to add different distros or if that would be valuable to the community. In the past, i had a fedora, container working but would be open to community feedback here.

Have you pulled together the data to know how long it takes to run the build? My expectation is the container download adds some time and unfortunately that occurs before the CI system can evaluate the change to determine if the build even needs to happen. But interested to see real numbers.

[osteffenrh]

I think in the end it would be nice to have all tools included in the
image. I think it would be OK to have one combined image for all
architectures. But it could of course also get split up, if needed.
I'll take a look at the RiscV compiler.

I am thinning out the list of packages right now. The biggest dependency
is mono-devel, which pulls in everything needed for downloading the
external dependencies via Nuget. I am sure it does not need the whole
development package, but I have not investigated yet.

There is a working Fedora 35 image:

• [Test] CI: Use Fedora 35 container #2711

In the RP, in order to use GCC for X64, AARCH64, and ARM form the
distribution, the GCC5_*_PREFIX environment variables are set (currently
in the CI yaml files).
I think stuart still downloads the external dependencies though.

In the pipeline runs for the PRs mentioned here, pulling the image took
about 30 seconds. Container images might even be cached by Azure (?).

Yes, all the pre-checks will now also run inside the container.
There is a way to only run some tasks inside a container, instead of
moving the whole job into it. That way the checks could run on the VM
directly and only the actual builds are done in the container, if they
are needed at all. But that would require more changes.

Building a fresh image takes 4 minutes:
https://github.com/osteffenrh/edk2-build-images/actions

The images (once complete) could also be used by developers to build
EDK2 locally, not only in the CI.

[osteffenrh]

Status update / summary:

• Images now include gcc for all architectures.
• [Test] CI: Use Fedora 35 container #2711
  • Patches on the Mailing list (https://edk2.groups.io/g/devel/message/89058)
• [Test] CI: Use Ubuntu 22.04 container #2752
  • requires -fno-stack-protector flag for gcc, patch
• Fedora 36 image with gcc 12 also exists, but no test-PR on EDK2 yet.

[spbrogan]

github.com/tianocore/containers has been created.

[osteffenrh]

tianocore/containers#1