Slim down the invocation image and perform multistage builds (Base image + mixins)

[MChorfa]

Backgound

The bundles image size is getting bigger and bigger proportional to the complexity of the bundles and the numbers of mixins and dependencies required for a given bundle. Also Porter builds time increase as the instructions increase

Related issues

• Use Debian-slim Use debian-slim #1049
• Pluggable Build Driver Pluggable Build Driver #361
• Howto: Porter credentials and Docker secrets Howto: Porter credentials and Docker secrets #1057 could be enabled by using BUILD KIT
• Toggle buildkit features: Toggle buildkit features in porter.yaml #1268

Suggestions

• Use a slim distribution for the base porter image
• Use Multistage techniques to build the invocation image
• Add a new driver to build the invocation image with Buildkit or Podman or Shipwright
• Modify the default Dockerfile to inform the docker engine that we are using Multistage
• Handle the #MIXIN_STAGE tag coming from the mixin, where this tag indicates what the mixin cares about
• Add driver flag to Porter to indicate which driver porter have to build the image with
• Modify porter build provider to generate the Multistage Dockerfile

Example of porter command building with a buildkit driver:

DOCKER_BUILDKIT=1
porter build --driver=buildkit

Example of kubectl mixin definition:

RUN apt-get update && \
apt-get install -y apt-transport-https curl && \
curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectl && \
mv kubectl /usr/local/bin && \
chmod a+x /usr/local/bin/kubectl

#MIXIN_STAGE /usr/local/bin/kubectl

Example of an invocation image definition before interpolation with buildkit:

FROM debian:buster-slim as base

ARG BUNDLE_DIR

RUN apt-get update && apt-get install -y ca-certificates


RUN apt-get update && \
apt-get install -y apt-transport-https curl && \
curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectl && \
mv kubectl /usr/local/bin && \
chmod a+x /usr/local/bin/kubectl


#MIXIN_STAGE /usr/local/bin/kubectl


COPY . $BUNDLE_DIR

Example of a invocation image definition after interpolation with buildkit:

# syntax=docker/dockerfile:experimental

FROM Debian:buster-slim as base

RUN apt-get update && apt-get install -y ca-certificates

RUN apt-get update && \
apt-get install -y apt-transport-https curl && \
curl -o kubectl https://storage.googleapis.com/kubernetes-release/release/v1.19.0/bin/linux/amd64/kubectl && \
mv kubectl /usr/local/bin && \
chmod a+x /usr/local/bin/kubectl

...

FROM debian:buster-slim

ARG BUNDLE_DIR

COPY --from=base /usr/local/bin/kubectl /usr/local/bin/kubectl

COPY . $BUNDLE_DIR

Sequence

Porter Drivers

References

• https://docs.docker.com/develop/develop-images/build_enhancements/
• https://github.com/kubernetes-sigs/kind/tree/master/pkg/cluster/internal/providers
• https://www.docker.com/blog/advanced-dockerfiles-faster-builds-and-smaller-images-using-buildkit-and-multistage-builds/
• https://github.com/shipwright-io/build
• https://github.com/containers/podman

[vdice]

Great ideas here @MChorfa , thanks for compiling. The references were helpful for me to get up to speed.

Besides the initial task of using a smaller base image, am I correct in sensing three major themes (and the benefits/functionality they bring)?

1. Support multi-stage builds

   • cut down overall image size

2. Pluggable build drivers

   • enable building images with alternatives to the local/host docker daemon, such as in Kubernetes (shipwright), daemonless (podman), Cloud Shell, etc.

3. Enable buildkit functionality

   • different frontend support
   • build secrets
   • ssh mounts
   • build cache import/export

Have I captured the high-level themes/goals accurately?

[MChorfa]

@vdice YAS! This is exactly what I was trying to propose. thank you for putting this is in a such wonderful clear way 😄

[carolynvs]

I love all these themes! 💯

We are just starting to work out a proposal process but all of these seem like perfect candidates for PEPs. Each one of these items would benefit from planning out ahead of time how it will work, in which cases they won't (e.g. answering questions such as can we switch from using debian once we have of multi-stage builds?), and considering impact to mixin creators and bundle authors.

[carolynvs]

I have been playing with getting porter to build with buildkit enabled. Right now it's working with docker + buildkit libraries but I think that we should consider shelling out to build tools such as buildx under the hood instead of integrating through libraries. It would require distributing the binary with porter (or providing a way to install through plugins), but is probably worth it because these tools don't always have a library to use (e.g. buildx doesn't export functions that we can use).

I think the next step is a PEP that just covers porter build --driver. This would include:

• porter build --driver where driver uses our configuration system
• moving the current implementation into a "docker" driver
• an implementation that enables buildkit features
• plans for distributing companion binaries for the drivers, e.g. buildx
• making the buildkit enabled driver the default if available