Implement Mixins as Bundles

[carolynvs]

I have a huge proposal to our mixin infrastructure: implementing mixins as bundles. Mixins would still be executables, and have roughly the same inputs and outputs. However instead of distributing them in binary format, and then building them into each bundle's invocation image, mixins are packaged into bundles, distributed via existing bundle infrastructure (Docker Registries) and execute in isolation within their bundle.

I have a lot of detail worked out for how this needs to be implemented but wanted to keep this overview short to give everyone an idea of the overall direction and impact. I will put an excruciating amount of detail in the PEP to follow. 😅

This would be a fundamental change to how bundles are executed and still stay true to the original intent of mixins (composable bundles). The bundle would be decomposed into separate components that are easier to validate, distribute, reason about provenance, and isolate.

The new bundle manifest would look like this:

Bundle Manifest

• bundle.json (enhanced with information about the workflow for executing the mixins)
• data volume containing files in the bundle, such as your porter.yaml, helper.sh and files shipped with the bundle (/cnab)
• data volume for the execution of Porter's workflow (/porter)
• references to the mixins (which are now bundles)

Pros

• Reduce size of invocation image.
• Optimized for layer caching.
• Reuse infrastructure for distribution and security.
• Firmer assurances of mixin provenance.
• Improved performance for build, publish and run.
• Builds on top of existing mixins.
• Demonstrates novel application of bundles and that we believe in using them ourselves.

Cons

• Requires more development on something that mostly works today.
• Running mixins in a bundle is more complex.
• Needs changes to the CNAB Spec to avoid creating "porter-only" bundles. Namely the CNAB runtime would need to define how to execute the bundle in a workflow, passing outputs from mixins into subsequent mixins.

Bundle Size

For thin bundles, the size of the bundle will be significantly reduced because the bundle no longer needs to "re-ship" an os distribution, porter binary, and mixins. It only contains the files unique to the bundle.

For thick (archived) bundles, the mixins would still be included in the archive so the tgz file would remain the same size.

In both cases, the size is less impactful because by separating mixins into bundles, we can take advantage of Docker caching.

CNAB Runtime

The composition of a bundle from other bundles (in Porter's case mixins) is one way to solve the desire for "declarative" bundles, where the bundle doesn't ship unique binaries and logic. Everything about the bundle and what it will do when run is declared in the bundle definition. We would use this to define a "declarative bundle" spec in CNAB, and if well received, as a starting point for discussing CNAB v2.

Changes to Existing Mixin Design

• The mixin code and binaries would remain largely unchanged. It is still adapting a snippet of yaml into (usually) a CLI call.
• The build command is replaced with your own logic in a Dockerfile used to distribute your mixin as a bundle. For example, the helm mixin would take the build command's Dockerfile lines, and use that to create it's own Dockerfile that installs the helm client and copies the mixin binary into the mixin's docker image.
• Switch from communicating with the mixin over stdin to input files
• Outputs would be saved to a different location
• Mixin is distributed via OCI (docker) registries
• PEP #002: Mixin Versioning would still be required to address locking mixins to a particular version. But a lot of the local mixin cache logic would be replaced by using your local docker cache.

Use Cases

This change is designed to address the following use cases:

When I install two bundles that use the same version of Porter, and the same mixins, the second installation should go faster.

I want smaller bundles.

I want to distribute bundles using someone else's infrastructure, instead of figuring out how to host the binaries myself.

I want to be able to verify that the mixins in a bundle came from their stated source.

I want to write mixins that are based on different linux distributions.

I want to use an architecture other than amd64 for the bundle's execution.

I want to limit which mixins can access bundle credentials. For example, only the az mixin should have access to my Azure credentials, not the exec mixin.

Roadmap

I don't have a plan yet for how this fits into our existing roadmap. It's a very large change, one that impacts mixin authors, porter.yaml schema, and Porter's compatibility with the v1 CNAB spec. It solves so many use cases, that I think it's worth working on soon, but how it fits in with our v1 milestone is TBD.

[carolynvs]

Here are some diagrams of what a declarative CNAB bundle could look like. Note that it doesn't have an invocation image and instead the runtime is responsible for reading the workflow definition from the bundle.json and executing it, passing bundle outputs as parameters/credentials to bundles used in the workflow, and mounting shared volumes so that they can share state, such as the original files shipped with the bundle in /cnab, or allowing a tool like porter to manage state with its own volume (/porter).

[carolynvs]

Leaving a note for myself that even if mixins are distributed separately, and not inside a single invocation image, we may still want to let the mixins run a command during build, for example to call terraform init.

[carolynvs]

This proposal has been assigned to PEP # 005

[richbai90]

I was literally about to propose the same idea. Glad I searched here first 😁

[carolynvs]

It is definitely on our roadmap for post v1 and is high on the list because of the performance and security gains.