Add ability for users to download EPR image with a subset of integrations

[akshay-saraswat]

Today, the boxed distribution of EPR for air-gapped environments contains all the integrations. In some cases, admins of Elastic deployments may want to ignore some integrations due to security concerns or the size of the EPR image. The ask here is to add support for “streamlined” EPR builds where we let the customer select which integrations and versions they need to host offline/air-gapped.

[mtojek]

1. WDYT about creating a creator that can help you bake a custom image?

It will use the Package Registry base image (just the application) and let use select (with checkboxes?) integrations they like to have.

2. Other possibility would require to create 2-3 distributions of Package Storage, but I guess it might be always a problem what to include.

[ruflin]

I don't think we should allow users to create any combination of images. This will require a complex build system and will make support cases more complex as it is not clear what registry is exactly used. I would also challenge if this is really the flexibility users want. Instead we should offer a 2, max 3 variants for users.

• Minimal: Only contain packages required to run everything like apm, endpoint etc.
• Full: Everything

If someone runs his own registry, one thing that can always be done is running the minimal version and "mount" the necessary packages in.

What exactly is the security concern around the packages? If an admin does not want the users to install certain packages this could also be solved on the Fleet side to have an "allow list" of packages or similar. Then it would be a feature that everyone can use.

[mtojek]

If someone runs his own registry, one thing that can always be done is running the minimal version and "mount" the necessary packages in.

Come on, nobody will play with mounting packages if they are deploying on the Kubernetes cluster.

BTW baking it on their own = we do not support it (unless you will use a creator tool we deliver)

[akshay-saraswat]

What exactly is the security concern around the packages? If an admin does not want the users to install certain packages this could also be solved on the Fleet side to have an "allow list" of packages or similar. Then it would be a feature that everyone can use.

It's not just about security. Elastic stack admins want to have an understanding of and control over what packages are hosted and used for various reasons. "Allow list" on the Fleet side is a great idea but that would not reduce the size of the image which would be a more critical requirement now that we are planning to host ML packages too. In my opinion, either we should provide a few supported variants as @ruflin mentioned or we should provide a creator tool as @mtojek explained.

[QuinnBast]

I would love to see this.

A good solution for this might be to take the grafana route:
Allow users to create a custom dockerfile for the elastic package registry in a non-airgapped environment that downloads or provides only the desired packages.

Something like:

FROM elastic/something/elastic-package-registry-base:someVersion

RUN ./getIntegrations system kafka metrics postgres prometheus

And then the resulting image is a much smaller image with only the desired integrations.