API to write platform specific behaviour

[lahabana]

Problem Statement

We have a few issues that require platform specific behaviour.
While ideally there shouldn't be any, sometimes it is still required.

We should have a component to detect Kubernetes platform information so we can easily write code that's conditional to the platform detected.

Issues that would leverage this component:

• AKS' Admission Enforcer mutates ControlPlane's ValidatingWebhookConfiguration which causes a perpetual reconciliation loop #239
• GKE: Using pod dns names (service scoped) doesn't work on GKE thus preventing ControlPlane -> DP's Admin API traffic #179
• GKE Autopilot: DataPlane and ControlPlane never get marked as ready #888

Proposed Solution

A lot of the detection code already exists in telemetry: https://github.com/Kong/kubernetes-telemetry/blob/a8f70df68515496b91e46e04fead4116e1df25b9/pkg/telemetry/workflowsk8s.go#L32-L63

Some open questions:

• Should we include the version there?
• Should we include os/arch?
• What kind of refresh policy should we adopt here (my guess is that we don't need to as to the controller pod needs to be restarted to get on the latest k8s node version)?

As this detection is based on heuristics it should be possible for users to set it in configuration in case the heuristics are wrong.

Acceptance Criteria

• There's an API in controllers that enables us to write platform specific code without writing the discovery ourselves
• Users can explicitly set their platform parameters and not rely on heuristics

[pmalek]

X-referencing archived issue: https://github.com/Kong/gateway-operator-archive/issues/867.

[programmer04]

It does not need to be implemented consider Close as not planned

For

• AKS' Admission Enforcer mutates ControlPlane's ValidatingWebhookConfiguration which causes a perpetual reconciliation loop #239
• GKE Autopilot: DataPlane and ControlPlane never get marked as ready #888

the approach described in #500 is broader and more robust, and it does not require platform-specific behavior.

For

• GKE: Using pod dns names (service scoped) doesn't work on GKE thus preventing ControlPlane -> DP's Admin API traffic #179

an alternative solution that will always work is proposed in Kong/kubernetes-ingress-controller#7033