APIVIP validation doesn't consider node type

[hardys]

In the image validation we check for the number of nodes, and if it's more than one we enforce configuration of an APIVIP

https://github.com/suse-edge/edge-image-builder/blob/main/pkg/image/validation/kubernetes.go#L52

However it's valid to deploy a single controlplane (type: server) host without any APIVIP, but also define one or more compute (type: agent) hosts.

So I think the validation (and other relevant checks for number of nodes related to configuration/defaults) should consider not only the node list length, but also filter by type.

[atanasdinov]

While it is valid, do we want to do it? Joining a node at a later point in time is much more robust if it's based on a virtual IP backed by MetalLB. I'd generally assume that MetalLB (and Endpoint Copier Operator) do not really bring that much complexity and / or load to the cluster so I went with the assumption that it's better to always use virtual IP even in said architecture models.

[hardys]

@atanasdinov the issue is in many PoC situations you don't control the lab networking, and in that case obtaining an additional address for the VIP is problematic for many users.

So for example you have two machines and you want to create a 1 controlplane and 1 worker/agent, but all IPs are managed via DHCP - with this current validation it's not possible to do that using a single EIB image AFAICS.

[agracey]

Is there any way we could look at using mdns/avahi/zeroconfig to broadcast a join address? It wouldn't be too hard to do but I'm not sure about the security implications

[atanasdinov]

We'll revisit this for the next version of EIB.

[jdob]

One option is to remove the strict validation requirement and simply log a message to the user indicating that they may want to consider an explicit IP. I don't have examples off the top of my head, but I know we do that sort of output for things that are not strictly enforced but probably a good idea.

[atanasdinov]

This will not work. We need another way for nodes to join the cluster if apiVIP is not set. Hence why I bumped this out of v1.1 - it requires too many changes and careful testing.