[Request] Consider using minikube for CI

[tthvo]

Describe the feature

Seems like we can use minikube in CI instead of kind. This should give us a more reliable cluster and easy way to test ingress hostname on k8s.

Tutorial: https://minikube.sigs.k8s.io/docs/tutorials/setup_minikube_in_github_actions/
Github Action: https://github.com/medyagh/setup-minikube

Anything other information?

I am suggesting this as I am currently having experiencing a very flanky (faulty?) nginx ingress controller with kind when working on #698 . The request seems to hang from time to time. Even with tls disabled, the problem still persists.

$ curl -L -k -H 'HOST: testing.cryostat' https://ingress-nginx-controller.ingress-nginx.svc/health -v --trace-time
03:55:20.493697 * Host ingress-nginx-controller.ingress-nginx.svc:443 was resolved.
03:55:20.493773 * IPv6: (none)
03:55:20.493788 * IPv4: 10.96.74.150
03:55:20.493831 *   Trying 10.96.74.150:443...
03:55:20.494022 * Connected to ingress-nginx-controller.ingress-nginx.svc (10.96.74.150) port 443
03:55:20.497414 * ALPN: curl offers h2,http/1.1
03:55:20.497678 * TLSv1.3 (OUT), TLS handshake, Client hello (1):

A "compressed" log is shown below when inspecting ingress controller pod.

[alert] 29#29: sendmsg() failed (9: Bad file descriptor)
[alert] 29#29: fork() failed while spawning "worker process" (11: Resource temporarily unavailable)
[alert] 1258#1258: pthread_create() failed (11: Resource temporarily unavailable)
[alert] 28#28: worker process 1067 exited with fatal code 2 and cannot be respawned

Moving to minikube seems to solve the issue.

[tthvo]

easy way to test ingress hostname on k8s.

By patching /etc/hosts with $(minikube ip) testing.cryostat, we no longer need to specify Host header or workaround to send API request directly to ingress controller service. I suppose this should be possible on Github runner :D

[ebaron]

Thanks anyways for taking the initiative!

[tthvo]

Ahha thanks! Tho, I kinda like how neatly minikube can be configured and the variety of cluster configs it offers...