Why are the samples using /hazelcast/health/node-state for OpenShift/Kubernetes readiness & liveness probe?

[yihtserns]

HttpGetCommandProcessor (assuming this is the one that handles /hazelcast/health/node-state) will return HTTP 503 if and only if NodeState=SHUT_DOWN.

I have a few questions about this:

1. When liveness probe gets HTTP 503, it will then attempt to kill and restart the container in the Pod. Why do we want to restart the container when the NodeState=SHUT_DOWN?
2. Remove readinessProbe form OpenShift templates #241 removed readiness probe from the OpenShift sample, but it was reinstated in the latest OpenShift samples. Why is that?
3. Up until https://github.com/hazelcast/hazelcast-docker/tree/v3.10/hazelcast-kubernetes, there is a readiness.sh that will only mark the Pod as ready, if and only if ClusterSafe=TRUE. Seeing how readiness.sh has been removed & none of the samples are using /hazelcast/health/cluster-safe as readiness probe, does that mean a Hazelcast node is always ready when it has started up?

[leszko]

@yihtserns Thanks for the questions.

1. The livenessProbe should actually point to /hazelcast/health endpoint. There is an open issue for that: Change the default livenessProbe endpoint to /health charts#83

2. To be honest I don't remember why we removed it, but it might have been the problem that we used Deployment which started all PODs at the same time. Now, we recommend using StatefulSet and the readinessProbe should be in place.

3. We actually introduced a new endpoint /hazelcast/health/ready and this should be used for readinessProbe (here is an example).

[yihtserns]

@leszko Thanks for the clarification.