Operator: unhealthy ingesters not leaving the ring

[aleert]

Describe the bug
After network issues within our clusters we found, that ingesters were not able to join ring so we had to manually remove them.
There are multiple issues, describing this behavior, eg #8615 and #14847 .

Suggested fix would be to add autoforget_unhealthy flag to ingester config by default, as there seems to be no downsides for it.

Expected behavior
Unhealthy ingester leaving their ring after a timeout.

Environment:
Kubernetes 1.27

[xperimental]

Hi @aleert ,

Can you provide more information on how to reproduce issue?

I have recently tried to reproduce a very similar report, but for me the ingesters instantly became healthy again after the network issues were removed.