Fleet server standalone crash on startup if es return 429

[nchaulet]

Description

It seems Fleet server standalone crash during startup if elasticsearch return a 429 (could happen in serverless if the proxy refuse the connetion)

It's probably something we should handle, and recover from.

[michel-laterman]

@nchaulet, I've added e2e tests for a 429 response (and timeout) from ES when fleet-server initializes, but have not reproduced the crash. Do we know anything else about what caused the issue?

[nchaulet]

@nchaulet, I've added e2e tests for a 429 response (and timeout) from ES when fleet-server initializes, but have not reproduced the crash. Do we know anything else about what caused the issue?

No other than the logs there was nothing else, (it seems it only happens one time, so we may want to close this and re-open if this happen more frequently)

[michel-laterman]

Sounds good, I think we should get the e2e tests merged too so we can make sure we don't introduce a regression that causes this more often