Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add configurable retry policy for GCS integration connection failures #11580

Open
cpascale43 opened this issue Oct 29, 2024 · 0 comments
Open
Assignees
Labels
Integration:google_cloud_storage Custom GCS (Google Cloud Storage) Input

Comments

@cpascale43
Copy link

Currently, when the GCS integration encounters connection issues, it has a fixed retry policy of 2 retries (3 total attempts) for file downloads. This is not configurable by users, which can lead to integration failures when encountering temporary network instability.

Example error:

elastic_agent.filebeat [elastic_agent.filebeat][error] Input 'gcs' failed with: input.go:130: input gcs-google_cloud_storage failed: read tcp: connection reset by peer

Investigation is needed to:

  • Determine if adding configurable retry settings is possible
  • Assess which retry parameters could be exposed/which aspects of retry behavior are controlled by Elastic vs. GCS

See: https://github.com/elastic/enhancements/issues/22943

@cpascale43 cpascale43 added the Integration:google_cloud_storage Custom GCS (Google Cloud Storage) Input label Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Integration:google_cloud_storage Custom GCS (Google Cloud Storage) Input
Projects
None yet
Development

No branches or pull requests

2 participants