Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

malformed docs in bulk requests interfere with normal operations #3160

Open
michel-laterman opened this issue Dec 11, 2023 · 0 comments
Open
Labels
bug Something isn't working Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team

Comments

@michel-laterman
Copy link
Contributor

fleet-server makes a few different bulk reads where all documents retrieved are immediately parsed to structs defined in model/schema.json. For example, when we query for the latest policies or handling actions.

If a single document fails to unmarshal correctly, we return nil, err even if it's a single doc out of a large set.
This will stop the fleet-server from processing policies or actions, which could result in agents going offline.
For example, an agent with the policy A checks in, policy A is queried and parsed correctly, yet policy B fails to parse due to an incorrect attribute, fleet-server will return a 503 error to the agent.

We should change how bulk returns are handled to either:

  1. log and ignore the parsing error
    or
  2. return the parsed hits with any errors (using errors.Join) and have the callers handle these cases
@michel-laterman michel-laterman added bug Something isn't working Team:Fleet Label for the Fleet team labels Dec 11, 2023
@jlind23 jlind23 added Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team and removed Team:Fleet Label for the Fleet team labels Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

No branches or pull requests

2 participants