Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: provide mechanism to test unhappy path behaviour #1941

Open
efd6 opened this issue Jul 3, 2024 · 0 comments
Open

proposal: provide mechanism to test unhappy path behaviour #1941

efd6 opened this issue Jul 3, 2024 · 0 comments
Labels
Team:Ecosystem Label for the Packages Ecosystem team

Comments

@efd6
Copy link
Contributor

efd6 commented Jul 3, 2024

Currently elastic-package test provides mechanisms to test pipeline and integrated source⇒input⇒pipeline tests. These all express tests as a successful outcome, lacking an error.message value. Refining the definition of success behaviour is in #1392.

However, we do not have a way of expressing a test that intends to check the behaviour of an integration for the unhappy path. This is important because good error handling is essential for quickly identifying issues and correcting them; we are in the process of improving the reporting of API and data processing errors to allow more rapid resolution of user issues, but enhancements that we make to this aspect of integrations cannot currently be mechanically tested. Improvements to error handling and reporting require manual alteration of tests and human assessment of behaviour. This doesn't scale and makes missing regressions in behaviour inevitable.

I propose that a mechanism be added to elastic packages testing infrustructure that allows a developer to express the intention that a particular test will "fail", that is it will have some error reporting, and be able to define the shape/format of the reported error.

/cc @andrewkroh

@kpollich kpollich added the Team:Ecosystem Label for the Packages Ecosystem team label Jul 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Ecosystem Label for the Packages Ecosystem team
Projects
None yet
Development

No branches or pull requests

2 participants