Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Print verbose: 2 execution summary during the run #5086

Open
gruhn opened this issue Jun 28, 2024 · 1 comment
Open

Print verbose: 2 execution summary during the run #5086

gruhn opened this issue Jun 28, 2024 · 1 comment

Comments

@gruhn
Copy link
Contributor

gruhn commented Jun 28, 2024

🚀 Feature Request

How much trouble would it be to print the verbose: 2 execution summary during the test run, instead of printing it in batch at the end? Using the example from the documentation:

Execution summary:
× [[2147483632,-1868311174,2,-14,-1309756521,948824737,2147483631,1352827217]]
. √ [[]]
. × [[-1309756521,948824737,2147483631,1352827217]]
. . √ [[2147483631,1352827217]]
. . × [[948824737,2147483631,1352827217]]
. . . √ [[2147483631,1352827217]]
. . . √ [[0,2147483631,1352827217]]
. . . × [[474412369,2147483631,1352827217]]
. . . . √ [[2147483631,1352827217]]
. . . . × [[237206185,2147483631,1352827217]]
. . . . . √ [[2147483631,1352827217]]
. . . . . √ [[118603093,2147483631,1352827217]]
. . . . . × [[177904639,2147483631,1352827217]]
...
. . . . . ... . . × [[2,1000000000]]
. . . . . ... . . . √ [[1000000000]]
. . . . . ... . . . √ [[1,1000000000]]
. . . . . ... . . . √ [[2]]
. . . . . ... . . . √ [[2,999999999]]

Presumably, each of these lines can (in principle) be printed as soon as the corresponding test has failed/succeeded. The level of indentation is also known, right? I know there is some effort going on with workers to run tests in parallel? Would that be a problem here?

Motivation

This is just a minor improvement from my view. And I guess for fast running tests, this would make no difference. I have some fairly long running (model-based) tests though. There it would be nice to have more lively feedback. In particular during shrinking. Sometimes shrinking takes many iterations. I noticed that team members, who are not so familiar with the tests, get the impression that the tests are "stuck". Recently, I also gave a presentation on our testing setup with a live demo. I started the tests, then bridged 2 minutes with other talking points, and finally explained the idea of shrinking, once the execution summary was available. No problem but it also would have been slightly nicer to explain the shrinking process as it was happening.

@dubzzz
Copy link
Owner

dubzzz commented Jul 5, 2024

I might be doable at some point today by hook something around a property. The more I think of such problems the more I feel I should offer a better set of APIs to offer users with ways to extend fast-check with custom reporters, custom executors...

Not planning to do anything for this topic in the coming releases. But I keep it on the side as a great idea to have in mind for later evolution (including providing an API o extend fast-check in a way simpler way).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants