CI pipeline needs optimization

[huiyuxie]

I do think it is an issue that it takes too long to run a full round of tests in the CI pipeline. At least from the contributor/developer's point of view, it slows down the feedback loop when debugging.

One straightforward problem is that the test infrastructure is mixed with examples, which introduces some unnecessary output that slow down the tests. So probably tests should be separate from examples.

@ranocha @sloede

[sloede]

We concur, the long cycle time for running the full set of tests is annoying. Unfortunately, however, we currently have no better way of testing all features that is sustainable in terms of maintenance. Furthermore, we consider the examples to be part of the Trixi.jl implementation, thus they need to be tested as well.

If you have a concrete suggestion on how to improve testing times without losing coverage and without increasing the test complexity even further, we'd be happy to discuss it!

[huiyuxie]

If you have a concrete suggestion on how to improve testing times without losing coverage and without increasing the test complexity even further, we'd be happy to discuss it!

I can give it a try but it won’t start soon due to my schedule.

[ranocha]

We should check the results we get without splitting into tests with/without coverage. I would expect better behavior in Julia 1.10 than in the old versions that led to the splitting we have right now.

[JoshuaLampert]

This can probably be closed after #2254 is merged.

[huiyuxie]

Thanks for referring