Memory

[henrikbjorn]

If you have a lot of iterations memory skyrockets when used. Maybe results could be temp saved in a file somewhere when doing the calculations?

[staabm]

How many iterations do you use?

[henrikbjorn]

1.000.000

[staabm]

Do you see an obvious benefit when using that much iterations? We use most of the time 1000-10000 iterations which results in rather stable results...

[henrikbjorn]

None other that i could and the output have a lot of space for the iterations number :)

[polyfractal]

Heya, sorry for the delay. I'd tend to agree with @staabm, there really isn't any benefit to more iterations once the timings start to stabilize.

That said, memory usage could be improved -- the fact that it bloats up is not terribly surprising. Athletic stores everything in a hash which will just keep growing.

A new SuiteRunner and ClassRunner could be implemented, which stream results to disk as they arrive. Then when it comes to publishing, instead of sending the entire result set, it could incrementally call publish() with fragments...effectively giving a streaming interface without introducing the complexity of a generator or passing around file handles.

PRs welcome :)