Successful Retries Never Output Exceptions

[dylanlive]

I recently moved from rspec-rerun to rspec-retry. I'm using Capybara & Selenium Rspec Tests. These can occasionally be flakey, which is why I am glad for this gem's existence.

However, I try to improve my tests to retry at last resort. Perhaps I'm not waiting long enough for a page to load, etc.

Unfortunately rspec-retry doesn't give me insight as to why the test failed in the first place. If it passes on attempt 3, I'll never know why 1 & 2 failed.

Would the community be opposed to including the exception to the verbose_retry output?

My fear is that this can make the test output rather unreadable at times in the log - especially if it's a big exception. Perhaps a better solution would be to include a snippet when the test finishes with stats about the exceptions rspec-retry caught.

I can fork, experiment and submit a PR, but curious on your thoughts before I do this.

[sbonebrake]

I'd like to go a step further and keep track of the exceptions and runtimes within the example and potentially let a custom logger decide what to do with them. I think this can be accomplished with the following:

def clear_exception
  if @exception
    finish(reporter)
    metadata[:prior_execution_results] ||= []
    metadata[:prior_execution_results] << metadata[:execution_result]
    metadata[:execution_result] = Example::ExecutionResult.new

    metadata[:prior_exceptions] ||= []
    metadata[:prior_exceptions] << @exception
    @exception = nil
    start(reporter)
  end
end

Obviously this could be cleaned up, renamed and split into a few methods, but I think it would work. Would this solve what you're trying to accomplish?