Behat re-run fails on server not availalbe.

[kabalin]

Here is an example: https://travis-ci.org/github/kabalin/moodle-plugin-ci/jobs/710005386#L2424

Not sure how re-run works, is it triggered as part of same command, or we somehow stop servers by the time it tries again. Need to investigate. It is easy to replicate with build of moodle-plugin-ci itself for MOODLE_34_STABLE.

[kabalin]

Discovered while testing #14

[stronk7]

Wow, curious. Only 34_STABLE? And you did nothing to force the automatic rerun? Maybe we used to rerun automatically then? In any case, I think it's all the same script (run.php?) the one providing that. From memory behat does not provide that by default (but I may be wrong!).

[stronk7]

Ah, I see we are passing the auto-run option defaulting to 2. Ok, that explains why it's called. But I cannot imagine why it's failing on rerun, as far as it's the same script (run.php) the one handling that. So it should be the same (env-wise) for the original run and any rerun.

Uhm... sure there is an explanation...

[kabalin]

Wow, curious. Only 34_STABLE?

I did not test with any other version. I think it is just local_travis fixture incompatibility (no idea why, may be it will become clear on investigation), and it fails on chrome run only.

And you did nothing to force the automatic rerun?

It defaults to 2 as you found already. Thanks for input re same script use BTW.

May be I'll add some debugging output and see what happens.

[kabalin]

The issue is purely related to Moodle 3.4 used in test, that fails on behat test (using chrome profile) in the the latest moodle-plugin-ci. It probably not worth investigating why since 3.4 is not supported and those users who still wish to test can use default profile (firefox) that does not have this issue. Also, we have #15 which will allow passing explicit selenium images (assuming the issue is caused by selenium-chrome driver version in use).

Behat test forcing failure has been run using 3.9 stable to check re-run behaviour (in both Chrome and Firefox):
https://travis-ci.org/github/kabalin/moodle-plugin-ci/jobs/710787644
Re-run was is triggered correctly, there are no "The base URL is not a behat test site" errors shown.

Closing the issue.