You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the feature you'd like byexampleexpects the main prompt (PS1) to know when an example finished, recollects all the output before the prompt and compares it with the expected output for that example.
This reasonably assumes that all the output arrives before the prompt and nothing arrives after.
But not always is the case: an example may run in background and the interpreter may print the prompt before the example finishes. The result would be the last part of the output of the example may be missing and byexample will mark the example as failed but it also the missing output may appear at the begin of the next example output, unexpected for it, and byexample will mark it as failed too.
In #176 we added a -x-delayafterprompt to wait a fixed amount of time and try to read any output after the prompt but a fixed sleep isn't good for the performance and it is fragile anyways.
byexample could use the example's expected last lines as part of the synchronization.
Proposed solution
We could expect the prompt as usual and then check if the last expected line is in the output read. If isn't, try to do a second expect for that line and add any new output to the already read.
This extra check may fail due a timeout if:
the output never arrives and we are still out of sync
there is no output at all because the example is really printing something totally different
We cannot do much for 1 except increase its timeout. But that will go against 2: the second case is the happy case where we are not out of sync (but the example still fails) and increasing the timeout of the extra check will slowdown the execution.
We also needs to define which it means "last line". Which would be the minimum length for such thing to be useful for synchronization?.
Additional context (optional)
This issue probably a hard-dependency for #176 because evcxr, the rust interpreter redirects the examples' outputs (stdout and stderr) to the terminal in background. This seems to affect only when the example does println!()-like something so it is not going to be the most common case but certainly if the user hit the issue it will be confusing for him/her.
The text was updated successfully, but these errors were encountered:
Describe the feature you'd like
byexample
expects the main prompt (PS1) to know when an example finished, recollects all the output before the prompt and compares it with the expected output for that example.This reasonably assumes that all the output arrives before the prompt and nothing arrives after.
But not always is the case: an example may run in background and the interpreter may print the prompt before the example finishes. The result would be the last part of the output of the example may be missing and
byexample
will mark the example as failed but it also the missing output may appear at the begin of the next example output, unexpected for it, andbyexample
will mark it as failed too.In #176 we added a
-x-delayafterprompt
to wait a fixed amount of time and try to read any output after the prompt but a fixed sleep isn't good for the performance and it is fragile anyways.byexample
could use the example's expected last lines as part of the synchronization.Proposed solution
We could expect the prompt as usual and then check if the last expected line is in the output read. If isn't, try to do a second expect for that line and add any new output to the already read.
This extra check may fail due a timeout if:
We cannot do much for 1 except increase its timeout. But that will go against 2: the second case is the happy case where we are not out of sync (but the example still fails) and increasing the timeout of the extra check will slowdown the execution.
We also needs to define which it means "last line". Which would be the minimum length for such thing to be useful for synchronization?.
Additional context (optional)
This issue probably a hard-dependency for #176 because
evcxr
, therust
interpreter redirects the examples' outputs (stdout
andstderr
) to the terminal in background. This seems to affect only when the example doesprintln!()
-like something so it is not going to be the most common case but certainly if the user hit the issue it will be confusing for him/her.The text was updated successfully, but these errors were encountered: