pytest's own unit tests pass or fail depending on terminal width

[lanzz]

Some of pytest's own unit tests (ones dealing with asserting expected output, it seems) are sensitive to the terminal width and will pass when the width is 80 but fail for wider terminals (looks like substrings it expects to see on separate lines end up on the same line due to the extra space). I see that some tests have explicit patches for the COLUMNS environment variable that seems to deal with a similar issue, but the ones I see failing do not even attempt to ensure a repeatable testing environment.

Here are the affected tests I encountered:

testing/test_config.py::TestDebugOptions::test_debug_help
testing/test_helpconfig.py::test_help
testing/test_stepwise.py::test_sw_skip_help
testing/test_terminal.py::TestTerminal::test_verbose_skip_reason

I think this list might not be exhaustive, because sometimes the failure is caused by the expected string getting split between two lines, so it can still pass on some widths greater than 80, but not on others.

Not sure if this is considered a "bug" per se. Should I open a bug issue for it? I'm not even sure if it's a genuine issue, I looked for existing reports of this and didn't find any, so now I'm wondering if there's something off in the way I'm running the tests?

[RonnyPfannschmidt]

This is most likely a genuine overlook as Small with terminal are more and more uncommon

[bluetech]

@lanzz I haven't been able to reproduce; tried 24 columns and 190 columns. Can you share your COLUMNS that makes pytest fail?

[lanzz]

My typical terminal is about 200-column (those four tests above fail), but I've tried it with 80 (all pass) and 100 (the four above fail as well). If I run COLUMNS=80 tox -e py39 to override COLUMNS, everything passes regardless of terminal size. If I run without overriding COLUMNS and with --pdb, I can see that inside the failing tests, os.environ['COLUMNS'] indeed reflects the actual terminal width, it hasn't been patched.